Software Engineer (3 openings)
Title: Data Engineer (3 openings)
Terms: 6 month+
Location: San Francisco
We're seeking a software engineer for our creative social media client in San Francisco. You'll help migrate their pipelines for extracting core signals/features in almost every product feature.
- Cascading is a plus
What You'll Do:
- Understand and analyze some of our large, monolithic data pipelines
- Break apart these pipelines into smaller modularized jobs written in our new MapReduce based framework.
- Test and tune the performance of these new jobs.
- Verify the correctness of the migrations, which may involve writing new data checking tools.
- Migrate consumers of the legacy dataset to read from the newly built signals.
What You'll Have:
- 2-4 years of experience
- Big Data ETL Jobs; a solid understanding of how MapReduce works and how to write & tune jobs for it
- Hive, Java, Python