Software Engineer (3 openings)

Category: Information Technology Location: San Francisco Job ID: 17775 Date Posted: June 26th, 2019

Title: Data Engineer (3 openings)

Terms: 6 month+ 

Location: San Francisco

We're seeking a software engineer for our creative social media client in San Francisco. You'll help migrate their pipelines for extracting core signals/features in almost every product feature.

Key Skills:

  • MapReduce
  • Hive
  • Python
  • Java
  • Cascading is a plus

What You'll Do: 

  • Understand and analyze some of our large, monolithic data pipelines
  • Break apart these pipelines into smaller modularized jobs written in our new MapReduce based framework.
  • Test and tune the performance of these new jobs.
  • Verify the correctness of the migrations, which may involve writing new data checking tools.
  • Migrate consumers of the legacy dataset to read from the newly built signals.

What You'll Have:    

  • 2-4 years of experience
  • Big Data ETL Jobs; a solid understanding of how MapReduce works and how to write & tune jobs for it
  • Hive, Java, Python

Apply Now