Data Engineer

Category: Information Technology Location: San Francisco Job ID: 17424 Date Posted: April 26th, 2019

Title: Data Engineer

Location: San Francisco

Term: 6mo+; Start: ASAP


Our well-known financial services client in San Francisco is looking for 2 data engineers to join their team and demonstrate their skills in improving the client’s data pipeline that powers their recommendation systems, enterprise tools, and data warehousing.


Key Skills

  • Scala, Kafka, Spark, Akka and Google Cloud


Must Haves: Project 1- ENGINEER 1

  • Will assist with a migration project
  • This is more operational not data warehousing
  • This is BI data aggregation/Bi Information
  • Docker container experience is a must
  • Experience with Google Cloud Technology (GCS), Big Query a must
  • Strong Spark/Scala hands-on experience
  • Pretty good understanding of ACL (Access Control List) to make sure the whole deployment process is secure

Must Haves: Project 2 – ENGINEER 2

  • Will assist with a deployment project
  • Docker container experience is a must
  • Will be running with Scala project
  • Experience with Google Cloud Technology (GCS), Big Query a must
  • Strong Java/Scala hands-on experience, minimum 5 years exp.

What You’ll Do

  • Work with analysts and data scientists to define processes and standards to inform system design, transform their needs into streaming or batch processing
  • Design infrastructure and systems to scale easily as data ingest grows
  • Implement new data pipeline features with verified high quality from unit test coverage and production monitoring
  • Focus on data quality! Detect data/analytics quality issues and implement bug fixes and data validation for prevention
  • Help understand our day to day operations for continuous improvement of production systems

What You’ll Have

  • 5+ years experience with Big Data technologies
  • Experience with Scala/Java, Spark, Kafka, or demonstrated ability to pick up new technology quickly
  • Fundamental knowledge about databases and strong SQL skills
  • Familiar with Google Cloud ecosystem such as BigQuery, GCS, DataProc, etc
  • Enjoys working collaboratively; CK’s values include empathy and helpfulness
  • Able to estimate and meet deadlines
  • Excellent verbal and written communication skills

Nice to have

  • Experience scaling data throughput or building low latency streaming pipelines
  • Experience solving for data quality; cloud technologies experience

Apply Now