Manufacturing Data Engineer
Overview
Build the data foundation powering manufacturing intelligence in this hands-on contract opportunity. As a Manufacturing Data Engineer, you will design, develop, and maintain reliable data pipelines and models that bring together manufacturing data from on-premises databases, SAP MES, SAP HANA, test and diagnostic systems, and related sources. You will transform this data into trusted lakehouse tables that support manufacturing KPIs, dashboards, analytics, and AI use cases.
Approximately 80% of the role will focus on hands-on data engineering, with the remaining time dedicated to keeping pipelines deployable, observable, governed, and cost efficient. You will collaborate closely with data engineering, data analytics, manufacturing operations, infrastructure, and Databricks platform teams while supporting the migration of existing Redshift models and pipelines to Databricks.
Required Skills
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience
- 6+ years of professional data engineering experience
- Strong SQL and Python skills, including PySpark
- Hands-on production experience with Databricks, including:
- Delta Lake
- Unity Catalog permissions and governance
- Databricks Workflows or jobs
- Spark performance tuning
- Production experience with dbt, including incremental models and testing
- Strong understanding of:
- Dimensional data modeling
- Medallion architecture
- Change data capture (CDC)
- Incremental data loading
- Experience building batch and streaming ingestion pipelines using technologies such as Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect, or Structured Streaming
- Experience with incremental models, MERGE and upsert patterns, SCD Type 2, and late or out-of-order records
- Experience deploying data code through Git-based workflows and CI/CD, such as GitHub Actions or Databricks Asset Bundles
- Working knowledge of AWS data services, including S3, IAM, DMS, or Kinesis
- Experience with infrastructure as code, such as Terraform
- Ability to process large and semi-structured datasets, including JSON test payloads
- Experience with data quality, pipeline monitoring, freshness SLAs, and alerting
- Strong communication and documentation skills, with the ability to partner with technical teams, analysts, and manufacturing stakeholders
Nice to Have Skills
- Databricks Certified Data Engineer Associate or Professional certification
- AWS certification, such as Solutions Architect Associate
- Experience preparing data for Databricks AI/BI dashboards, Genie, machine learning, or LLM use cases
- Background working with manufacturing or industrial data, including quality, yield, traceability, station results, work orders, or MES
- Experience migrating Redshift models and pipelines to Databricks
- Familiarity with SAP MES, SAP HANA, AWS DMS, SAP SLT, Airflow, or Kinesis
- Understanding of data governance and compliance as they relate to data pipelines
- Experience managing Unity Catalog catalogs, schemas, volumes, grants, service principals, secret scopes, and OAuth machine-to-machine authentication
- Experience monitoring cloud data platform usage, job health, and compute costs
Preferred Education and Experience
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related discipline, or equivalent practical experience
- 6+ years of data engineering experience in production environments
- Experience designing and supporting enterprise-scale data platforms, lakehouses, and analytics systems
- Experience working with manufacturing, operational, or industrial data is preferred
Other Requirements
- Contract duration: 12 months
- Work arrangement: Fully onsite in Foster City, California
- W2 engagement only
- Ability to support production data pipelines, operational monitoring, incident response, and root cause analysis
- Ability to document pipelines, data lineage, data contracts, table definitions, runbooks, and operational procedures
- Ability to collaborate across data engineering, analytics, manufacturing operations, infrastructure, and platform teams
