Backend Data Engineer
Posted Updated 4
views
Summary
Data engineer on a web team supporting semiconductor chip verification: designs and maintains scalable pipelines that ingest, parse, and aggregate data from EDA tools and MongoDB, manages metrics databases, and automates reporting and alerting for chip tapeout runs. Core stack is Python plus Spark, Airflow, Kafka, and ClickHouse.
Job Description
As a Data Engineer in our web development team supporting chip verification, you will design, implement, and maintain data pipelines that manage data and metrics from physical verification outputs in the semiconductor design process. You will work closely with web developers and DevOps teams to automate collection, processing, and visualization of run results, empowering smarter decision-making and accelerating our chip tapeouts.
Responsibilities
Develop scalable data pipelines to ingest, parse, and aggregate data from EDA tools and Mongo databases.
Collaborate with web teams to integrate processed data into dashboards or web-based reporting tools.
Design and manage databases for tracking job metrics, violation summaries, and run metadata.
Automate reporting and alerting systems for regressions, failure trends, and job status.
Optimize storage and retrieval of large log files, job results, and associated metadata.
Maintain and improve log parsing scripts.
Requirements
Bachelor’s degree or higher in Computer Science, Electrical Engineering, or related field.
Strong programming skills in Python for data processing.
Deep hands-on experience in Big Data Frameworks, e.g. Apache Spark, Airflow, Kafka, ClickHouse.
Other Preferred Skills
Experience with containerization (Docker/Podman), monitoring tools (Prometheus, Grafana, ELK Stack), and CI/CD pipelines.
Ability to quickly prototype and iterate on solutions.
Responsibilities
Develop scalable data pipelines to ingest, parse, and aggregate data from EDA tools and Mongo databases.
Collaborate with web teams to integrate processed data into dashboards or web-based reporting tools.
Design and manage databases for tracking job metrics, violation summaries, and run metadata.
Automate reporting and alerting systems for regressions, failure trends, and job status.
Optimize storage and retrieval of large log files, job results, and associated metadata.
Maintain and improve log parsing scripts.
Requirements
Bachelor’s degree or higher in Computer Science, Electrical Engineering, or related field.
Strong programming skills in Python for data processing.
Deep hands-on experience in Big Data Frameworks, e.g. Apache Spark, Airflow, Kafka, ClickHouse.
Other Preferred Skills
Experience with containerization (Docker/Podman), monitoring tools (Prometheus, Grafana, ELK Stack), and CI/CD pipelines.
Ability to quickly prototype and iterate on solutions.