Data Engineer
Summary
Mid-level Data Engineer on the Unidata data team, building and deploying data pipelines, keeping them running within SLAs, and improving data quality, governance, and documentation. Core stack is Python and SQL with cloud platforms (AWS preferred) and orchestration tools like Dagster or Airflow.
The Data Engineer plays a key role in supporting the Data Team’s vision of building a data ecosystem that delivers accurate, timely, and well-structured data to enable Unidata’s cycle of data asset enrichment for broader enterprise impact. This role is primarily technical but also collaborative — contributing to the delivery of high-quality data pipelines and infrastructure while supporting Unidata’s data foundations through good engineering practices, documentation, and participation in data governance and observability initiatives.
The Data Engineer will work closely with senior and lead data engineers, data scientists, product leads, and other stakeholders as they enable use cases, advance operational efficiency and effectiveness, and champion data-driven culture within and beyond Unidata.
Key Responsibilities
Contribute to the development and successful deployment of new data pipelines that support priority use cases and product features, under the guidance of senior and lead data engineers.
Monitor and maintain uptime of assigned data pipelines, addressing straightforward data quality issues within agreed turnaround times and escalating complex issues when needed.
Respond to data requests and incidents within defined SLAs and committed deadlines.
Ensure successful pipeline execution rate across assigned workflows by following operational runbooks, tests, and deployment processes.
Maintain and update technical documentation and basic test coverage for owned or assigned pipelines.
Apply and follow industry best practices in data engineering, including CI/CD, data quality frameworks, governance, and monitoring.
Participate in cross-functional projects, technical presentations, and knowledge-sharing activities (e.g., brownbag sessions, R&D initiatives).
Proactively communicate and flag issues, risks, and blockers to the team.
Continuously build proficiency in data engineering tools, patterns, and infrastructure relevant to the team’s platform.
Deliver incremental enhancements to data pipelines or infrastructure that improve reliability, performance, or usability.
Minimum Qualifications
Bachelor’s degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field, or equivalent practical experience.
2–3 years of experience working in projects as data engineer, analytics engineer, software engineer, or a related technical role.
Solid understanding of Python (including writing scripts, basic data processing, and working with common libraries) and SQL (including querying, filtering, aggregations, and basic joins).
Experience working with relational databases, data warehousing, and ETL/ELT pipelines.
Experience working with at least any cloud platform: AWS (most preferred), GCP, Azure.
Experience working with at least one data orchestration tool: Dagster, Airflow, etc.
Ability to write clear, maintainable code and technical documentation.
Strong willingness to learn new tools and frameworks in the data ecosystem.
Good to have
Experience using Git for version control and collaborative development.
Exposure or experience working with dbt or similar tools.
Experience with containerization tools such as Docker.
Exposure to data quality tools (e.g., Pandera or similar libraries) and data validation/testing practices.
Familiarity with Terraform or similar tools for Infrastructure as Code.
Experience with schema management and migration tools like Alembic, atlasgo, or similar tools.