Principal Data Engineer
Posted
Description
You will design and implement scalable cloud-based data management and reporting platform components. You will build data pipelines and integration solutions, develop metrics frameworks and reporting models, and craft CI/CD pipelines for data platforms. You will troubleshoot complex pipeline issues, optimize for performance and cost, document standards, and mentor less-experienced engineers while partnering with product owners and analysts.
Responsibilities
- Design and develop scalable database, data-pipeline, and data integration solutions
- Build and refine CI/CD pipelines for data platforms
- Develop reusable metrics frameworks and reporting models
- Architect scalable data consumption patterns from cloud data lakes
- Troubleshoot and resolve complex data pipeline and platform issues
- Optimize cloud data platforms for performance, reliability, and cost
- Define and document technical standards and best practices
- Support SIT and UAT planning, release coordination, and production readiness
- Partner with product owners, analysts, and engineers
- Mentor less-experienced engineers
- Advise senior leadership on data infrastructure
Requirements
- Bachelor's degree in Computer Science, Engineering, IT, or related field and 5 years of experience building financial and Medicare data warehouse applications using data modeling and ETL, or Master's degree and 3 years
- Expertise with Python, Apache NiFi, Snowflake, PostgreSQL, AWS
- Expertise with IBM DataStage, UNIX Shell scripting, Oracle, DB2, Netezza
- Expertise with Apache Airflow, TWS, Control-M, DAGs
- Expertise with advanced SQL, CTEs, window functions, and Change Data Capture
Skills
Sign in to read and join the discussion.