Hiring! Senior Data Engineer
Summary
Build and maintain scalable data pipelines and analytical datasets on AWS for a risk-focused datamart using PySpark, Airflow, and SQL.
Role Purpose
Design, develop, and support data integration, transformation, and analytical data products for a risk-focused datamart. The role will be responsible for onboarding new data domains, enhancing existing analytical assets, and enabling scalable, governed data consumption across reporting, analytics, and decision-support use cases.
Key Responsibilities
- Develop and maintain data ingestion, transformation, and data quality pipelines.
- Analyze and enhance existing data marts, analytical datasets, and reporting assets to improve data usability, consistency, and scalability.
- Build reusable data products, views, and curated datasets for enterprise consumption.
- Implement reconciliation, monitoring, audit, and data quality controls.
- Optimize data processing workflows and query performance.
- Support data platform modernization initiatives, including migration and rationalization of existing analytical assets.
- Collaborate with business analysts, architects, and stakeholders throughout solution delivery.
- Support testing, deployment, operationalization, and production support activities.
Expected Outputs
- Schema definitions for creating databases & tables on AWS S3 lakehouse
- Airflow DAGs
- Pyspark scripts for data ingestion
Required Skills
- ETL/ELT Development via Python, Pyspark, and SQL
- Data Orchestration via MWAA/Airflow DAGs
- AWS services
- Data Pipeline Development
- Data Quality and Reconciliation Frameworks
- Data Modeling Fundamentals
- Git and CI/CD Practices
Experience
- Experience building enterprise data pipelines and analytical data products.
- Experience working with data warehouses, data lakes, data marts, and reporting platforms.
- Experience reviewing, enhancing, and integrating existing enterprise analytical assets into a standardized architecture