Data Engineer
Summary
QBrainX is hiring a Data Engineer in Coimbatore to design, build, and optimize scalable ETL/ELT data pipelines and analytics-ready datasets. Day-to-day work centers on Python, SQL, PySpark, Databricks, AWS, Azure, and Snowflake, with CDC/SCD patterns and BI support via Power BI/QuickSight.
Data Engineer
- Design and develop scalable ETL/ELT data pipelines using Python, PySpark, and Databricks.
- Work with AWS, Azure, Snowflake, and data warehousing solutions.
- Build reliable, analytics-ready datasets and ensure data quality and performance.
Requirements
Key Roles & Responsibilities
- Develop and optimize ETL pipelines and data ingestion workflows.
- Implement CDC, SCD Type 1/2, and Medallion Architecture.
- Perform data validation, transformation, and quality checks.
- Develop data models and support BI reporting using Power BI/QuickSight.
- Work with AWS Glue, S3, Azure Data Factory, Airflow, and Snowflake.
- Collaborate with stakeholders to understand requirements and deliver data solutions.
Primary Skills:
- Python, SQL, PySpark
- Databricks & Apache Spark
- ETL/ELT & Data Pipelines
- AWS (S3, Glue)
- Snowflake
Secondary Skills:
- Azure Data Factory
- Apache Airflow, dbt, Fivetran
- Data Modeling & Data Warehousing
- CDC & SCD Type 1/2
- Power BI & Amazon QuickSight
- Delta Lake, Terraform & CI/CD