Data Engineer - SQL/Python

Role Summary :

We are looking for an experienced Data Engineer with 6 - 8 years of hands-on experience in designing, building, and managing scalable data pipelines, data platforms, and analytics-ready data assets. The ideal candidate should have strong expertise in data engineering, cloud platforms, ETL/ELT pipelines, data lakehouse architecture, data quality, and production-grade data solutions.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines for structured, semi-structured, and unstructured data.

- Build and manage data ingestion frameworks from ERP, CRM, SCADA, IoT, Historian, APIs, databases, and external data sources.

- Develop ETL/ELT workflows for data transformation, cleansing, validation, and enrichment.

- Implement data lake, data warehouse, and lakehouse architectures to support analytics, AI/ML, and reporting use cases.

- Ensure data quality, reliability, lineage, metadata management, and governance across data platforms.

- Collaborate with Data Scientists, ML Engineers, Product Managers, Business Teams, and BI Developers to deliver data solutions.

- Optimize data pipelines for performance, scalability, cost, and reliability.

- Build reusable data assets, data marts, and curated datasets for business intelligence and AI use cases.

- Support deployment, monitoring, troubleshooting, and improvement of production data pipelines.

Required Skills :

- Strong hands-on experience in SQL, Python, PySpark, and data pipeline development.

- Experience with cloud data platforms such as Azure Data Factory, Azure Synapse, Databricks, ADLS, AWS Glue, Redshift, S3, BigQuery, Cloud Function, Data Flow, or Snowflake.

- Good understanding of data lake, data warehouse, lakehouse, data modeling, and medallion architecture.

- Experience in batch and real-time/streaming data processing.

- Knowledge of Apache Spark, Kafka, Airflow, dbt, Delta Lake, or similar technologies.

- Experience with data quality, validation, metadata, lineage, and governance frameworks.

- Exposure to CI/CD, Git, DevOps practices, and production deployment of data pipelines.

- Understanding of APIs, file formats, databases, and enterprise integration patterns.