Senior Data Engineer - Python/ETL

Key Responsibilities:

- Design, develop, and maintain real-time and batch data ingestion pipelines.

- Build scalable ETL workflows using PySpark, Python, and Airflow.

- Develop and optimize enterprise-grade data processing solutions.

- Design and manage data lakes, data warehouses, and big data architectures.

- Work with structured and unstructured datasets to deliver reliable data solutions.

- Optimize data pipelines for scalability, performance, and reliability.

- Collaborate with cross-functional teams to translate business requirements into technical solutions.

- Monitor, troubleshoot, and improve production data pipelines.

- Implement best practices for data quality, governance, security, and version control.

- Contribute to CI/CD, automation, and cloud-native data engineering initiatives.

Required Qualifications:

- Bachelor's degree in Computer Science, Information Technology, or a related field.

- 48 years of professional experience in Data Engineering or related fields.

- Strong hands-on experience with Python, PySpark, SQL, and Apache Airflow.

- Experience building ETL pipelines and processing large-scale datasets.

- Knowledge of real-time data streaming and workflow orchestration.

- Experience with SQL and NoSQL databases such as Cassandra, MongoDB, or HBase.

- Strong understanding of Data Warehouse concepts.

- Hands-on experience with AWS services including EMR, Redshift, Glue, Lambda, Kinesis, S3, IAM, and CloudWatch.

- Experience with Hadoop ecosystem tools including Hive and Sqoop.

- Proficiency with Git/GitHub and software development best practices.