Data Engineer

• Design, develop, and maintain scalable batch and real-time data pipelines.
• Build data ingestion frameworks to process structured, semi-structured, and unstructured data from diverse sources.
• Develop and optimize data transformation workflows using Python, PySpark, and Databricks.
• Implement and manage ETL/ELT processes for large-scale analytics and reporting.
• Design and maintain Data Lakes, Lakehouse architectures, and Data Warehouses.
• Collaborate with Data Scientists, Business Analysts, Architects, and stakeholders to understand business requirements and translate them into technical solutions.
• Ensure data quality, integrity, security, and governance across data platforms.
• Optimize Spark jobs, SQL queries, and cloud resources for performance and cost efficiency.
• Implement monitoring, logging, and alerting mechanisms for data pipelines.
• Contribute to architecture discussions, code reviews, and best practice implementation.

Required Skills & Experience

• 7-10 years of experience in Data Engineering, Big Data, or Cloud Data Platforms.
• Strong programming experience in Python.
• Hands-on expertise in PySpark and Apache Spark.
• Extensive experience with Databricks, Delta Lake, and Lakehouse architecture.
• Strong proficiency in SQL and database performance optimization.
• Experience building enterprise-scale ETL/ELT pipelines.
• Strong hands-on experience with AWS Cloud Platform and cloud-native data engineering solutions.
• Experience with data modeling, schema design, and data warehousing concepts.
• Experience with workflow orchestration tools such as Apache Airflow or equivalent.
• Strong understanding of software engineering principles, version control, and testing methodologies.
• Experience using Git.