AWS Data Engineer
Summary
Build and optimize AWS-based ETL/ELT pipelines using Redshift, S3, Glue, and Lambda to move and transform data for analytics and AI/ML workloads.
Key Responsibilities
- Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
- Build and maintain robust data pipelines using AWS cloud services.
- Partner with Business Intelligence (BI), AI/ML, and Infrastructure teams to deliver reliable and scalable data platform solutions.
- Develop efficient SQL queries and Python scripts for data processing, automation, and analytics.
- Create and maintain technical documentation, including solution designs, data flow diagrams, and operational guides.
- Develop comprehensive unit tests and perform code reviews to ensure high-quality, reliable, and maintainable code.
- Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
- Work on multiple projects simultaneously while managing priorities in a fast-paced environment.
- Follow coding standards, best practices, and data governance guidelines.
Required Skills & Qualifications
- 3–5 years of hands-on experience in Data Engineering.
- Strong experience in designing and developing ETL/ELT pipelines.
- Proficiency in SQL and Python programming.
- Hands-on experience with AWS services, including:
- Amazon Redshift
- AWS Lambda
- Amazon S3
- AWS Glue (preferred)
- AWS IAM
- Amazon CloudWatch
- Experience with workflow orchestration tools such as Apache Airflow (preferred).
- Understanding of data warehousing concepts and dimensional data modeling.
- Knowledge of version control systems such as Git.
- Experience writing unit tests and following software development best practices.
- Strong analytical, problem-solving, and debugging skills.
- Excellent communication and documentation skills.
- Ability to manage multiple tasks and work effectively in a collaborative environment.
Preferred Skills
- Experience with CI/CD pipelines and DevOps practices.
- Knowledge of Spark, PySpark, or Databricks is an advantage.
- Exposure to AI/ML data pipelines and feature engineering.
- Familiarity with Agile/Scrum development methodologies.
- AWS Certification (Developer Associate or Data Engineer Associate) is a plus.
Key Competencies
- AWS Data Engineering
- ETL/ELT Development
- SQL & Python
- Amazon Redshift
- AWS Lambda
- Amazon S3
- Data Warehousing
- Unit Testing
- Technical Documentation
- Problem Solving
- Team Collaboration
- Multitasking
- Agile Development
Educational Qualification
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- Relevant AWS certifications will be an added advantage.
