AWS And Azure Data Engineer
Posted
Summary
Data engineer at Tata Consultancy Services in Bangalore who designs, builds, and maintains data transformation pipelines using AWS services (Lambda, Step Functions), PySpark, and Databricks in Python, while managing large datasets and SQL/NoSQL databases like PostgreSQL or MongoDB.
- Design, build, and maintain data transformation pipelines using AWS services (Lambda Functions, Step Functions .. ) in Python.
- Design, build, and maintain data transformation pipelines using PySpark and Databricks. Implement and maintain complex calculation engine.
- Manage effeciently large dataset.
- Manage SQL database - Postgresql or MongoDB.
- Implement and maintain complex data calculation formula.
- Open to collaborate with Software Engineers.
- Participate in code reviews, design discussions, and software planning.
- Contribute to automation, testing, and deployment processes.