Senior Data Engineer
Posted Updated
Description
Responsibilities:
- Design, build, and maintain efficient and scalable data pipelines that collect, transform, and load data.
- Develop the logical data model and physical data models, including data warehouse designs.
- Monitoring and Maintenance: Implement monitoring and alerting systems to ensure data pipeline reliability, and perform routine maintenance tasks.
- Work with upstream and downstream teams to ensure end-to-end execution of the data pipelines.
- Help set up processes and streamline workflows to help in the expansion of the team and collaborate with other teams.
- Optimize data processing and storage solutions for efficiency and scalability, working with large volumes of data.
Requirements:
- Should have worked on the Data Platform.
- Should have 2-5 years of experience working on ETL pipelines.
- Strong programming skills in Python for data manipulation and transformation.
- Should have in-depth knowledge of a NoSQL or relational database.
- Should be able to model data in them.
- Should have experience with workflow management tools like Apache Airflow.
- Should help the team with all phases of development, including design, implementation, and operation of production systems.
- Should know what good code looks like and be able to write the same.
- Should have a solid understanding of object-oriented design and concepts.
- Should be able to deal well with ambiguous/undefined problems.
- Understanding of data security and privacy principles.
Skills
Sign in to read and join the discussion.