Senior Data Engineer
Posted Updated
Description
Job Summary
We are looking for a highly motivated Senior Data Engineer to join our team. The successful candidate who knows data bricks/SQL/PySpark/Power BI/ADF. The Data Engineer will be responsible for designing, building, and maintaining the data infrastructure that supports our data bricks. The ideal candidate will have a strong background in data management and be well-versed in the latest data engineering technologies.
Duties and Responsibilities - Design and implement data pipelines to support the data bricks/data lakehouse
- Test and validate data integrity in the lakehouse
- Develop data pipeline for data extractions (ADF)
- Optimize data query/pipeline performance
- Good understanding of Financial service domains and understanding of data
- Stay current with the latest data engineering technologies and best practices
- Developing and managing data bricks/ADF/Power BI/Microsoft Fabric
- Strong experience in data management and data engineering and data modelling
- Proficient in SQL and one or more programming languages such as Python or Pyspark
- Experience with cloud-based data lakehouse technologies such as AWS Lake Formation, Azure Data Lake Storage, or Google BigQuery
- Strong problem-solving and analytical skills
- Excellent communication and collaboration skills
- Bachelor's degree in computer science, engineering, or a related field