Big Data Developer
Posted Updated
Description
Job Description:
- Design, develop, and maintain data pipelines and ETL workflows using Databricks, Spark, and ADF.
- Implement data lakehouse architecture and optimize data storage and retrieval for performance and cost efficiency.
- Write efficient, reusable, and scalable Python code for data processing and automation tasks.
- Orchestrate complex workflows using Azure Data Factory and integrate with other Azure services.
- Develop and maintain Power BI dashboards and reports, ensuring accurate and timely insights for stakeholders.
- Ensure data quality, security, and governance across all data solutions.
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data products.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
Stay updated on emerging technologies and best practices in cloud data engineering. - Experience with data lakehouse architecture, Delta Lake, and data modeling.
Knowledge of CI/CD pipelines, DevOps practices, and version control (Git).
Skills
Sign in to read and join the discussion.