Data Engineer
Posted Updated
Description
Machine Learning Technologies LLC in Plano, TX is seeking a Data Engineer to design, develop, and optimize data pipelines using Databricks and Azure Data Factory. You will build ETL/ELT processes, write SQL queries, and work with PySpark to handle large-scale data.
The role focuses on data quality, reliability, and collaboration with analytics teams to deliver robust data assets for reporting and analytics in a cloud-based environment.
Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF). Work with Databricks for data processing and transformation. Write and optimize SQL queries for data extraction and transformation. Utilize PySpark or Python for large-scale data processing tasks. Integrate data from multiple sources into data warehouses or data lakes. Monitor, troubleshoot, and optimize data pipelines. Collaborate with teams to ensure data quality and reliability. ETL/ELT concepts and data integration expertise. SQL proficiency for data extraction and transformation. Basic Python or PySpark for large-scale data processing.Skills
Sign in to read and join the discussion.