Data Engineer (Azure) - Remote, Latin America
Posted Updated
Description
Bluelight Consulting LLC is seeking an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) within Azure Synapse Analytics. You will collaborate with data scientists, analysts, and stakeholders to ensure accurate, accessible analytical data and scalable pipelines.
The role emphasizes data modeling, data quality, performance tuning, and governance within a fast-paced consulting environment that values continuous learning and professional growth.
ETL data engineering: develop and maintain ETL pipelines using Python (PySpark) in Azure Synapse notebooks or pipelines. Data warehousing: design and build scalable data structures in an MPP SQL pool. Source data extraction: pull data from REST APIs, SQL tables, and CSV files. Azure Synapse: optimize notebooks/pipelines for performance and scalability. Data fabric concepts: contribute to data lakes, lakehouses, delta lakes, and cataloging. Data modeling: collaborate with data architects to align with business needs. Data quality: implement validation and quality checks. Performance tuning: resolve bottlenecks to meet SLAs. Monitoring and troubleshooting: monitor ETL jobs and diagnose issues. Documentation: maintain comprehensive data flow and transformation docs. Collaboration: work with cross-functional teams on data initiatives. Security/compliance: ensure data governance and privacy standards. Bachelor’s degree in Computer Science, IT, or related field; relevant data engineering certifications are a plus. Proven ETL experience using Python (PySpark) for data extraction, transformation, and loading from REST APIs, SQL tables, and CSVs. Strong knowledge of Azure Synapse resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault. Proficient in writing complex SQL queries (SparkSQL/MS SQL) for data extraction, transformation, and loading.Skills
Sign in to read and join the discussion.