Data Engineer
Posted Updated 5
views
Design and implement robust data pipelines that ensure seamless data flow from various sources to data storage solutions Optimize data ingestion and transformation processes to improve performance and reduce latency for real-time analytics Collaborate with data scientists and analysts to understand data requirements and translate them into technical specifications Ensure data quality and integrity by implementing automated data validation and monitoring systems
Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
- 4+ years of experience in Data Engineering, Data Integration, and Cloud Data Platforms.
- Strong expertise in Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Azure Synapse Analytics, and Storage Accounts.
- Hands-on experience with Databricks Medallion Architecture and Delta Lake implementation.
- Extensive experience building solutions using Python and PySpark.
- Proven experience integrating Microsoft Dynamics 365, Salesforce, Oracle Fusion, SharePoint, REST APIs, SFTP, and database systems.
- Strong understanding of ETL/ELT frameworks, data modeling, and data warehousing concepts.
- Experience with Self-Hosted Integration Runtime (SHIR) configuration and maintenance.
- Strong SQL development and database optimization skills.
- Excellent troubleshooting, analytical, and problem-solving abilities.
- Strong communication, stakeholder management, and documentation skills.