IT Data Engineer

Job Purpose

The IT Data Engineer is responsible for designing, developing, and maintaining scalable data solutions that enable the organization to collect, process, store, and analyze data effectively. The role supports business intelligence, analytics, reporting, and data-driven decision-making by ensuring that high-quality, reliable, and secure data is available across the enterprise.

Key Responsibilities

Data Architecture & Engineering

• Design, build, and maintain data pipelines and integration solutions.
• Develop and optimize ETL/ELT processes to ingest, transform, and load data from multiple sources.
• Design and maintain data warehouses, data lakes, and lakehouse architectures.
• Implement and manage data models that support reporting and analytics requirements.
• Ensure scalability, performance, and reliability of data infrastructure.

Data Integration & Management

• Integrate data from internal and external systems, applications, and databases.
• Monitor and resolve data quality issues.
• Maintain data dictionaries, metadata, and documentation.
• Implement data governance and data management best practices.

Cloud & Platform Administration

• Support cloud-based data platforms and services.
• Configure and maintain storage, compute, and data processing environments.
• Optimize cloud resources for performance and cost efficiency.
• Implement monitoring and alerting for critical data systems.

Security & Compliance

• Ensure compliance with data privacy, security, and regulatory requirements.
• Implement appropriate access controls and data protection measures.
• Support auditing and governance initiatives.

Operational Support

• Troubleshoot and resolve data-related incidents and performance issues.
• Perform ongoing maintenance and upgrades of data platforms.
• Participate in disaster recovery and business continuity planning.

Stakeholder Collaboration

• Work closely with business analysts, data analysts, developers, and data scientists.
• Translate business requirements into technical data solutions.
• Provide technical guidance on data architecture and best practices.

Required Qualifications

Education

• Bachelor's Degree in Computer Science, Information Systems, Information Technology, Data Science, Engineering, or related field.
• Relevant certifications are advantageous.

Preferred Certifications

• Microsoft Certified: Azure Data Engineer Associate
• Microsoft Fabric Analytics Engineer Associate
• Azure Fundamentals (AZ-900)
• Databricks Certified Data Engineer
• Snowflake Certification (advantageous)

Experience

• Minimum 4-8 years' experience in data engineering, database development, or data integration.
• Experience designing and maintaining enterprise-scale data solutions.
• Experience working in cloud environments (Azure preferred).
• Experience supporting business intelligence and analytics initiatives.

Technical Skills

Essential

• Advanced SQL
• Python
• Data modeling and database design
• ETL/ELT development
• REST APIs and data integration
• Data warehouse design
• Git version control

Preferred Technologies

• Microsoft Fabric
• Azure Data Factory
• Azure Databricks
• Azure Synapse Analytics
• Azure Data Lake Storage (ADLS)
• SQL Server
• Power BI
• Apache Spark
• Snowflake
• Kafka
• Airflow

Competencies

Technical Competencies

• Data architecture design
• Data pipeline development
• Performance tuning and optimization
• Data quality management
• Cloud data platform administration
• Troubleshooting and problem-solving

Behavioral Competencies

• Strong analytical thinking
• Attention to detail
• Effective communication skills
• Stakeholder management
• Collaboration and teamwork
• Continuous learning mindset
• Ability to work under pressure and meet deadlines

Key Performance Indicators (KPIs)

• Data pipeline uptime and reliability
• Data processing performance and efficiency
• Data quality and accuracy metrics
• Incident resolution time
• Delivery of data engineering projects within agreed timelines
• Compliance with data governance standards
• User satisfaction with data services