Data Engineer

We are looking for a talented Data Engineer to join a leading international energy company and contribute to the development, support, and continuous improvement of a modern data platform. This role offers the opportunity to work with cloud technologies, large-scale data environments, and cross-functional teams to deliver reliable and scalable data solutions.

Key Responsibilities

• Provide operational support for the data platform, troubleshooting and resolving technical issues.
• Maintain and improve data pipelines and ETL/ELT processes to ensure reliable and scalable data flows.
• Collaborate with platform architects to guarantee platform performance, security, and high availability.
• Optimize data processes, monitoring capabilities, and overall platform efficiency.
• Support data quality frameworks and monitoring mechanisms to ensure data accuracy, consistency, and reliability.
• Manage and support data lakes, data warehouses, and streaming data solutions.
• Work closely with data scientists, analysts, and business stakeholders to address platform-related challenges.
• Automate repetitive tasks and enhance data ingestion, transformation, and delivery processes.
• Troubleshoot and optimize SQL queries and data workflows for performance and cost efficiency.
• Document platform changes, operational procedures, and best practices.
• Review, evaluate, and approve data product releases before deployment to production environments.

Required Skills & Experience

• Proven experience working with data pipelines in production environments.
• Strong expertise in SQL and distributed data processing frameworks such as Apache Spark.
• Hands-on experience supporting and troubleshooting dbt, Apache Kafka, Apache Ignite, Apache NiFi, and Apache Airflow.
• Proficiency in Python or Scala for data engineering and automation tasks.
• Experience with AWS cloud services and cloud-based data solutions.
• Knowledge of monitoring and observability tools such as Grafana.
• Understanding of data governance, security, and compliance best practices in cloud environments.
• Experience implementing CI/CD pipelines and testing frameworks for data workflows.
• Strong analytical, problem-solving, and communication skills.
• Fluent in Portuguese and English, both written and spoken.

Nice to Have

• Familiarity with Dremio and Data Lakehouse architectures.
• Experience provisioning and managing cloud infrastructure with Terraform.
• Knowledge of Power BI On-Premises Gateway.
• Experience with data catalog and governance platforms such as Collibra or Alation.
• Exposure to machine learning pipelines and feature engineering processes.