[Job-32448] Master Data Developer (Databricks), Colombia
Description
At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions.
With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy.
We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day.
As CI&T grows, we seek a talented, motivated, and experienced Data Developer to join our team and build robust data-driven solutions for our customers. This role is pivotal in designing, developing, and maintaining scalable data infrastructure and pipelines that enable our data initiatives to thrive.
The Data Developer will be responsible for implementing and optimizing data architectures, ensuring data quality and reliability, and collaborating closely with analytics and business teams to deliver high-performance data solutions. This position requires strong technical expertise in data engineering, cloud platforms, and modern data stack technologies, combined with the ability to translate business requirements into effective technical solutions.
Responsibilities:
- Data Pipeline Development: Build, optimize, and maintain scalable data pipelines and ETL/ELT processes to support analytics and business intelligence needs.
- Data Architecture Implementation: implement data architectures, including data lakes, warehouses, and integration layers that support the organization's data strategy and modernization.
- Data Quality & Governance: Implement data quality checks, validation processes, and monitoring systems to ensure data accuracy, consistency, and reliability.
- Integration & API Development: Develop and maintain integrations between various data sources, third-party tools, and analytics platforms.
- Collaboration: Work closely with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions that enable data-driven decision making.
- Performance Optimization: Monitor and optimize data pipeline performance, query efficiency, and storage costs to ensure scalable and cost-effective solutions.
- Documentation: Create and maintain comprehensive technical documentation for data architectures, pipelines, and processes.
- Data Security: Implement security best practices and ensure compliance with data privacy regulations across all data systems.
- Powered by AI: Leverage AI tools and automation to enhance data engineering workflows and productivity.
Requirements:
- Hands-on Databricks migration with the databricks/migrate tool (notebooks,
jobs, clusters, secrets, ACLs), Unity catalog to Glue Catalog - Notebook and job migration: notebooks, DAG pipelines, DLT pipelines
- Storage path refactoring: /mnt/ DBFS references to S3-backed locations
- Data migration execution: Deltalake format to Iceberg format on S3 (Delta tables, raw files), incremental sync
- Streaming: Auto Loader / DLT checkpoint re-bootstrap, event_id dedup
- Validation: row counts, checksums, sample queries, ETL runtime comparison
- Secrets: recreate secret scopes in AWS Secrets Manager
- AWS services: S3, Glue, Secrets Manager, IAM basics, Athena, LakeFormation and
EMR - Languages: Spark SQL, PySpark, Java, Python and Scala
- Agile pod delivery: daily ticket updates, standups, code review
Skills
As published by lever · 19 questions · 1 written answer
Basics
Resume/CV, Full name, Pronouns, Email, Phone, Current location, Current company, LinkedIn URL
Short answers (5)
- How comfortable do you feel working in an English-speaking environment?
- Please, confirm country and city where you are currently living
- When will you be available to start working with us? (DD/MM/AAAA)
- If the previous answer is Yes, please share the name(s) of these individuals. optional
- Confirm the name of the company where you work/ Confirme o nome da empresa em que você trabalha atualmente
Pick from a list (13)
- [EN] What is your level of confidence when communicating and collaborating in an English-speaking work environment? optional
- [ES] What is your level of confidence when communicating and collaborating in an Spanish-speaking work environment? optional
- How did you hear about our job opening? optional
- Do you have any relatives (such as parents, siblings, in-laws, spouses, or children, stepchildren, stepfather/stepmother, son-in-law/daughter-in-law), close friends, or acquaintances (with whom you have a significant and ongoing relationship) who currently work at CI&T?
- Are you legally authorized to work in Colombia?
- Do you require any type of accessibility for the selection process and/or in your daily life??
- Will you now or in the future require sponsorship for Colombia's employment visa status?
- Would you like to tell us your gender identity? We ask about gender identity to better understand and support the diverse identities of our employees and to ensure our workplace is inclusive. optional
- Are you part of the LGBTQIAP+ community? optional
- How would you self-declare? Collecting information on ethnic backgrounds allows us to ensure our diversity and inclusion efforts are effective and to address any potential disparities. optional
- To understand better your necessities or any accommodations you need for our process & interviews, we would like to know if you are a person with disabilities? optional
- If applicable, please tell us what type of disability you have: (Edit to include options:) optional
- By selecting the option below, I declare that I have read, understood, and agree to the terms of this document, and I consent to the processing of my personal data by CI&T and Lever for the purposes described herein. Read the document here: https://drive.google.com/file/d/1dMaskYl2VD1AtN9MfiHrZiA-YFYwq8aI/view optional
Written answers (1)
- What is your salary expectation? (please inform the gross amount in Colombian Pesos)