Databricks Data Engineer – GenAI & Lakehouse Innovator
Summary
Build and maintain scalable data pipelines and Lakehouse architectures using Databricks, PySpark, and Delta Lake to power analytics and GenAI solutions for enterprise clients.
Databricks Community of Practice – Join our Databricks Community of Practice in Poland, where delivery excellence is at the core of everything we do. We design and build modern data products as well as advanced GenAI and agentic solutions powered by Databricks.
Responsibilities
- Design and implement scalable end‑to‑end data pipelines (ETL/ELT) using Databricks, including batch and streaming, and modern orchestration patterns such as Delta Live Tables.
- Develop and maintain data models using Lakehouse and Medallion architecture (Bronze, Silver, Gold layers).
- Build robust data transformations using PySpark, Spark SQL, and Delta Lake.
- Integrate diverse cloud and enterprise data sources into unified, high‑quality, analytics‑ready datasets.
- Collaborate with architects, analysts, and data scientists to deliver production‑grade data products.
- Implement DevOps and DataOps practices, including Git‑based version control, CI/CD pipelines, and testing of data workflows.
Requirements
- Experience Requirements (Cumulative):
- At least 2 years hands‑on experience with relational or analytical databases, applying SQL, Python or Spark for development, testing, debugging, and performance optimisation in production environments.
- At least 1 year practical experience designing or implementing ETL/ELT processes using Databricks and Apache Spark in cloud‑based environments.
- Exposure to data modelling and architecture: practical experience creating conceptual, logical and physical data models using dimensional, relational or Data Vault techniques in analytical environments.
- Technical Skills:
- Hands‑on experience with Databricks platform, including Delta Lake and Lakehouse architecture concepts.
- Practical knowledge of Apache Spark (PySpark, Spark SQL), batch and streaming processing.
- Understanding of Medallion architecture design patterns.
- Strong SQL skills in analytical and distributed data environments (e.g., Spark SQL).
- Ability to profile, tune and optimise SQL queries for large‑scale data processing workloads.
- Data Modelling & ETL:
- Solid understanding of dimensional modelling; ability to translate business requirements into conceptual, logical and physical models.
- Hands‑on experience building scalable ELT/ETL pipelines in modern cloud‑native environments.
- Cloud & Big Data (Experience or Interest):
- Experience with, or strong interest in, processing and integrating data on major cloud platforms (GCP, Azure, AWS).
- Familiarity with cloud storage, managed databases and serverless / data pipeline services is desirable.
- Understanding of CI/CD pipelines and version control (Git) in data engineering projects.
- Nice to Have:
- Experience with Databricks Workflows, Unity Catalog, Auto Loader, Structured Streaming.
- Knowledge of streaming technologies such as Kafka.
- Experience with dbt, MLflow, Feature Stores or Infrastructure as Code (e.g., Terraform).
- Exposure to data quality frameworks and monitoring solutions.
- Experience supporting AI/GenAI use cases (e.g., LLM data pipelines, vector databases, AI‑ready data preparation).
- Programming & Analytics:
- Practical scripting or programming skills (SQL, Python, Spark) to support data transformation, automation and basic analytics.
- Ability to translate complex business requirements into robust, scalable data solutions.
- Soft Skills & Language:
- Strong analytical thinking, problem‑solving and attention to detail.
- Professional working proficiency in English and effective communication with technical and non‑technical stakeholders.
- Willingness to travel to client locations across Europe as required.
Work Conditions
- Employment contract: full‑time (umowa o pracę).
- Headquarters in Warsaw.
- Traveling required: work with prestigious clients and deliver large‑scale transformational change; hybrid model with remote, office and client‑site work.
- Flexible working arrangements to support work/life needs.
Benefits
- Individual support of a People Lead and a specific path of professional development.
- Extensive training package (soft, technical, language training, e‑learning platforms, Gallup test, GenAI training, co‑financing of courses, certification).
- Employee Assistance Program (legal, financial, psychological consultations).
- Employee share purchase plan and quarterly dividends for share holders.
- Paid employee referral program.
- Private medical care and life insurance.
- Access to WorkSmile benefits platform (incl. Multisport card).
Equal Employment Opportunity
Accenture does not discriminate against employees on the basis of race, religion, gender, age, disability, national origin, political beliefs, or any other basis impermissible under Polish law. We are committed to building a better, stronger and more diverse company for future generations.