Senior Data Engineer
Posted
Description
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer based in Spain.
This role offers the opportunity to take ownership of a data platform that transforms information from multiple sources into reliable, continuously updated company intelligence. You will lead the platform's development from architecture and ingestion through processing, storage, and production operations. Working with diverse data sources, you will design scalable pipelines that combine event-driven processing with orchestrated batch workflows. You will also establish robust data quality, validation, and monitoring practices to ensure information remains accurate, consistent, and accessible. The position combines hands-on engineering with architectural responsibility, offering the opportunity to build substantial data infrastructure and integrate modern technologies, including LLM-powered enrichment and extraction.
Accountabilities
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
This role offers the opportunity to take ownership of a data platform that transforms information from multiple sources into reliable, continuously updated company intelligence. You will lead the platform's development from architecture and ingestion through processing, storage, and production operations. Working with diverse data sources, you will design scalable pipelines that combine event-driven processing with orchestrated batch workflows. You will also establish robust data quality, validation, and monitoring practices to ensure information remains accurate, consistent, and accessible. The position combines hands-on engineering with architectural responsibility, offering the opportunity to build substantial data infrastructure and integrate modern technologies, including LLM-powered enrichment and extraction.
Accountabilities
- Design and develop a multi-source data platform supporting concurrent ingestion, event-driven processing, and orchestrated batch workflows.
- Build reliable data ingestion systems connecting to provider APIs, webhooks, files, and web scraping sources, with appropriate rate limiting and robust error-handling mechanisms.
- Implement scheduled workflows and historical data backfills using orchestration tools such as Airflow, Dagster, or Prefect.
- Develop event-driven processing systems using messaging infrastructure and task queues, including retries, dead-letter queues, and controlled replay capabilities.
- Store original source payloads in cloud storage and use dbt to transform data into structured raw, standardized, and curated layers.
- Implement idempotent upserts, data contracts, automated validation, pipeline testing, and monitoring to detect freshness issues and processing failures.
- Design, maintain, and optimize PostgreSQL schemas and data-loading strategies to support performance, reliability, and scalability.
- Integrate external provider enrichment and structured data extraction capabilities using Large Language Models (LLMs).
- Take end-to-end ownership of the platform, covering architectural decisions, implementation, deployment, ongoing maintenance, and production reliability.
- At least 5 years of experience building and operating production data platforms, particularly those involving multi-source ingestion and end-to-end platform ownership.
- Production experience with workflow orchestration tools, including managing dependencies, retries, recovery processes, and complex workflows. Airflow is preferred, while Dagster, Prefect, Temporal, or comparable technologies are also relevant.
- Hands-on production experience with dbt and Google Cloud Platform (GCP).
- Strong Python programming skills, including experience with asynchronous HTTP requests and concurrent processing.
- Advanced SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading techniques.
- Experience designing and operating event-driven processing systems, messaging queues, and reliable data pipelines.
- Knowledge of data contracts, raw and curated storage architectures, data retention policies, automated testing, and pipeline monitoring.
- Ability to design resilient ingestion and transformation processes that handle failures, maintain data consistency, and support recovery.
- Experience building data platforms or substantial pipelines from the ground up is a strong advantage.
- Familiarity with web scraping, entity resolution, company data enrichment, and LLM-based data extraction is beneficial.
- Experience working in a small engineering team or startup environment is also an advantage.
- Annual salary range of USD 100,000–140,000.
- Fully remote working arrangement within eligible European locations, including Switzerland, the United Kingdom, Norway, Ukraine, Iceland, and Liechtenstein.
- Opportunity to take end-to-end ownership of a data platform and contribute to its architecture and long-term development.
- Opportunity to work with modern data engineering technologies, cloud infrastructure, event-driven systems, and LLM-powered data enrichment.
- Visa sponsorship is not available for this position.
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Skills
Sign in to read and join the discussion.