Data Engineer: Databricks & BigQuery Pipelines (Remote)
Posted Updated
Description
AgileEngine in Valencia is seeking a Data Engineer who will design and optimize scalable ETL/ELT pipelines using Databricks and PySpark. You will productionize agentic workflows and integrate with GCP, BigQuery, and Delta Lake to support data automation.
Ideal candidates have 5+ years of hands-on Databricks experience, strong SQL, and a track record of building reliable data solutions. The role offers flexible times, remote/onsite options, and collaboration with Fortune 500 brands.
Design, develop, and optimize scalable ETL/ELT data pipelines with Databricks, PySpark, and SQL. Build and operationalize agentic workflows to automate data engineering tasks. Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake. Develop data pipelines to support new business requirements. Create reusable frameworks and components for multiple use cases. Implement data quality checks, monitoring, and production controls. Optimize PySpark and SQL workloads for performance and scalability. Support testing, deployment, and productionization of data solutions. Troubleshoot complex data and production issues, delivering sustainable solutions. Collaborate with business and platform teams to identify more automation opportunities. 5+ years hands-on experience with Databricks and PySpark. Advanced SQL and data-processing skills. Hands-on GCP experience, esp. BigQuery. Experience with Delta Lake and lakehouse architectures. Strong ETL/ELT, data pipelines, performance optimization, and data quality. Proven ability to build scalable, production-grade data solutions. Strong analytical and troubleshooting skills. Understanding of software engineering practices (testing, VCS, deployment, monitoring). Professional growth Competitive compensation Exciting projects Remote & office optionsSkills
Sign in to read and join the discussion.