Data Engineer: Databricks, GCP & AI Agentic Workflows
Posted Updated
Description
AgileEngine, LLC. is seeking a Data Engineer with strong Databricks and GCP experience to design, develop, and productionize scalable ETL/ELT pipelines and agentic workflows.
You will work across Databricks, GCP, BigQuery, and Delta Lake, building reusable data engineering components and ensuring data quality. The role emphasizes scalable data solutions, performance optimization, and collaboration with cross-functional teams to automate data workflows and support evolving business needs.
Design, develop, and optimize scalable ETL/ELT data pipelines. Build and operationalize agentic workflows to automate data processes. Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake. Develop pipelines and solutions for new business requirements. Build reusable frameworks across multiple data use cases. Implement data quality checks, monitoring, validation, and production controls. Optimize PySpark and SQL workloads for performance and scalability. Support testing, deployment, productionization and enhancements. Troubleshoot complex data and production issues. Collaborate with business and platform teams to identify more automation. 5+ years of hands-on Databricks and PySpark experience. Advanced SQL and data-processing skills. Hands-on GCP experience, especially BigQuery. Experience with Delta Lake and data lake architectures. Strong ETL/ELT, pipeline design, performance optimization, and data quality. Production-grade data solutions development. Strong analytical and troubleshooting abilities. Understanding of software engineering practices (testing, VCS, deployment, monitoring). Professional growth Competitive compensation Exciting projects FlextimeSkills
Sign in to read and join the discussion.