Data Engineer
Posted Updated
Description
Quantum Technologies LLC is seeking an experienced Data Engineer to build scalable data pipelines, data warehouses, and integration solutions. You will transform raw data into analytics-ready datasets to support BI, reporting, and data science initiatives.
Responsibilities include designing ETL/ELT workflows, ingesting data from APIs and databases, and optimizing data warehouse models. Proficiency in Python, SQL, Spark, and cloud data platforms is required; experience with CI/CD and data
Design, develop, and maintain robust ETL/ELT pipelines for structured and semi-structured data. Ingest data from APIs, databases, files, and streaming sources. Build scalable data transformation workflows using Python, SQL, and Spark. Develop and optimize data warehouse tables, data models, and analytical datasets. Implement batch and real-time data processing solutions. Improve query performance, pipeline reliability, and data processing efficiency. Implement data quality checks, validation rules, reconciliation, and error-handling mechanisms. Monitor production pipelines and troubleshoot data failures. Apply data security, access controls, and governance standards. Collaborate with data analysts, data scientists, architects, and business stakeholders. Automate deployments and support CI/CD practices. Maintain technical documentation and participate in Agile development activities. Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field or equivalent experience. 5+ years of experience in data engineering, data integration, or ETL development. Strong proficiency in Python, SQL, and data transformation techniques. Hands-on experience with cloud data platforms and distributed processing. Understanding of relational databases, data warehousing, and data lake architecture. Experience with performance tuning, debugging, and production support.Skills
Sign in to read and join the discussion.