Data Engineer
Posted Updated
Description
卡杯 is seeking a Data Engineer in India (Gurugram) with 2-5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. You will leverage SQL and Python/PySpark to run ETL/ELT pipelines, address data quality, and support reporting platforms.
Responsibilities include ingesting data from multiple sources, building transformations, and implementing monitoring, logging, and alerts.
Design, develop, and maintain scalable ETL/ELT pipelines for batch processing. Build data transformations using SQL, Python, and PySpark. Ingest data from databases, APIs, files, and cloud storage. Create analytics-ready datasets for Power BI, Superset, and reporting tools. Implement data-quality checks, monitoring, and alerting. Troubleshoot pipeline failures, performance bottlenecks, and data discrepancies. Optimize data processing for runtime, memory, and cost. Automate workflows with orchestration tools like Apache Airflow. Maintain data mappings and runbooks; collaborate with BI analysts and stakeholders. 2-5 years of professional data engineering experience. Strong SQL skills and data validation. Hands-on Python and/or PySpark for data processing. Production experience with ETL/ELT pipelines. Experience with large datasets, incremental loads, and partitioned data. Familiarity with relational databases, data warehousing, and common file formats. 5 Days working Remote job Medical InsuranceSkills
Sign in to read and join the discussion.