Associate Data Engineer

• Degree in Computer Science or Computer Engineering


• Minimum 5 years workingexperience in system operations compliance and management areas


• Project hands-on experiencespecifically with AWS platform (primary requirement)


• project experience in cloudoperations or cloud architecture


• Must be cloud certified (AWS)

Core Technical Skills:

• proficiency in Databricksplatform, including workspace management, cluster configuration, and joborchestration


• Strong expertise in ApacheSpark within Databricks environment, including Spark SQL, DataFrames, and RDDs


• Good in-depth understanding ofdata warehouse concepts, data profiling, data verification and advancedanalytics techniques


• Strong knowledge ofmonitoring, incident management, and cloud cost control

Technology Stack Experience:

• Databricks


• AWS cloud services andarchitecture


• IDMC (Informatica DataManagement Cloud)


• Tableau for data visualization


• Oracle Database management


• ML Ops practices withinDatabricks environment


• STATA for statistical analysisis advantage


• Amazon SageMaker integrationwith Databricks


• DataRobot platformintegration

Responsibilities:

Operational Responsibilities:

• Monitor and maintainproduction data pipelines to ensure 99.9% uptime and optimal performance


• Implement comprehensivelogging, alerting, and monitoring systems using Application monitoring tools


• Perform regular health checksperformance, job execution times, and resource utilization to identify andresolve bottlenecks proactively


• Manage incident responseprocedures for pipeline failures, including root cause analysis, resolution,and post-incident reviews


• Establish and maintaindisaster recovery procedures and backup strategies for critical data assetswithin the Databricks environment