Data Lake Technical Staff

Role & responsibilities

Data Engineering: ETL pipeline design, full-load processing, data

transformation, and data integration.

• API Development: OData report APIs, external API integrations, Flask APIs,

and service migration.

• Big Data Technologies: Apache Spark, PySpark, Hadoop, and HDFS-based

data processing.

• Data Modeling: Development and optimization of denormalized tables and

reporting datasets using PySpark.

• Pipeline Orchestration: Apache Airflow DAG development for scheduled and

automated processing.

• Reporting Solutions: OData reports, Report Master configuration, filtering, and

report generation.

• Application Modernization: Migration of .NET 3.1 applications to .NET 8.0 and

planned Angular upgrades.

• Automation and Deployment: Deployment agents, CI/CD processes, and multi-

environment support.

• Production Operations: Production support, troubleshooting, and pipeline

performance optimization.

Preferred candidate profile