Data Engineer

Key Responsibilities

Responsibilities and essential job functions include but are not limited to the following:

1. Demonstrate deep knowledge of the data engineering domain to build and support non-interactive (batch, distributed) & real-time, highly available data, data pipeline, and technology capabilities

2. Build fault-tolerant, self-healing, adaptive, and highly accurate data computational pipelines

3. Provide consultation and lead the  implementation of complex programs

4. Develop and maintain documentation relating to all assigned systems and projects

5. Tune queries running over billions of rows of data running in a distributed query engine

6. Perform root cause analysis to identify permanent resolutions to software or business process issues

Basic Qualifications

Bachelor’s  degree in computer science, management information systems, or related discipline, or equivalent work experience

MUST HAVE Technology skills (7/10 or higher):

1. Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks (preferred)

2. Hands-on data pipeline development, ingest patterns in Azure

3. Orchestration tools, ADF or Airflow

4. SQL

5. Denormalized Data modeling for big data systems

MUST HAVE competencies:

1. Collaborative, able to work remotely, and still be an engaging team member.

2. Strong analytical and design skills.