Data Engineer

Summary

Build and maintain data pipelines, warehouses, and analytics apps using Python, Spark, and open-source big-data tools.

Here’s what you will be doing:

Develop, test, monitor, and optimize end-to-end data processing jobs and workflows from data acquisition to data exposure.
Develop and enhance applications and software that consume data from data lakes or warehouses for visualization and advanced analytics.
Develop, test, and maintain big data architecture and open source databases such as Hadoop, Hive, HBase, and Presto.
Establish controls on data collection, processing, transformation, and presentation; perform regular data quality audits ensuring accuracy, completeness, and timeliness.
Communicate regularly with business users, data owners, IT teams, and other stakeholders to clarify requirements and present solutions.
Manage daily operations of scheduled jobs and programs, enforce policies, maintain systems, respond to user requests, and ensure operational-level agreement compliance.
Ensure timely resolution of issues related to end-to-end data platform components.
Oversee 3rd party vendor partners to meet service-level agreements for support, troubleshooting, patching, and escalation.
Support technology upgrades or migrations to keep systems up to date.
Diagnose issues and provide recommendations to improve overall delivery.
Create solutions design and technical architecture documentation.
Perform other related duties as assigned.

The company is looking for:

Graduate with a bachelor’s degree in Computer Science, Information Systems/Technology, Engineering, or related field, or equivalent project‑related experience.
Experience in software development.
Working knowledge of Python, Perl, C/C++, PL/SQL, NoSQL, and Java.
Proficiency in Linux and Unix operating systems.
Knowledge and experience with Big Data technologies such as HDFS, Hadoop, and Apache Spark is a plus.
Familiarity with the end-to-end software development lifecycle.