Data Engineer (Databricks)

Location: Columbia, MD (Remote/Hybrid if local to Maryland; MON & WED in the office)

Residency Requirement: Candidates must have lived in the U.S. for at least 3 of the past 5 years and be authorized to work in the U.S. (Citizen, Permanent Resident, or EAD).

Sponsorship: This position does not offer any type of sponsorship. Candidates must already be authorized to work in the U.S. (e.g., Citizen, Permanent Resident, or EAD).

Clearance: Public Trust Clearance (or ability to obtain)

Job Overview:

Cogent People Inc. is seeking a skilled and motivated Data Engineer with Databricks experience to join our team. You will build and maintain data pipelines for a mission-critical, data-centric system supporting a large-scale Federal Government IT program in the healthcare domain.

Responsibilities:

· Understand business requirements and turn them into working pipelines that fit the overall architecture and security boundaries.

· Build, test, and maintain data pipelines on Databricks using Python, PySpark, and Spark SQL.

· Design Delta Lake tables and ingestion patterns that work for both batch loads and recurring reports.

· Convert existing Oracle SQL and PL/SQL logic to Databricks, and check that the results match the source system.

· Automate recurring reports on Databricks.

· Schedule and monitor jobs with Databricks Workflows, and tune them for performance, cost and availability.

· Handle day-to-day Databricks workspace administration: compute and cluster policies, users, groups and service principals, permissions, and secrets.

· Apply data governance and access controls using Unity Catalog inside federal security boundaries.

· Follow DevSecOps practices: code in Git, CI/CD for pipelines, and promoting changes through environments.

· Work with developers, project managers, business analysts and testers in an Agile team, and document what you build.