Senior Data Software Engineer – Build Scalable Data Pipelines
Posted Updated
Description
EPAM Systems Inc. seeks a Senior Data Software Engineer to architect scalable data pipelines and data solutions powering critical business decisions.
You will collaborate with a disciplined team focused on clean, well-tested code and robust engineering practices, tackling large-scale data problems with Lakeflow, Delta Lake, and Databricks. You will design ingestion workflows, perform data quality checks, participate in code reviews, and contribute to architecture evolution within an agile
Architect and construct scalable data pipelines across multiple platforms. Build and sustain data ingestion workflows from diverse sources. Write clean, high-performing, well-tested code. Collaborate with cross-functional teams to shape data solutions. Monitor and optimize data pipeline performance and reliability. Participate in code reviews to maintain quality. Diagnose and fix data quality issues to ensure accuracy across datasets. Contribute to evolving data architecture practices and standards. Create documentation for technical workflows and solutions. Operate within an agile framework and participate in sprint planning. At least 3 years of hands-on data engineering experience. Strong experience with PySpark and Spark SQL for large-scale data workflows. Experience building and overseeing Lakeflow pipelines and Jobs for data orchestration. Proficient with Auto Loader and JDBC/CDC ingestion for various sources. Experience with Delta Lake and Unity Catalog for storage and governance. Experience writing unit tests with pytest to ensure code reliability. Familiarity with Git and deployment workflows. Healthcare benefits Paid time off Upskilling and certification courses LinkedIn Learning accessSkills
Sign in to read and join the discussion.