Data Engineer
Summary
Data Engineer building data ingestion pipelines on the Databricks platform and data transformations with Python/PySpark, running on AWS, to support life-science clients with discovery, clinical insights and R&D challenges. A 3-6 month engagement (extendable); must be available to work until 5 pm PST.
Time Zone: The candidate should be available to work till 5 pm PST
Duration:from 3-6 months with possible extension
Job Description:
We are currently looking for a Data Engineer to assist in developing intelligent solutions enabling our customers to accelerate discovery, acquire clinical insights, and address R&D challenges.
Responsibilities:
- Design and create pipelines for data ingestions based on the Databricks Platform.
- Preparing data transformations with Python and PySpark library.
What we expect:
- Familiarity with the Databricks platform
- Ability to write robust code with Python
- Proficiency with Spark (PySpark)
- Knowledge of cloud platforms: AWS
Nice to have:
Knowledge of Lakehouse concept including experience using Delta Lake optimized storage layer.