Data Analytics Engineer | .Net / Sql | Remote
Posted Updated
Description
Toptal is seeking a data engineering professional to design resilient ETL pipelines and standardized data lakes. You will implement Medallion architecture, leverage Parquet/Iceberg/Delta formats, and perform embedded analytics with Python and DuckDB.
You will optimize PostgreSQL schemas and support lightweight SQLite storage, while enabling distributed processing with Databricks or Spark in a cross-cloud setup.
Pipeline design: build robust data processing pipelines in Python using Medallion architecture. Open storage management: standardize data in Parquet, Iceberg, Delta Lake for cross-cloud use. Embedded analytics: use DuckDB with Python for fast, localized ETL analytics. Database operations: architect and optimize PostgreSQL schemas and lightweight SQLite storage. Distributed compute: scale ETL and analytics with Databricks or Spark. Advanced SQL with PostgreSQL expertise and SQLite knowledge. Hands-on experience building analytical pipelines with Python and DuckDB. Mastery of open formats like Parquet, Iceberg, and Delta Lake. Proven track record designing data lakes using Medallion architecture. Familiarity with Databricks and PySpark for distributed processing.Skills
Sign in to read and join the discussion.