Data Engineer - Web Scraping (3 months contract)
Posted Updated
Description
CXG is looking for a Data Engineer - Web Scraping to join remotely on a 3-month contract. You will maintain and manage website scraping configurations using Python, monitor for errors, and ensure reliable data collection across multiple sites.
You will collaborate with stakeholders to specify scraping requirements, develop data pipelines, and prepare periodic reports. Strong Python, ETL, Airflow/DBT and cloud experience (Azure) are preferred, along with Git and Terraform familiarity.
Maintain and manage website scraping configurations using Python. Monitor scraping configurations for errors and crashes. Oversee retrieved data for issues and blockages. Coordinate with stakeholders to define scraping task requirements and report issues. Prepare and share periodic reports on scraping activities with stakeholders. Develop pipelines to ingest data into the Datalake and perform transformations. Proven data engineering experience with scalable architectures. Strong ETL, data warehousing, and modelling capabilities. Experience with SQL and NoSQL databases.Skills
Sign in to read and join the discussion.