Web Scraping Engineer (U.S. Citizen)
NOTE: This is a full-time employment opportunity (No C2C, subcontractor or 1099 engagements, please.). The candidate MUST be a U.S. Citizen and able to complete/pass/hold at a minimum public trust investigation (active public trust is preferred). This is a remote opportunity; however, candidate must reside in the DC, Maryland, Virginia (DMV) metro area.
Daily Responsibilities
- Support the development and maintenance of the agency’s web scraping infrastructure. Responsible for extracting data from various websites and APIs, ensuring data quality and accuracy, and optimizing the scraping process for efficiency. Duties include:
- Develop and maintain web scraping scripts and tools to extract data from websites and APIs.
- Collaborate with cross-functional teams to understand data requirements and implement scraping solutions accordingly.
- Monitor and troubleshoot scraping processes to ensure data quality and accuracy.
- Optimize scraping scripts for performance and efficiency, considering factors such as speed, scalability, and resource utilization.
- Stay up to date with the latest web scraping techniques, tools, and best practices.
- Conduct data analysis and validation to ensure the integrity of scraped data.
- Collaborate with data engineering and data science teams to integrate scraped data into our data pipelines and systems.
- Document and communicate technical solutions, processes, and best practices to team members.