Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

ParagonCorp

NewBe an early applicant

Senior Scraper Engineer

Posted

Description

Key Responsibilities

  • Design, build, and maintain scalable Python-based scraping systems (e.g., using requests, BeautifulSoup, Selenium, Playwright, Scrapy), architecting for reliability and maintainability rather than one-off scripts
  • Design and implement anti-detection strategies at scale, rotating proxies, CAPTCHA-bypass solutions, user-agent/device fingerprint randomization, to sustain high scraping success rates across changing site and app defenses
  • Reverse-engineer and extract data from mobile apps (Android/iOS) via API traffic interception, SSL pinning bypass, and emulator/device automation, in addition to standard web scraping
  • Architect data pipelines that ingest and normalize structured and unstructured data from web APIs, mobile app APIs, HTML, JSON, and XML sources
  • Own orchestration and scheduling of scraping workflows (Airflow, Prefect, n8n, or similar), including retry logic, alerting, and failure recovery
  • Lead integration of pipelines into downstream systems (Snowflake, Google Sheets, S3, GCS, SharePoint), ensuring data quality and consistency
  • Establish monitoring, logging, and observability practices for scraping infrastructure to catch breakages proactively rather than reactively
  • Evaluate and prototype new scraping targets (web and mobile), data enrichment sources, and tooling; make build-vs-buy recommendations
  • Anticipate and adapt to web/app structure, anti-bot, and API changes ahead of failures; mentor junior engineers on scraping best practices


Requirements

  • Bachelor's degree in Computer Science, Information Systems, or related field
  • At least 5 years of experience in web scraping, crawling, or automation scripting, including at least 1-2 years designing scraping systems at scale (not just writing individual scripts)
  • Strong proficiency in Python and its scraping ecosystem (requests, BeautifulSoup, Selenium, Playwright, Scrapy)
  • Hands-on experience with mobile scraping/reverse engineering, tools like Frida, mitmproxy, Charles Proxy, Objection, or Appium; experience bypassing SSL pinning and root/jailbreak detection
  • Proven experience with headless browser automation (Puppeteer, Playwright) and defeating modern anti-bot measures (proxy rotation, CAPTCHA handling, rate-limiting, fingerprinting)
  • Hands-on experience with containerization (Docker) and Git-based CI/CD in production environments
  • Track record of scraping high-defense targets such as social media or e-commerce platforms (web and/or mobile app) at scale
  • Experience owning end-to-end data pipelines into a warehouse (Snowflake preferred) or cloud storage
  • Prior experience mentoring or leading other engineers is a plus

Skills

Apply

See also