Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

People Data Labs

NewBe an early applicant

Senior Software Engineer (Data Acquisition)

Posted Updated

Description

  • We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset
  • Our customers are trying to solve complex problems, and we only help them achieve their goals as a team
  • Our Data Engineering & Acquisition Team ensures our customers have standardized and high quality data to build upon
  • You will be crucial in accelerating our efforts to build standalone data products that enable data teams and independent developers to create innovative solutions at massive scale
  • In this role, you will be working with a team to continuously improve our existing datasets as well as pursuing new ones
  • If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you
  • We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things
  • Failure is embraced at PDL as long as we continue to learn and grow from it
  • Contribute to the architecture and improvement of our data acquisition and processing platform, increasing reliability, throughput, and observability
  • Use and develop web crawling technologies to capture and catalog data on the internet
  • Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web
  • Design and develop backend services that manage distributed job orchestration, data pipelines, and large-scale asynchronous workloads
  • Structure and model captured data, ensuring high quality and consistency across datasets
  • Continuously improve the speed, scalability, and fault-tolerance of our ingestion systems
  • Partner with data product and engineering teams to design and implement new data products powered by the data you help collect, and enhance and improve upon existing products
  • Learn and apply domain-specific knowledge in web crawling and data acquisition, with mentorship from experienced teammates and access to existing systems

Benefits

  • Remote-first work, always: As a remote-first company, we believe in flexible work that fits your life. It’s that simple.
  • Unlimited PTO: No checking how many vacation days you have left! Rest, recharge, and take time off when you need it.
  • Monthly Employee Stipends: We provide monthly stipends automatically for employees. Think internet bills bills, running shoes, your new pickleball obsession, etc. — use it as you wish!
  • Medical, Dental, Vision Insurance: We provide multiple insurance plan options to make sure you and your family are covered!
  • Experience with streaming data systems (Kafka, Pub/Sub, Spark Streaming, etc.)
  • Scope and break down complex projects into deliverable milestones, and communicate progress, risks, and blockers effectively
  • Proficiency with SQL and data warehousing (Snowflake, Redshift, BigQuery, or similar)
  • Experience as a Red Teamer
  • Balance pragmatism with craftsmanship, shipping reliable systems while continuously improving them
  • Communicate clearly and thoughtfully in writing (Slack, docs, design proposals)
  • Degree in a quantitative field such as computer science, mathematics, or engineering
  • Familiarity with network architecture and debugging (HTTP, DNS, proxies, packet capture and analysis)
  • Solid understanding of distributed systems concepts: parallelism, asynchronous programming, backpressure, and message-driven design
  • Familiarity with message queues, orchestration, and distributed task systems (Kafka, SQS, Airflow, etc.)
  • Experience working on large-scale data ingestion, crawling, or indexing systems
  • Experience evaluating and monitoring data quality, ensuring consistency, completeness, and reliability across releases
  • Strong grasp of software architecture and backend fundamentals; you can reason clearly about concurrency, scalability, and fault tolerance
  • Proficiency with Linux / Unix command-line tools and system resource management
  • Experience designing or maintaining resilient data ingestion, API integration, or ETL systems
  • Solid programming experience in Python, Go, Rust, or similar, including experience with async / await, coroutines, or concurrency frameworks
  • Work independently in a fast-paced, remote-first environment, proactively unblocking themselves and collaborating asynchronously
  • Experience with cloud platforms (AWS preferred, GCP or Azure also great)
  • Experience with Apache Spark, Databricks, or other distributed data platforms
  • Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, http request / response)
  • 7+ years of professional experience building or operating backend or infrastructure systems at scale
  • Write and maintain technical design documents, including pipeline design, schema design, and data flow diagrams
  • Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
  • Experience building and maintaining data pipelines on modern big-data or cloud platforms (Databricks, Spark, or equivalent)
  • Understanding of modern data storage and design patterns (parquet, Delta Lake, partitioning, incremental updates)

Skills

What Senior Software Engineering jobs ask for — and how much of it you have →
Apply

See also

Software Engineering jobs by country — openings, pay and top skills →