Senior Data Engineer - Data Engineering
Description
Welcome to GEP
Who We Are
GEP, with over 30 offices internationally, is the fastest growing procurement and supply chain solutions firm – consulting, software and managed services. GEP has succeeded by being smart and creative; solving complex problems and finding opportunities for the world’s largest organizations.
How You Grow at GEP
We recognize people who make a genuine difference, work hard and drive achievements. Results are noticed and rewarded. It’s how you will grow a career at GEP, and in a much shorter time frame than at other firms.
Celebrating Everyone
GEP succeeds through the ideas and creativity of our team members so we embrace people of all experiences, nationalities, abilities, cultures, races, gender identities, sexual orientations and ages. What makes you unique and different is celebrated and will help GEP stand out even more. And we are a women-founded and -owned company so our foundation is making GEP a great place to work for women, a place where women can learn, advance and give back.
What You Will Do
As a Senior Data Engineer at GEP, you will play a pivotal role in shaping our data architecture and driving strategic data initiatives to support business growth. You will lead the design and implementation of complex data systems, mentor junior team members, and collaborate with cross-functional teams to ensure our data infrastructure meets the highest standards of performance and reliability. You will:
- Lead the design, development, and optimization of scalable web scrapers for large-scale data extraction and analytics.
- Architect scraping solutions and data workflows that align with business goals, ensuring scalability, security, and efficiency using tools like Apache Airflow for orchestration.
- Collaborate with analysts, product managers, and business leaders to define scraping strategies and deliver actionable insights from web data.
- Oversee data quality initiatives, implementing robust validation and monitoring frameworks to ensure accuracy and consistency of scraped data.
- Drive the adoption of best practices in web scraping and data pipeline management, including CI/CD for scraper workflows and automation of repetitive tasks with Airflow.
- Mentor and guide junior web scraping engineers, fostering a culture of learning and technical excellence.
- Evaluate and integrate cutting-edge scraping and workflow technologies, such as Airflow for scheduling and monitoring data pipelines, to enhance our ecosystem.
- Provide technical leadership in troubleshooting critical scraping and workflow issues, optimizing system performance.
What Should You Bring
- Bachelor’s or Master’s degree in Computer Engineering, Computer Science, Information Technology, or a related field, or equivalent experience.
- Solid knowledge in data engineering, with a strong background in designing and leading data infrastructure projects.
- Fluent in English (spoken and written) is required due to collaboration with global teams. Spanish is a plus.
Other Skills
- Expert-level proficiency in programming languages such as Python for data processing and automation.
- Expertise in web scraping frameworks (e.g., Scrapy, Selenium) and data modeling for complex extractions.
- Knowledge with cloud platforms (e.g., AWS, Google Cloud, Azure) and advanced data warehousing solutions.
- Mastery of anti-scraping techniques, distributed scraping (e.g., using proxies or headless browsers), and ethical compliance.
- Strong experience with Apache Airflow for orchestrating and automating data pipelines, including scheduling and monitoring web scraping workflows.
- Eagerness to learn Big Data tools like Spark and Databricks.
- Proven ability to lead projects in agile environments, balancing technical and business priorities.
- Excellent communication and mentorship skills to guide teams and present solutions to stakeholders.
Skills
Sign in to read and join the discussion.