Senior Data Engineer
Summary
Senior Data Engineer builds and maintains a high-volume data platform (7TB every 90 minutes) using Python, SQL, PostgreSQL, and Airflow to power analytics for internal and external stakeholders.
Beam is seeking a Senior Data Engineer focused on data development to help create and maintain the data platform, infrastructure, and processes that power business intelligence for both internal and external stakeholders. The data platform processes over 7TB of data every 90 minutes, delivering complex analytics to partners and actionable data to internal teams. This role offers the opportunity to work with a wide range of open source data technologies, build new features, support existing functionality, and have a significant impact on a core part of the business. The team is highly collaborative and motivated by impact in a high-growth environment.
Responsibilities will include:
- Design and plan new platform data solutions in collaboration with the Head of Engineering, Engineering and Product teams, and internal stakeholders using RFC and Tech Spec documents
- Collaborate with peers to execute and implement high-quality software
- Assist in planning and execution of new software features with a focus on data platform requirements
- Drive execution of work from spec document to delivery through the SDLC
- Share knowledge through code reviews, pair programming, 1-1 conversations, and team trainings
- Develop subject matter expertise in the Beam data ecosystem and reporting platform, supporting Client Strategy, Business Operations, and partners
- Balance tech debt with practicality while designing solutions
- Gain knowledge of the nonprofit giving and e-commerce enablement spaces
- Foster a team culture around Beam’s values of community, inclusivity, care, accountability, and support
- Strive for continuous improvement through goal setting, feedback, and growth opportunities
You’re a great fit if you:
- Have 4+ years as a Data Engineer or Senior Data Engineer utilizing Python and SQL to develop data-intensive applications or insights, or 5+ years as a Software Engineer using Python and SQL in a primarily Python codebase
- Have experience working on production-level data systems in a collaborative team environment
- Are proficient with Python and SQL
- Are proficient with relational databases, particularly PostgreSQL
- Have experience with Apache Airflow for task orchestration
- Have experience using a Business Intelligence tool, such as Tableau
- Are familiar with development principles and processes such as observability, performance optimization, continuous integration, automated testing, and cloud infrastructure
- Are familiar with distributed computing architecture, data warehouse, and data lake storage patterns
- Have strong communication skills and can explain and advocate for technical projects to non-technical roles
Nice to have:
- Experience in an early startup
- Experience with setting up and administrating a Business Intelligence tool, like Tableau
- Experience with Apache Spark
- Experience with managing a Kubernetes cluster
- Bachelor’s degree in Computer Science or related technical or analytical field (or equivalent training, fellowship, or work experience)