Cloud Dataflow Engineer/Analyst
Description
Spatial Front, Inc. (SFI), a two-time USA Today Top Workplaces awardee and Washington Post Top Workplaces honoree, is seeking a Cloud Dataflow Engineer / Analyst to support a mission-critical enterprise data modernization initiative. In this role, you will join our Dataflow Team to serve as the critical operational bridge moving high-volume, real-time datasets from legacy on-premise infrastructure into a centralized, high-availability cloud platform.
Over the next 6–12 months, your primary focus will be supporting the ingestion, automation, and reliability of real-time data streams to power scalable data access, advanced scientific analytics, and future AI/ML capabilities across hybrid environments.
Key Responsibilities
Data Pipeline & Cloud Ingestion
- Execute and streamline large-scale data ingestion and migration processes from legacy on-premise systems to cloud environments (AWS/Azure/GCP).
- Transition legacy dataflow pipelines into automated, API-driven workflows using Python, Bash, and modern CI/CD practices.
- Maintain, refactor, and migrate legacy scripts and tools (Bash, Perl, SQL) to scalable, cloud-native solutions.
- This role offers a unique opportunity to learn the foundational legacy systems of the National Weather Service while simultaneously building the "Gold Standard" of cloud-based weather data dissemination.
Operations, Monitoring & Reliability
- Ensure continuous, low-latency delivery of mission-critical, high-volume data streams in a 24/7 operational environment.
- Monitor, troubleshoot, and optimize data routing applications, ingestion processes, and underlying cloud services.
- Serve as a Tier 2 technical escalation point for complex data ingestion, formatting, and delivery issues.
- Participate in a 24/7 on-call rotation to support mission-critical availability.
Data Lifecycle & System Integration
- Manage end-to-end data flow operations, including validation, quality control, archival, and retrieval.
- Collaborate with internal engineering teams and external stakeholders to resolve data access, formatting, and integration challenges.
- Author and maintain technical documentation, standard operating procedures (SOPs), and configuration standards.
Skills
Sign in to read and join the discussion.