Junior Data And Automation Engineer
Engagement context
The candidate will be expected to support a pre-existing technical environment that may include:
- A search and analytics platform in the ELK / OpenSearch family, fed by structured ingestion pipelines.
- A low-code workflow automation layer used for business and operational workflows.
- Internally developed Python services and AI-assisted agents integrated with third-party SaaS APIs.
Data pipeline & search platform support
- Monitor the health of ingestion pipelines from relational data sources into the search platform; identify backpressure, stalled jobs, and mapping or schema issues.
- Reprocess failed batches and support recovery from dead-letter queues.
- Assist with index lifecycle management, templates, and routine cluster-health checks.
- Support incremental changes to dashboards and visualizations, including new fields, metric fixes, and cosmetic updates.
- Raise capacity and performance concerns to the senior engineer before they become incidents.
Workflow automation support
- Monitor scheduled and event-driven workflow executions; triage failures and retry as appropriate.
- Apply small to medium changes, including updated templates, new data fields, credential refreshes, and webhook endpoint rotations.
- Maintain workflow stability when upstream third-party APIs change behavior.
- Track queue depth, rate-limit handling, and idempotent retry behavior.
Python services & AI-assisted workflows
- Monitor execution logs, error rates, and usage metrics of internal Python services and AI-assisted workflows.
- Troubleshoot data ingestion into internal knowledge and retrieval systems.
- Apply tested configuration or prompt adjustments under senior review.
- Track and report on operational cost anomalies, including API usage and compute.
General engineering & documentation
- Write and maintain runbooks and known-good recovery procedures for every recurring incident class.
- Keep a clean changelog of production changes via pull requests where applicable.
- Participate in change windows and post-incident reviews.
- Escalate complex issues with clear, structured diagnostic context, including logs, timestamps, and reproduction steps.
Requirements
Must-have experience (1-3 years)
- Working experience with the ELK / OpenSearch family, such as Elasticsearch, OpenSearch, Kibana, OpenSearch Dashboards, Logstash, or equivalent, with at least one of them used in a real environment.
- Working experience with at least one low-code or workflow automation platform, such as n8n, Make, Zapier, Node-RED, Apache Airflow, or equivalent.
- Comfortable with a relational database, such as PostgreSQL, MySQL, or similar: able to read schemas, write SELECT queries, and investigate data issues.
- Comfortable on Linux: shell, SSH, log inspection, and process/service management.
- Docker basics: running containers, reading compose files, and inspecting logs.
- Reads and writes JSON and YAML fluently.
- Able to read and modify existing Python and JavaScript / Node.js code. The candidate is not expected to design services from scratch.
- REST API fluency: curl or Postman, headers, bearer tokens, pagination, rate-limit semantics, and webhooks.
- Git fundamentals: branches, commits, pull requests, and resolving simple merge conflicts.
- Structured troubleshooting: isolates variables, reads stack traces, bisects failures, and documents findings.
- Exposure to any commercial LLM API or vector database.
- Familiarity with common business SaaS APIs, including CRM, email, messaging, or data-enrichment platforms.
- Prior exposure to a data-engineering, operations, analytics, or security domain.