HL Software Engineer/Opensearch Developer
Posted 1
view
Summary
Probity is hiring a software engineer in Chantilly, VA (active TS/SCI with polygraph required) to design, build, test, and deploy cloud-based data tools on AWS. Day to day they write Python services, deploy into an AWS data ingest pipeline storing data in S3 and queried via Athena/OpenSearch, and own the OpenSearch data layer including indexing strategy and query optimization.
PLEASE NOTE: This position requires an ACTIVE Top Secret/SCI Clearance with Polygraph. To be considered for this position, you MUST have an ACTIVE Clearance Level of Top Secret/SCI with Polygraph
Position Code: 13-SO0825-1
JOB DESCRIPTION
The Software Engineer will design, develop, code, test, and debug complex software products to be deployed in a mission critical environment. The ideal candidate is a hands-on developer with significant experience in scalable data platforms, cloud-based services, and big data analytics, and OpenSearch or Elasticsearch. They must have strong, firsthand technical expertise in a variety of data ingestion/computation technologies and the proven ability to fashion robust scalable solutions that can manage large data sets. They must be at ease working in an agile environment with a cross-functional team. This person should embody a passion for innovation and continuous improvement.
KEY RESPONSIBILITIES
- Work with a team to write new cloud-based applications and tools and recommend new technologies and methodologies to improve system performance and usability.
- Design, develop, test, debug, and deploy new software tools that extend or replace existing capabilities to parse raw data for analysis.
- Collaborate cross-functionally with data scientists, analysts, project managers and other engineers and technical SMEs.
- Deploy new software into an AWS-based data ingest pipeline.
- Write software that interacts with multiple data types and stores data into S3 buckets that is queried by processes running Athena or OpenSearch.
- As an OpenSearch expert, drive the design of indexing strategies, query optimizations, and maintain overall architectural ownership of the OpenSearch data layer.
- Troubleshoot complex problems and provide support to mission areas for software systems and application issues.
- Maintain good documentation for new software written and new procedures to be followed.
- Provide recommendations for continuous improvement.
- Work alongside other engineers on the team to sustain and advance our organization’s capabilities.
EDUCATION AND EXPERIENCE
- Bachelor's degree in Computer Science, Information Technology, or other related technical discipline, or equivalent combination of education, technical certifications, training, and work/military experience.
REQUIRED QUALIFICATIONS
- Demonstrated hands-on experience with Python, Git, and unit testing frameworks.
- Demonstrated hands-on experience with building APIs and services using REST.
- Demonstrated hands-on experience with OpenSearch or Elasticsearch.
- Demonstrated hands-on experience with AWS CDK 1 and/or 2 or Terraform.
- Demonstrated hands-on experience applying object-oriented design principles and patterns to architect scalable, maintainable, and testable software systems in production environments.
- Demonstrated hands-on experience designing and developing applications using core AWS services, including EC2, S3, IAM, and VPC
DESIRED QUALIFICATIONS
- Demonstrated hands-on experience building event-driven or serverless architectures using AWS services such as Lambda, API Gateway, and SQS or SNS
- Demonstrated hands-on experience implementing data pipelines or analytics workflows using AWSnative services (Glue, Athena, EMR, S3)
- Demonstrated experience optimizing AWS costs using best practices such as right-sizing, storage tiering, and workload-aware architecture decisions.
- Demonstrated hands-on experience with PRESTO, Trino, or Apache Spark
- Demonstrated hands-on experience with reverse proxy solutions e.g NGINX
- Demonstrated hands on experience working with DynamoDB, RDS, or HIVE.
- Demonstrated hands-on experience with Parquet and/or ORC file formats.
- Demonstrated hands-on experience working with Hadoop, Apache Spark and their related ecosystems.
- Demonstrated hands-on experience building and managing ETL pipelines using orchestration frameworks like Dagster, Airflow, or NiFi.
- Demonstrated hands-on DevOps experience to include creation and deployment of container-based applications
- Demonstrated hands-on experience working with cloud formation for automation.
- Demonstrated hands-on experience building and deploying software within a CI/CD pipeline.
- Demonstrated hands-on experience with Linux-based systems and shell scripting.
Skills
- Agile
- Airflow
- Analytics
- API
- Athena
- Automation
- AWS
- Bash
- CDK
- CI/CD
- Cloud
- Dagster
- Data Analytics
- Data Ingestion
- Data Pipelines
- DevOps
- DynamoDB
- EC2
- Elasticsearch
- ETL
- Event Driven Architecture
- Git
- Hadoop
- Hive
- IAM
- Lambda
- Linux
- Nginx
- NiFi
- OOP
- OpenSearch
- Parquet
- Python
- RDS
- S3
- Serverless
- SNS
- Spark
- SQS
- Terraform
- Trino
- Unit Testing
- VPC