Software Engineer (Developer, OpenSearch)

BT-391 - Software Engineer (Developer, OpenSearch)

Location- Chantilly

Job Description

The Software Engineer will design, develop, code, test, and debug complex software products to be deployed in a mission critical environment. The ideal candidate is a hands-on developer with significant experience in scalable data platforms, cloud-based services, and big data analytics, and OpenSearch or Elasticsearch. They must have strong, firsthand technical expertise in a variety of data ingestion/computation technologies and the proven ability to fashion robust scalable solutions that can manage large data sets. They must be at ease working in an agile environment with a cross-functional team. This person should embody a passion for innovation and continuous improvement.

Key Responsibilities:

• Work with a team to write new cloud-based applications and tools and recommend new technologies and methodologies to improve system performance and usability.
• Design, develop, test, debug, and deploy new software tools that extend or replace existing capabilities to parse raw data for analysis.
• Collaborate cross-functionally with data scientists, analysts, project managers and other engineers and technical SMEs.
• Deploy new software into an AWS-based data ingest pipeline.
• Write software that interacts with multiple data types and stores data into S3 buckets that is queried by processes running Athena or OpenSearch.
• As an OpenSearch expert, drive the design of indexing strategies, query optimizations, and maintain overall architectural ownership of the OpenSearch data layer.
• Troubleshoot complex problems and provide support to mission areas for software systems and application issues.
• Maintain good documentation for new software written and new procedures to be followed.
• Provide recommendations for continuous improvement.
• Work alongside other engineers on the team to sustain and advance our organization’s capabilities.

Required Qualifications:

• Demonstrated hands-on experience with Python, Git, and unit testing frameworks.
• Demonstrated hands-on experience with building APIs and services using REST.
• Demonstrated hands-on experience with OpenSearch or Elasticsearch.
• Demonstrated hands-on experience with AWS CDK 1 and/or 2 or Terraform.
• Demonstrated hands-on experience applying object-oriented design principles and patterns to architect scalable, maintainable, and testable software systems in production environments. Demonstrated hands-on experience designing and developing applications using core AWS services, including EC2, S3, IAM, and VPC

Desired Qualifications:

• Demonstrated hands-on experience building event-driven or serverless architectures using AWS services such as Lambda, API Gateway, and SQS or SNS
• Demonstrated hands-on experience implementing data pipelines or analytics workflows using AWS-native services (Glue, Athena, EMR, S3)
• Demonstrated experience optimizing AWS costs using best practices such as right-sizing, storage tiering, and workload-aware architecture decisions.
• Demonstrated hands-on experience with PRESTO, Trino, or Apache Spark
• Demonstrated hands-on experience with reverse proxy solutions e.g NGINX
• Demonstrated hands on experience working with DynamoDB, RDS, or HIVE.
• Demonstrated hands-on experience with Parquet and/or ORC file formats.
• Demonstrated hands-on experience working with Hadoop, Apache Spark and their related ecosystems.
• Demonstrated hands-on experience building and managing ETL pipelines using orchestration frameworks like Dagster, Airflow, or NiFi.
• Demonstrated hands-on DevOps experience to include creation and deployment of container-based applications
• Demonstrated hands-on experience working with cloud formation for automation.
• Demonstrated hands-on experience building and deploying software within a CI/CD pipeline.
• Demonstrated hands-on experience with Linux-based systems and shell scripting.

Education and Experience:

• Bachelor's degree in Computer Science, Information Technology, or other related technical discipline, or equivalent combination of education, technical certifications, training, and work/military experience.