Data Engineer - TS Clearance, Medallion Architecture
Posted Updated
Description
Creative Global Consulting is seeking an experienced Data Engineer to design scalable database architectures and modern data platforms in a United States-based environment. The role emphasizes secure, compliant data models and event-driven architectures, with hands-on work in RedShift, AWS Glue, Spark/PySpark, and CDC using DMS.
You will collaborate with cross-functional teams, optimize ETL/ELT pipelines, and ensure robust data governance and metadata management across cloud-native services.
Design scalable, secure, and high-performance database architectures to support Mirador modernization Define data models and storage strategies aligned with cloud migration goals Optimize performance, availability, and disaster recovery across environments Collaborate with IBM IVM team to identify data needs for use stories and update unified data model Leverage data lake to capture functionality-specific data Ensure compliance with data governance, security, and retention policies Document database designs, configurations, and operational procedures Active Top Secret (TS) clearance or TS eligibility with ability to complete adjudication/transfer requirements along with a Security Plus certification on day one. Active Top Secret clearance or TS eligibility with ability to complete adjudication/transfer requirements along with a Security Plus certification on day one. 7+ years of experience designing, building, and supporting enterprise data engineering solutions and modern data platforms. Strong expertise in relational databases (PostgreSQL, Oracle, SQL Server) and experience integrating semi-structured and NoSQL data sources (DynamoDB, JSON, APIs). Extensive experience designing and implementing Medallion Architecture (Bronze, Silver, Gold) and dimensional data models for analytics and operational reporting. Hands-on experience with RedShift Hands-on experience developing scalable ETL/ELT pipelines using AWS Glue, Apache Spark/PySpark, SQL, and Python. Experience implementing Change Data Capture (CDC) using AWS DMS to synchronize operational databases with analytical platforms such as RedShift. Strong knowledge of cloud-native AWS data services, including Amazon S3, AWS Glue, Redshift, Lambda, EventBridge, SQS/SNS, IAM, CloudWatch, and Secrets Manager. Experience designing metadata-driven ingestion frameworks, data lineage, and data quality validation processes. Proficiency in data modeling, schema evolution, data versioning, and master/reference data management. Strong understanding of event-driven architectures and application/database Experience optimizing SQL, Spark jobs, partitioning strategies, indexing, and Redshift Experience working within Agile/Scrum teams and collaborating with application developers, architects, and business stakeholders.Skills
- Agile
- Analytics
- API
- AWS
- Aws Glue
- Cloud
- Cloud Native
- CloudWatch
- Data Engineering
- Data Governance
- Data Lake
- Data Lineage
- Data Modeling
- Data Quality
- DynamoDB
- ELT
- ETL
- Event Driven Architecture
- Eventbridge
- IAM
- JSON
- Lambda
- Medallion Architecture
- Metadata Management
- NoSQL
- Oracle
- PostgreSQL
- PySpark
- Python
- RDBMS
- Redshift
- S3
- Scrum
- SNS
- Spark
- SQL
- SQL Server
- SQS
- TypeScript
Sign in to read and join the discussion.