Big Data Engineer
Summary
Builds and optimizes scalable big data pipelines using Python, SQL, Kafka, and cloud tools like AWS and Terraform.
Developed scalable big data pipelines using Python, SQL, Kafka, and cloud technologies. Built and optimized ETL/ELT workflows for high-volume transactional and enterprise data. Designed data models and databases using SQL Server, PostgreSQL, and Oracle. Implemented real-time data processing and event-driven pipelines using Kafka. Developed data validation, reconciliation, and quality frameworks to ensure accurate datasets. Automated cloud deployments and data infrastructure using AWS, Terraform…