Data Engineering
Posted Updated 6
views
Summary
Senior data engineer (5–10 yrs) in Bangalore architecting scalable cloud data platforms on AWS — enterprise data lakes/hubs, Snowflake pipelines, and Spark/Hadoop big-data processing — with orchestration tools like Airflow/NiFi and CI/CD via Jenkins/GitLab.
Role: Data Engineering
Notice Period: Immediately to 15 days
Experience: 5–10 Years
Location: Bangalore
Must Have
- Experience in architecting and delivering highly scalable, distributed, cloud-based enterprise data solutions.
- Strong expertise in the end-to-end implementation of Cloud data engineering solutions like Enterprise Data Lake, Data hub in AWS.
- Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, ETL data Pipelines, Big Data model techniques using Python / Java.
- Experience in loading disparate data sets and translating complex functional and technical requirements into detailed design.
- Should be aware of deploying Snowflake features such as data sharing, events and lake-house patterns.
- Should have experience with data security and data access controls and design.
- Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modeling).
- Proficient in Lambda and Kappa Architectures.
- Strong AWS hands-on expertise with a programming background preferably Python/Scala.
- Good knowledge of Big Data frameworks and related technologies - Experience in Hadoop and Spark is mandatory.
- Strong experience in AWS compute services like AWS EMR, Glue and Sagemaker and storage services like S3, Redshift & Dynamodb.
- Good experience with any one of the AWS Streaming Services like AWS Kinesis, AWS SQS and AWS MSK.
- Troubleshooting and Performance tuning experience in Spark framework - Spark core, Sql and Spark Streaming.
- Experience in one of the flow tools like Airflow, Nifi or Luigi.
- Good knowledge of Application DevOps tools (Git, CI/CD Frameworks) - Experience in Jenkins or Gitlab with rich experience in source code management like Code Pipeline, Code Build and Code Commit.
- Experience with AWS CloudWatch, AWS Cloud Trail, AWS Account Config, AWS Config Rules.
- Strong understanding of Cloud data migration processes, methods and project lifecycle.
- Good analytical & problem-solving skills.
- Good communication and presentation skills.