Sr. Data Engineer
Posted Updated
Summary
Senior data engineer (6-8 years) in Bengaluru designing and running real-time and batch big-data pipelines using Hadoop, Hive, Spark SQL, Kafka, Flink, and Airflow. Work also spans SQL databases, graph databases, CI/CD, and cloud-native Docker/Kubernetes deployments.
Job Description
Sr. Data Engineer – Big Data
- Experience: 6-8 Years
- Location: Bengaluru, KA
Key Skills
- Big Data Technologies
- Hadoop, HDFS, Hive
- Spark / Spark SQL
- SQL
- Real-Time and Batch Data Processing
Required Skills and Experience
- Extensive experience with Apache Kafka, Apache Flink, and other streaming technologies.
- Strong expertise in Hadoop ecosystem tools such as Hive, Iceberg, and Spark SQL.
- Good understanding of Apache Airflow for orchestrating complex data workflows and pipelines.
- Proven experience designing and implementing automated data pipelines and materialized views.
- Proficiency in Python, Unix/Linux, or similar technologies.
- Strong SQL skills, including experience with Oracle, SQL Server, or similar databases.
- Experience with graph databases such as Neo4j or JanusGraph and graph query languages.
- Experience with monitoring and CI/CD tools such as Prometheus, Grafana, Jenkins, and GitHub Actions.
- Strong engineering fundamentals, including data structures, algorithms, testing (JUnit/pytest), Git, and clean coding practices.
- Experience with cloud-native technologies, including Docker and Kubernetes (deployment, networking, scaling, and troubleshooting).
- 6+ years of relevant professional experience.
- Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.