Data Engineer
Summary
Builds and optimizes big-data pipelines using Hadoop, Spark, and SQL for a banking analytics platform, ensuring scalable ETL and reporting.
Overview
Job Title: Big Data Engineer with Strong SQL Exp
Location: Mississauga, ON (Hybrid/ 03 days onsite)
Job Type: Full Time Permanent
Responsibilities
- Experience: 7+ years
- Strong Experience in Apache Hadoop, Spark, Spark SQL, Hive, Impala, Yam, Talend. Have strong experience in SQL queries/stored procedures/functions/triggers. Spark calculators based on business logic/rules.
- Programming and reverse engineering skills with Scala, Java, Python, SQL, Unix/Linux shell, Spark RD. Understanding of OOP and Functional design approach.
- Big Data reporting, querying, and analysis. Create T-SQL queries/stored procedures/functions/triggers using SQL or Server 2014 and 2017 or any other query language.
- Should have fair ETL knowledge.
- Strong writing, communication, time-management, decision-making, and basic task organization skills.
- Knowledge of software architecture, design, and testing. Familiar with standard SDLC and Agile processes.
- Knowledge of DevOps tools like Jenkins/TeamCity/UDeploy/Harness/Ghithub, OpenShift. Knowledge of version control tools and batch control such as GIT-bash, Autosys, Control-M.
- Highly motivated, needs minimal hand holding, ability to multitask and work under pressure.
- Strong analytical and problem-solving skills; ability to analyze data for trends and quality checking.
Seniority level
- Mid-Senior level
Employment type
- Full-time
Job function
- Information Technology
Industries
- Banking