Data Engineer
Description
Job Description & Specification:
Post Title: Data Engineer
Work Mode: Kochi Onsite - UK Time zone
Role Overview:
We are seeking a talented and experienced Data Engineer to join our team. The ideal candidate will have expertise in technologies such as Metabases, Dbt, Stitch, Snowflake, Avo, and MongoDB. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure to support our analytics and data-driven decision-making processes.
Responsibilities:
- Designing, developing and implementing scalable data pipelines and ETL processes using tools such as Stitch and Dbt to ingest, transform, and load data from various sources into our data warehouse (Snowflake).
- Implement data modeling best practices and standards using Dbt to create and manage data models for reporting and analytics.
- Collaborating with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
- Develop and maintain dashboards and visualizations in Metabases to enable self-service analytics and data exploration for internal teams.
- Building and optimizing ETL processes to ensure data quality and integrity.
- Optimizing data processing and storage solutions for performance, scalability and reliability, leveraging cloud-based technologies.
- Implementing monitoring and alerting systems to proactively identify and address data issues.
- Implementing data quality checks and monitoring processes to ensure the accuracy, completeness, and integrity of data.
- Managing and optimizing databases (like MongoDB for performance and scalability).
- Developing and maintaining documentation, best practices, and standards for data engineering processes and workflows.
- Stay up to date with emerging technologies and trends in data engineering, machine learning, and analytics, and evaluate their potential impact on data strategy and architecture.
Requirements:
- Bachelor's or Master's degree in Computer Science.
- Minimum of 4 years of experience working as a data engineer with expertise in Metabases, Dbt, Stitch, Snowflake, Avo, MongoDB.
- Strong programming skills in languages like Python, and experience with SQL and database technologies (e.g., PostgreSQL, MySQL, MongoDB).
- Hands-on experience with data integration tools (e.g., Stitch), data modeling tools (e.g., Dbt), and BI platforms (e.g., Metabases).
- Experience with cloud platforms such as AWS.
- Strong understanding of data modeling concepts, database design, and data warehousing principles
- Experience with big data technologies and frameworks (e.g., Hadoop, Spark, Kafka) and cloud-based data platforms (e.g., AWS EMR, Azure Databricks, Google BigQuery).
- Familiarity with data integration tools, ETL processes, and workflow orchestration tools (e.g., Apache Airflow, Apache NiFi).
- Excellent problem-solving skills and attention to detail.
- Strong communication skills with the ability to work effectively in a global team environment.
- Experience in the education or Edtech industry is a plus.
- Knowledge of Avo for schema management and versioning will be an added advantage.
- Familiarity with machine learning algorithms, data science workflows, and analytics tools (e.g., TensorFlow, PyTorch, scikit-learn, Tableau).
- Knowledge of distributed computing concepts and containerization technologies.
- Experience with version control systems (e.g., Git) and CI/CD pipelines.
- Certifications in cloud computing (e.g., AWS Certified Developer, Google Cloud Professional Data Engineer) or data engineering (e.g., Databricks Certified Associate Developer) are desirable.
Benefits:
- Competitive salary and bonus structure based on performance and achievement of goals.
- Comprehensive benefits package including medical insurance.
Join us in shaping the future of technology by applying your expertise as a Data Engineer. If you are passionate about driving innovation and delivering impactful solutions, we invite you to be part of our dynamic team. Apply now!!
Skills:- Snowflake , Metabase, MongoDB, Data Pipelines, Amazon Web Services (AWS), Data Transformation Tool (DBT), stitch, ETL and AVOSkills
- Airflow
- Analytics
- AWS
- Azure
- Big Data
- BigQuery
- CI/CD
- Cloud
- Containerization
- Data Engineering
- Data Modeling
- Data Pipelines
- Data Quality
- Data Science
- Data Warehousing
- Databricks
- dbt
- Distributed Computing
- ETL
- GCP
- Git
- Hadoop
- Kafka
- Machine Learning
- Metabase
- MongoDB
- MySQL
- NiFi
- PostgreSQL
- Python
- PyTorch
- scikit-learn
- Snowflake
- Spark
- SQL
- Tableau
- TensorFlow
- Version Control
- Workflow Orchestration
Sign in to read and join the discussion.