Big Data Engineer- Full Time
Posted
Description
Position: Big Data Engineer
Location: Alpharetta, GA (3 days office).
Employment: Full Time
Hadoop Development:
- Design and implement Hadoop MapReduce jobs for complex data processing tasks.
- Utilize HiveQL for querying and manipulating large datasets within Hadoop.
- Leverage Spark for real-time data processing and streaming analytics when needed.
Snowflake Development:
- Design efficient data models in Snowflake to optimize query performance and data access.
- Create Snowflake views and materialized views to enhance query efficiency for data analysis.
- Implement data governance policies within Snowflake to ensure data quality and security.
- Write complex SQL queries on Snowflake to extract insights from large datasets.
Required Skills:
- Very strong in Snowflake
- Strong Hadoop migration experience
- Programming Languages: Strong proficiency in Python, Java and SQL
- Big Data Frameworks: Deep understanding of ecosystem (HDFS, MapReduce, Hive, Spark, PySpark, Snowspark)
- Cloud Data Warehousing: Expertise in Snowflake architecture, data manipulation and query optimization
- Data Engineering Concepts: Knowledge of data ingestion, transformation, data quality checks and data security practices.
- Data modeling: Ability to design efficient data models for data warehousing environments.
Skills
Sign in to read and join the discussion.