Sr. Data Engineer
Posted
Summary
Senior Data Engineer on Yara's Farm & Field Data Products team (Digital Ag Solutions) building and maintaining the data lake, warehouse, catalog, analytics APIs and ETL pipelines from sources like Kafka events, IoT sensors and weather APIs. Core stack: Python, SQL, Kafka, AWS and Kubernetes; currently remote/WFH from Bengaluru.
Experience: 6 to 8 years
Work Type: Remote (Currently WFH)
Employement Type: FullTime
Responsibilities:
- You shall be part of a growing team of Data Engineers, Data Scientists, Data Architect and Data Steward within Farm & Field Data Products (FFDP - Data specialization for Farm & Field initiatives within Digital Ag Solutions)
- Your core deliverables will be to build and maintain Data Lake, Data Warehouse, Data Catalog, Data & Analytics APIs, experimentation workbench for Data Science, Geospatial Analysis, and Business Intelligence
- You shall work closely with different Product and Engineering teams within DAS to build data flows, pipelines with traceable linkage, and lineage
- You shall study the input data source to develop ETL specifications for data integration development and implementation
- Input Data Source could be Kafka Events, streaming data from IoT Sensors, Weather APIs, CSV Files, Image Files, Bulk Data Stores, Batch-loading
- Proficient in data transformations on source data, perform joins and aggregations, build Data Warehouse, feature engineering for Machine Learning and Business Intelligence
- Participate in Data Profiling, Data Cleaning and Quality Management to support Data Stewards and Data Scientists in the team
- You would help run proof-of-concepts to evaluate new tools/software to recommend as potential features to Yara Farm and Field Data Products
Desired Candidate Requirements:
- Bachelor's degree in computer science, information technology, or a field related to computer science
- 6+ years experience working as a data engineer with hands-on experience in data engineering projects such as writing SQL queries, data pipelining, API provisioning, database optimization/maintenance, etc.
- Knowledge of data architecture concepts such as database systems, schemas, schema registry, data warehousing, etc.
- Worked with scalable architectures (micro-services, event-driven, data-mesh, Kubernetes, etc.) Able to code in Python, our preferred tech stack
- Experience with Kafka, Kafka Connect, KSQL is preferred
- Comfortable working with our tools: AWS, Github, Helmcharts, Jira
- Aligned with our principles: agile, customer-centric, lean, service-oriented architecture, selforganization, transparency
- Hands-on experience in metadata management, data modelling, data analysis, data integration tools
- Preferable experience in the Agriculture Technology domain
- Good communication and presentation skills
- Experience in Agile (Scrum) way of working
- Experience in working with cross-geography teams