SSE- Data Engineer
Data Pipeline Development: Design and develop scalable data pipelines to process and transform large volumes of structured and unstructured data. • Data Processing: Build and optimize data processing frameworks using Apache Spark. • Data Integration: Integrate data from multiple sources to create reliable and efficient data workflows. • Data Management: Ensure efficient storage, retrieval, and transformation of data for analytics and reporting. • System Performance: Monitor and optimize data pipelines for performance, scalability, and reliability. • Troubleshooting: Identify and resolve issues in data pipelines and processing systems to ensure data accuracy and availability. • Collaboration: Work closely with data scientists, backend engineers, and product teams to understand data requirements and deliver solutions. • Documentation: Maintain clear documentation of data pipelines, data models, and workflows. • Continuous Improvement: Stay updated with emerging technologies and best practices in big data and data engineering
Roles and Responsibilities
Data Pipeline Development: Design and develop scalable data pipelines to process and transform large volumes of structured and unstructured data. • Data Processing: Build and optimize data processing frameworks using Apache Spark. • Data Integration: Integrate data from multiple sources to create reliable and efficient data workflows. • Data Management: Ensure efficient storage, retrieval, and transformation of data for analytics and reporting. • System Performance: Monitor and optimize data pipelines for performance, scalability, and reliability. • Troubleshooting: Identify and resolve issues in data pipelines and processing systems to ensure data accuracy and availability. • Collaboration: Work closely with data scientists, backend engineers, and product teams to understand data requirements and deliver solutions. • Documentation: Maintain clear documentation of data pipelines, data models, and workflows. • Continuous Improvement: Stay updated with emerging technologies and best practices in big data and data engineering