Data Engineer
Posted
Summary
Senior Data Engineer at Remotestar who designs, builds, and maintains scalable data pipelines and distributed data processing solutions. Core stack is Python, PySpark, and SQL on AWS cloud data services, with ETL/ELT work across structured and unstructured data.
Required Qualifications & Skills – Senior Data Engineer
We are seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and distributed data processing solutions.
Mandatory / Primary Skills:
- Python
- PySpark
- AWS
- SQL
Experience
- 4 to 7 years of hands‑on experience in Data Engineering and building scalable data solutions.
- Strong experience working with AWS-based data engineering services and cloud data platforms.
Technical Skills
- Strong hands‑on expertise in Python and PySpark for large-scale data processing.
- Advanced proficiency in SQL, including writing complex queries, query optimization, and data analysis.
- Strong experience with AWS services relevant to data engineering and cloud-based data processing.
- Strong understanding of Apache Spark and distributed data processing systems.
- Experience designing, developing, and maintaining scalable and high-performance ETL/ELT data pipelines.
- Experience in data ingestion, transformation, integration, and processing from multiple data sources.
- Experience working with structured, semi-structured, and unstructured data such as JSON, logs, PDFs, and other file formats.
- Strong knowledge of relational and non-relational databases.
- Experience in data quality checks, validation, performance tuning, and optimization.
Analytical & Problem-Solving Skills
- Strong ability to perform deep-dive data analysis and troubleshoot complex data-related issues.
- Proven experience in building scalable data solutions to address complex business requirements.
- Strong problem-solving and analytical skills.
Soft Skills
- Strong verbal and written communication skills.
- Good organisational and stakeholder-management skills.
- Proactive self-starter with the ability to work independently and take ownership.
Technical Agility
- Ability to adapt quickly to new technologies and data platforms.
- Passion for staying current with emerging trends and best practices in Data Engineering, AWS, and Big Data technologies.