Senior / Data Engineer
Summary
Build and maintain AWS-based data pipelines and cloud infrastructure for Singapore’s legal sector, merging datasets into a centralized warehouse and preparing data for AI/LLM tools.
Senior / Data Engineer at GovTech
About the role The Attorney-General's Chambers is in the process of enhancing its legal frameworks and is seeking a skilled data engineer to establish a unified data infrastructure. In this role, you will serve as a crucial link between intricate legal requirements and their technical implementation, overseeing cloud environments and data pipeline management.
Key facts
- Location: Singapore
- Engagement: 2-year fixed term contract
- Team: Attorney-General's Chambers (AGC)
What you’ll do
- Design and oversee the creation of data pipelines, ensuring seamless system integrations and robust AWS cloud infrastructure.
- Merge disparate datasets into a cohesive, centralized data warehouse to enhance data accessibility and usability.
- Utilize Infrastructure-as-Code methodologies to establish secure and scalable cloud environments.
- Prepare and organize data to facilitate the use of AI tools and applications based on large language models (LLMs).
- Establish data governance protocols, including access control measures and lineage tracking to ensure data integrity.
- Convert unstructured legal documents and correspondence into structured formats that can be easily analyzed and utilized.
- Collaborate closely with legal experts, data analysts, and product development teams to align technical solutions with legal needs.
- Conduct regular assessments of data quality and integrity, implementing improvements as necessary.
- Provide technical guidance and support to team members and stakeholders on data-related issues.
- Stay updated with the latest trends and technologies in data engineering and cloud computing to continuously enhance the infrastructure.
- Document processes and workflows to ensure transparency and facilitate knowledge sharing within the team.
- Participate in strategic planning sessions to align data initiatives with the broader goals of the Attorney-General's Chambers.
Requirements
- A minimum of 5 to 8 years of experience in data engineering or cloud engineering roles.
- Strong proficiency in Python, particularly for scripting, automation, and data engineering tasks.
- Advanced knowledge of SQL, with a solid background in relational database design and modeling techniques.
- Practical experience with AWS services, including but not limited to S3, Glue, Redshift, and Lambda.
- Proven track record in building and maintaining effective data pipelines and integration processes.
- Ability to thrive in ambiguous or greenfield environments, demonstrating adaptability and problem-solving skills.
- Excellent communication abilities to effectively engage with non-technical stakeholders and senior leadership.
- Strong analytical skills to assess complex data challenges and propose effective solutions.
Nice to have
- Familiarity with Terraform or other Infrastructure-as-Code tools that enhance cloud management.
- Experience in handling unstructured or document-heavy data, particularly in legal contexts.
- Previous experience working within the public sector, legal field, or other highly regulated industries.
- Knowledge of data warehousing solutions such as Databricks, Snowflake, BigQuery, or PostgreSQL.
- Understanding of data privacy regulations and compliance standards relevant to data management.
Skills & tools
- Proficient in Python, SQL, and various AWS services (S3, Glue, Redshift, Lambda).
- Experienced with Infrastructure-as-Code practices, data governance frameworks, and natural language processing (NLP) tools.
- Familiar with large language model (LLM) applications and their integration into data workflows.
Practical notes
- This role is offered as a 2-year fixed term contract, providing stability and the opportunity for impactful work.
- GovTech offers a competitive compensation package that includes wellness initiatives and flexible working arrangements to support work-life balance.
- The organization is committed to being an equal opportunity employer, promoting inclusive hiring practices that welcome diverse candidates.
- Candidates should be prepared to engage in a collaborative environment that values innovation and continuous improvement in data management practices.
- This position is ideal for individuals looking to make a significant contribution to the modernization of legal systems through data engineering.