Senior Data Engineer
Summary
Designs, builds, and maintains automated data pipelines that extract text from documents (PDF, Excel, Word), transform it to required formats, and load it into databases. Core stack is Python, SQL, and relational/NoSQL DBMS, with vector databases, Airflow, big data tools, and cloud platforms as pluses.
- Extract text data from various sources (PDF, Excel, Word, etc.).
- Process text data into specific formats according to requirements.
- Store processed data in designated databases.
- Build and automate data pipelines for automatic updates.
- Debugging and maintain data pipelines.
Required Qualifications:
- Minimum Associate Degree (D3) in Computer Science, Information Systems, Data Science, or a related technical field.
- At least 1 years of experience as a Data Engineer
- Strong understanding of file formats (PDF, Excel, Word, etc.) and data extraction methods
- Experience in text data processing and transformation
- Proficiency in programming languages (Python, SQL)
- Knowledge of DBMS (MySQL, PostgreSQL, NoSQL)
- Strong analytical skills and attention to detail
Preferred Qualifications:
- Knowledge of vector databases (Qdrant, Weaviate, MongoDB, etc.) and experience in using them
- Experience in building and managing data pipelines (Apache Airflow, Luigi, etc.)
- Experience with big data and related tools (Hadoop, Spark, etc.)
- Experience with cloud services (AWS, GCP, Azure)
