PySpark Big Data Engineer
Description
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.
Qualification:
- At least 4+ years’ experience in Python, SQL and PySpark.
- Proficiency in distributed data processing and big data tools and technologies: Hadoop, HDFS, YARN, Spark, Hive
- API's and backend development (FASTAPI / FLASK)
- Devops: GIT, CI/CD basics, Linux/Unix commands
- Design, develop, and maintain big data pipelines using PySpark
- Experience with integration of data from multiple data sources
- Experience in DevTools like openshift, teamcity, uDeploy, BitBucket, GitHub
- Proactively contribute to stability of overall production systems and troubleshoot key components and processes
- Keep track of latest technology trends and proactively learn new technologies driving practical and reference implementations
Nice to have:
Experience with NoSQL databases, such as ElasticSearch
understand of LLMs, prompt engineering, Data qyery systems (NL to SQL), basic knowledge of vector databases (FAISS, pinecone etc)
Qualifications:
Bachelor’s degree (preferably in technology /engineering or related field).
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
------------------------------------------------------
Job Family Group:
Technology------------------------------------------------------
Job Family:
Applications Development------------------------------------------------------
Time Type:
Full time------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.