BR Data Engineer

🧠 Core Skills & Knowledge Areas

1. Python

• Data manipulation: Proficiency with libraries like pandas, numpy, and pyarrow.
• Scripting & automation: Writing reusable, modular scripts for data ingestion and transformation.
• APIs: Consuming and building RESTful APIs for data exchange.
• Testing: Unit testing with pytest or unittest.

1. Cloud Platforms

• AWS / Azure / GCP: Familiarity with services like:
  • AWS: S3, Lambda, Glue, Redshift, EMR
  • Azure: Data Factory, Blob Storage, Synapse
  • GCP: BigQuery, Cloud Functions, Dataflow
• Infrastructure as Code (IaC): Tools like Terraform or CloudFormation.
• Security & IAM: Managing access and permissions.

1. Back-End Development

• Databases: SQL (PostgreSQL, MySQL), NoSQL (MongoDB, DynamoDB).
• APIs: Building data services using frameworks like Flask, FastAPI, or Django.
• CI/CD: Familiarity with Git, Docker, Jenkins, or GitHub Actions.

1. ETL / ELT Pipelines

• Pipeline orchestration: Tools like Apache Airflow, Prefect, or Luigi.
• Data transformation: Using SQL, dbt, or Python scripts.
• Batch vs Streaming: Understanding of Kafka, Spark Streaming, or Flink.
• Monitoring & Logging: Ensuring data quality and pipeline reliability.

🧰Tools & Technologies

• Programming: Python, SQL.
• Cloud: AWS, Azure, GCP.
• Orchestration: Airflow, Prefect.
• Databases: PostgreSQL, BigQuery, Redshift.
• Data Lakes: S3, Azure Data Lake.
• Containers: Docker, Kubernetes.
• Version Control: Git, GitHub/GitLab.

✅ Soft Skills & Other Requirements

• Problem-solving: Ability to debug and optimize data workflows.
• Teamwork: Collaborating with Data Scientists, Analysts, and DevOps.
  
  

Some of our perks

• Fresh fruit sometimes, spoiled fruit all the time.
• You can work from anywhere, including your home.
• Flexible hours.
• Team lunches, Bday celebrations, happy hours.
• Wellness program and company retreats.
• English lessons.
• Courses and training.