Software Engineer Experts
Summary
Part-time, contract-based remote role for engineers to design and build Terminal-Bench tasks — realistic terminal environments, setup scripts, and verification tests — used to evaluate and improve AI coding agents. Core stack: Python, Linux/shell scripting, Git, and Docker.
Compensation: ₹40,000 – ₹55,000 • No equity
About HumanizedText × Neutals
HumanizedText is a subsidiary of Neutals and is working closely with Neutals AI Lab on experimental AI projects. Neutals is building infrastructure and workflows for high-quality AI training data, specializing in data annotation, AI-ready datasets, and technical solutions that make AI development faster, more reliable, and more scalable.
About the Role
We are looking for technically strong engineers who can contribute to Terminal-Bench task generation and development. You will work on a contract basis to help us build a batch of high-quality, challenging, and verifiable tasks designed to evaluate and improve AI agents' capabilities in real-world software engineering and terminal environments.
This role involves hands-on engineering, developing realistic task scenarios, creating execution environments, implementing verification mechanisms, and ensuring tasks meet the required quality standards.
What You'll Work On
Design and develop Terminal-Bench tasks based on project specifications and evaluation requirements.
Build realistic terminal-based environments involving software engineering, debugging, scripting, infrastructure, and data processing.
Develop task instructions, environment configurations, setup scripts, and required input files.
Create robust test cases and verification scripts to evaluate task completion.
Ensure tasks are reproducible, technically accurate, and aligned with defined benchmarks.
Debug environment issues, validate task execution, and resolve test failures.
Review and improve tasks based on quality checks and project feedback.
Collaborate with the founding team to deliver a batch of high-quality benchmark tasks within agreed timelines.
**What We're Looking For **
Strong programming and software engineering fundamentals.
Hands-on experience with Linux, terminal commands, shell scripting, and debugging.
Proficiency in Python or another relevant programming language.
Familiarity with Git, Docker, command-line tools, and development environments.
Ability to design realistic technical problems with clear requirements and verifiable outcomes.
Strong problem-solving skills and attention to detail.
Ability to follow technical specifications and deliver independently.
Willingness to work on a contract basis with defined deliverables and deadlines.
Nice to Have
Experience with Terminal-Bench, benchmark development, or AI agent evaluation.
Familiarity with software engineering agents and coding evaluation frameworks.
Experience with Docker-based environments, automated testing, and CI/CD.
Knowledge of databases, distributed systems, APIs, networking, or cloud infrastructure.
Experience creating reproducible technical challenges, coding tasks, or developer tooling.
Familiarity with reinforcement learning, synthetic data generation, or AI training pipelines.
Who Should Apply?
We welcome applications from fresh graduates, independent developers, software engineers, competitive programmers, and experienced technical contributors who enjoy solving complex engineering problems.
You do not need prior Terminal-Bench experience if you have strong technical fundamentals and can demonstrate your ability to build, debug, and validate software systems.
Engagement Details
Location: Remote
Employment Type: Part-time, Contract-Based
Engagement: Project-based, with a defined batch of tasks and deliverables
Quick Apply
If you're interested in building challenging technical tasks that contribute to AI agent evaluation and development, we would love to hear from you.
Apply through the Neutals Experts portal:
https://experts.neutals.com
Join us in building high-quality technical benchmarks for the next generation of AI systems.
