Remote Native-Language Benchmark Engineer (Freelance)
Posted Updated
Description
Lilt is seeking experienced native-speaking software engineers to design, build, and validate multilingual benchmarks. This remote, freelance role focuses on creating high-signal tasks in native languages and evaluating multilingual robustness without English translation crutches.
You’ll work on task engineering, asset creation, prompting and translation, verification scripts, and calibration across model tiers, within a rigorous 4‑layer quality control process.
Design, build and validate multilingual benchmarks. Create assets in the target language without English translation. Develop prompts and translations to reveal failure points in native language. Write deterministic verifier scripts and reference implementations. Calibrate task difficulty across model tiers and run configurations. Participate in a 4-layer quality control process for fairness and accuracy. 1+ years of industry software engineering experience. Native or near-native fluency in the target language with strong grammar and phrasing. High English proficiency. Strong Python, shell scripting and data processing skills. Experience with Terminal/CLI workflows and coding agents. Deep understanding of multilingual text processing pitfalls, including encoding/decoding robustness and Unicode normalization. Flexible schedule Remote workSkills
Sign in to read and join the discussion.