AI Benchmark Engineer, Indonesian (freelance)
LILT AI (United States, remote)
- Oct 2026 to Present
- L2:lilt-ai-2026
I design Indonesian-language tasks for Terminal-Bench, an agentic coding benchmark that evaluates frontier AI models on realistic terminal and programming work with Indonesian language and conventions.
Highlights
- Built each task end to end: native-language datasets, Dockerized environments, Python reference solutions and pytest verifiers
- Hardened verifiers with held-out data, sandboxed reruns and tamper checks
- Measured difficulty with agentic probe runs across several LLMs before submission
- Used failure analysis to separate language-understanding gaps from generic coding errors
Tech stack
Python
Docker
pytest- LLM evaluation