About the Role
This is a top-priority role on the Engineering team at an early-stage AI infrastructure company focused on reinforcement learning environments and post-training data. You will own the automation of quality control for training data produced by external data vendors, building systems that scale quality assurance as demand grows. The team is a small, high-calibre group of researchers and engineers working at the frontier of AI alignment.
What You'll Do
-
Automate QC for training data created by companies using the platform's infrastructure.
-
Build QC systems grounded in human judgment and true understanding, rather than heavy reliance on LLMs.
-
Define and enforce quality standards for RL training data.
-
Design experiments and metrics to grade agent outputs.
-
Partner with data vendors to debug quality issues, diagnose agent failure modes, and improve data generation processes.
-
Translate QC learnings into auditing systems, including sampling strategies and rule-based or model-assisted validation pipelines.
-
Continuously integrate QC learnings into infrastructure tools and the vendor portal to reduce anomalies, inconsistencies, and edge cases.
What We're Looking For
-
2 to 4 years of experience in research engineering or a similar role focused on QC automation.
-
Proficiency in Python, Docker, and Linux environments.
-
Proven experience building scalable data validation pipelines and automated QA/QC systems end-to-end, without a fully prescribed roadmap.
-
Experience working on benchmarks and evaluations for RL training data, including defining realistic tasks, reliable rubrics, and useful trajectories.
-
Experience creating QC systems based on human judgment rather than LLM-heavy approaches.
-
Experience designing experiments and metrics to grade agent outputs.
-
Experience partnering with data vendors to debug quality issues and provide actionable feedback.
-
Strong knowledge of statistics and comfort designing metrics and QA/QC processes.
-
Strong written and verbal communication skills for collaborating across time zones.
-
Ability to work autonomously and thrive in unstructured, fast-paced early-stage environments.
Compensation & Benefits
Salary range: $150,000 to $250,000 USD annually. Visa sponsorship is available.
Location
On-site in Singapore. Candidates based in San Francisco or working remotely as an independent contractor, particularly from Europe, may also be considered.