Senior Software Engineer AI Evaluation Coding Agents

The work

You will evaluate AI-generated code and coding-agent behavior using real-world software repositories. Your reviews will help AI systems improve through structured human judgments and clear technical feedback.

• Review model outputs and source materials against project guidelines.
• Apply evaluation rubrics and make evidence-based judgments.
• Explain why an implementation is correct or incorrect.
• Document technical decisions in clear written feedback.
• Help maintain consistent quality across repeated review tasks.
• Support evaluation processes for AI coding agents.

What it pays and takes

This is a part-time contractor role. The work requires strong real-world software engineering knowledge, but prior AI training experience is not required.

• Pay: $100 to $150 per hour.
• Time: 20 or more hours per week.
• Location: Open to candidates in the United States and Canada.
• Language: Professional fluency in English.
• Experience level: Entry level.
• Experience: At least 5 years of hands-on software engineering experience.
• Programming: Strong proficiency in Python, TypeScript or JavaScript, Go, or another major production language.
• Codebases: Experience working in substantial real-world codebases.
• Core skills: Strong code review, technical judgment, and written communication.
• Technical knowledge: Familiarity with modern large language models or AI coding tools.
• Helpful experience: LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that generate and understand content, including code. Reviewers use subject knowledge to assess examples and explain what good model behavior looks like, which is why experienced software engineers are needed for specialist projects.