【JAPAN AI】AI Quality Scientist / English

800万円〜1,600万円

東京都

As an AI Quality Scientist at JAPAN AI, you will lead research and implementation of AI agent quality evaluation. Key responsibilities include:

1. Evaluation Metric Research & Development
- Research LLM-as-Judge calibration methods
- Design and validate evaluation benchmarks
- Research reward modeling and preference learning
- Select and design evaluation metrics
- Design and maintain evaluation datasets

2. Automated Evaluation Pipeline Design & Development
- Implement scalable automated evaluation pipelines
- Integrate evaluation pipelines into CI/CD
- Design agent evaluation harnesses
- Ensure reproducibility of evaluation processes

3. Safety & Quality Verification
- Implement automated red teaming
- Build safety and policy compliance verification frameworks
- Research hallucination detection methods
- Design and execute prompt/tool regression tests

4. Statistical Analysis & Experimental Design
- Design and analyze statistical experiments
- Visualize quality trends
- Automate regression detection
- Create quality improvement proposals
- Feed evaluation signals back to R&D teams

The role focuses on establishing "AI Evaluation Science" within the context of Japanese enterprise AI, scientifically guaranteeing the quality of AI products used by approximately 200 companies.

株式会社ジーニー

東京都新宿区西新宿6-8-1住友不動産新宿オークタワー5/6階

正社員

健康保険 雇用保険 厚生年金 労災保険

完全週休二日制 所定休日：土・日・祝日 休暇：年次有給休暇、夏季休暇（3日）、年末年始休暇（12月31日?1月3日）、慶弔休暇

2026/04/13