Strategy Consultant AI Training & Evaluation📣 Job Ad

About Mindrift

Mindrift, powered by Toloka AI, supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks. The company connects top domain experts with cutting-edge AI initiatives, leveraging Toloka’s expertise in scalable data generation, crowd technology, and applied ML systems.

The Role: Strategy Consultant - AI Training & Evaluation

Mindrift is launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. As a Strategy Consultant - AI Training & Evaluation, you will join a growing team of consultants from leading strategy firms to convert authentic project experience into end-to-end examples for AI systems. This is a freelance, remote, project-based, individual-contributor role focused on analytical design and evaluation.

Key Responsibilities

• Build realistic consulting project environments, creating detailed scenarios grounded in real engagement dynamics, including industry context, financials, constraints, conflicting inputs, and incomplete information.
• Design structured consulting tasks for AI agents, breaking projects into discrete tasks that mirror real consulting work such as market sizing, commercial due diligence, cost optimization, growth strategy, and operational diagnosis.
• Define evaluation criteria and quality standards, developing grading frameworks, evaluation rubrics, and golden-answer solutions for each task. These will be used to train and calibrate an LLM-based grading system that evaluates AI outputs at scale.

Required Qualifications and Experience

This role is exclusively for consultants with direct project experience at a top-tier strategy consulting firm. Candidates must possess 3+ years of experience at one of the eligible firms listed below, with hands-on project experience in:

• Structuring ambiguous client problems into workable analytical plans.
• Building financial models, market analyses, or synthesizing findings from complex inputs.
• Producing client-ready deliverables under time pressure.
• Forming and defending recommendations under uncertainty.

A deep technical background is not required for this position.

Eligible Consulting Firms

To ensure the domain is built by practitioners trained to the highest standards of structured problem-solving and client delivery, applicants must have hands-on project experience at one of the following firms:

• McKinsey & Company
• Boston Consulting Group (BCG)
• Bain & Company
• Oliver Wyman
• Roland Berger
• Monitor Deloitte (Deloitte S&C)
• EY-Parthenon
• Kearney
• Strategy& (PwC)

Application Note

Candidates without direct project experience at one of the firms listed above are advised not to apply, as this is a mandatory requirement for the role.