Python QA Lead📣 Job Ad

About the Role

YO IT Consulting is seeking a Python QA Lead for an hourly, remote contractor role. This position involves overseeing quality, consistency, and trainer performance across Python AI training projects. The successful candidate will review AI-generated Python code and trainer/QA work, evaluate output quality against project guidelines, and provide precise written feedback. Candidates should possess 5-10 years of relevant experience in a similar capacity.

Role Context and Impact

This role is with a fast-growing AI Data Services company that delivers training data for major AI companies and foundation-model labs. The Python QA Lead will directly contribute to improving premier AI models by ensuring Python training data is accurate, executable, idiomatic, clearly explained, well-documented, and aligned with client expectations. This includes assessing work for code correctness, runtime behavior, debugging accuracy, readability, maintainability, performance, security awareness, test coverage, formatting, and instruction adherence.

Key Responsibilities

• Spot-check Python items, identify quality issues, provide feedback, and escalate recurring or critical issues.
• Review AI-generated Python code, debugging responses, algorithms, backend snippets, automation scripts, tests, and explanations.
• Update trainers and QAs on Discord regarding guidelines, workflow updates, and Python-specific quality expectations.
• Respond to questions concerning Python syntax, runtime behavior, exceptions, package usage, testing, typing, security, performance, and rubric interpretation.
• Engage inactive contributors, encourage activation, track follow-ups, and flag availability issues.
• Create and maintain Python documentation, style guides, trackers, FAQs, examples, honeypots, and onboarding materials.
• Conduct onboarding and training calls for Python contributors.
• Flag misleading, non-executable, insecure, inefficient, or non-production-ready Python recommendations.
• Identify recurring quality gaps and improve Python QA workflows.

Candidate Profile

• Strong understanding of Python fundamentals, including data structures, functions, classes, modules, exceptions, comprehensions, iterators, generators, decorators, context managers, virtual environments, packaging, and testing.
• Ability to evaluate Python content against rubrics and identify issues such as incorrect logic, non-executable code, poor exception handling, inefficient algorithms, unsafe file/network operations, hallucinated APIs, or incomplete explanations.
• Experience leading or supporting remote teams of trainers, engineers, reviewers, annotators, educators, or QAs is strongly preferred.
• Proficiency with Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Highly organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and documentation.

Work Arrangement

This is an hourly, remote contractor position. While the role is remote, candidates based in Jeddah, Makkah, Saudi Arabia are encouraged to apply. The role requires consistent work and engagement within a remote team environment, ensuring contributors follow expected quality standards.

Application Process

The selection process for this role includes an AI interview, a domain-specific task, and an interview with a recruiter. There is currently no immediate project for this role; however, qualified candidates will be among the first experts contacted when relevant opportunities arise. This also provides access to future projects available through our expert network.