Talent.com
This job offer is not available in your country.
AI Trainer - Agent Evaluation Infrastructure (MCP)

AI Trainer - Agent Evaluation Infrastructure (MCP)

MindriftIE
1 day ago
Job type
  • Quick Apply
Job description

At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We’re on the hunt for hands-on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team.

Who we're looking for :

Calling all security researchers, engineers, and penetration testers with a strong foundation in problem-solving, offensive security, and AI-related risk assessment.

We’re looking for someone who can bring a hands-on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability.

What you’ll be doing :

  • Developing and maintaining MCP-compatible evaluation servers
  • Implementing logic to check agent actions against scenario definitions
  • Creating or extending tools that writers and QAs use to test agents
  • Working closely with infrastructure engineers to ensure compatibility
  • Occasionally helping with test writing or debug sessions when needed

Although we’re only looking for experts for this current project, contributors with consistent high-quality submissions may receive an invitation for ongoing collaboration across future projects.

Requirements

  • 4+ years of Python development experience , ideally in backend or tools
  • Solid experience building APIs , testing frameworks, or protocol-based interfaces
  • Understanding of Docker, Linux CLI, and HTTP-based communication
  • Ability to integrate new tools into existing infrastructures
  • Familiarity with how LLM agents are prompted, executed, and evaluated
  • Clear documentation and communication skills - you’ll work with QA and writers
  • We also value applicants who have :

  • Experience with Model Context Protocol (MCP) or similar structured agent-server interfaces
  • Knowledge of FastAPI or similar async web frameworks
  • Experience working with LLM logs, scoring functions, or sandbox environments
  • Ability to support dev environments (devcontainers, CI configs, linters)
  • JS experience
  • Benefits

  • Get paid for your expertise, with  rates that can go up to $44 / hour  depending on your skills, experience, and project needs.
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
  • Influence how future AI models understand and communicate in your field of expertise.
  • Create a job alert for this search

    Infrastructure • IE

    Related jobs
    • Promoted
    Ai data annotation specialist

    Ai data annotation specialist

    WhatJobsIreland, Ireland, Ireland
    Company description : TELUS Digital AI Community Our global AI Community is a vibrant network of 1 million+ contributors from diverse backgrounds who help our customers collect, enhance, train, tran...Show moreLast updated: 4 days ago
    Freelance Earth Science Expert - Quality Assurance (AI Trainer)

    Freelance Earth Science Expert - Quality Assurance (AI Trainer)

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. We believe in using the power of collective intelligence to ethica...Show moreLast updated: 1 day ago
    Freelance Biology - Quality Assurance (AI Trainer)

    Freelance Biology - Quality Assurance (AI Trainer)

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. We believe in using the power of collective intelligence to ethica...Show moreLast updated: 1 day ago
    • Promoted
    Site engineer

    Site engineer

    WhatJobsRoscommon, Roscommon, Ireland
    Job title : Site Engineer Mixed Use Development (Athlone) Location : Athlone, Westmeath Project : Mixed-use Development Requirements : 3+ years Irish experience required We are seeking a motivated and d...Show moreLast updated: 12 days ago
    AI Agent Evaluation Analyst - AI Trainer

    AI Agent Evaluation Analyst - AI Trainer

    MindriftIE
    Quick Apply
    We believe in using the power of collective human intelligence to ethically shape the future of AI.The Mindrift platform, launched and powered by. AI projects from innovative tech clients.Our missio...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Ai experimentation lead

    Ai experimentation lead

    WhatJobsIreland, Ireland, Ireland
    Job Details Role Requirements Role Requirement 1 Leadership of AI Initiatives Champion the identification, prioritisation, and delivery of high-impact AI use cases across the organisation.Lead the ...Show moreLast updated: 3 hours ago
    • Promoted
    Gp img rural gp programme (icgp)

    Gp img rural gp programme (icgp)

    WhatJobsRoscommon, Roscommon, Ireland
    Job Title : GP Conversion Programme Overview Are you a doctor with 36 months of General Practice / Family Medicine experience? This is your opportunity to join a structured 2-year GP conversion progra...Show moreLast updated: 4 days ago
    Artificial Intelligence (AI) Engineers

    Artificial Intelligence (AI) Engineers

    SubstanceIE
    Quick Apply
    We’re seeking a highly skilled .Our client within the technology space is offering innovative projects that solve&nbs...Show moreLast updated: 6 days ago
    Freelance SalesForce Expert - AI Trainer

    Freelance SalesForce Expert - AI Trainer

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...Show moreLast updated: 1 day ago
    • Promoted
    Site Agent

    Site Agent

    INFORM3 RecruitmentRoscommon, Connacht, Republic of Ireland
    INFORM3 RECRUITMENT Site Agent Job Location : Roscommon Sector : Civil Inform3 are currently seeking a Site Agent for a top civils contractor working on a civils project in Roscommon.They are seeking...Show moreLast updated: 20 days ago
    • Promoted
    • New!
    AI Experimentation Lead

    AI Experimentation Lead

    PobalRepublic of Ireland, Republic of Ireland
    Job Details Role Requirements Role Requirement 1 Leadership of AI Initiatives Champion the identification, prioritisation, and delivery of high-impact AI use cases across the organisation.Lead the ...Show moreLast updated: 22 hours ago
    • Promoted
    Site Engineer

    Site Engineer

    Talent-CurveRoscommon, Connacht, Republic of Ireland
    Job title : Site Engineer Mixed Use Development (Athlone) Location : Athlone, Westmeath Project : Mixed-use Development Requirements : 3+ years Irish experience required We are seeking a motivated and deta...Show moreLast updated: 28 days ago
    • Promoted
    GP IMG rural GP programme (ICGP)

    GP IMG rural GP programme (ICGP)

    Match MedicsRoscommon, Connacht, Republic of Ireland
    Job Title : GP Conversion Programme - Train in Ireland in Just 2 Years ?? Overview Are you a doctor with 36 months of General Practice / Family Medicine experience? This is your opportunity to join a ...Show moreLast updated: 23 days ago
    • New!
    Freelance AI Trainer - Civil Engineering & Python

    Freelance AI Trainer - Civil Engineering & Python

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...Show moreLast updated: 16 hours ago
    • Promoted
    AI Data Annotation Specialist

    AI Data Annotation Specialist

    TELUS International AI Inc.Republic of Ireland, Republic of Ireland
    Company description : TELUS Digital AI Community Our global AI Community is a vibrant network of 1 million+ contributors from diverse backgrounds who help our customers collect, enhance, train, tran...Show moreLast updated: 5 days ago
    • Promoted
    Sales and Design Consultant - Roscommon

    Sales and Design Consultant - Roscommon

    Cregg RecruitmentRoscommon, Connacht, Republic of Ireland
    Sales & Design Specialist - Roscommon.CREGG are recruiting a permanent Sales & Design Specialistto join our clients Roscommon team. This role combines your strong sales abilities with creative desig...Show moreLast updated: 14 days ago
    Freelance Physics Expert with Python - AI Trainer

    Freelance Physics Expert with Python - AI Trainer

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...Show moreLast updated: 1 day ago
    Freelance AI Trainer - Mathematics & Python

    Freelance AI Trainer - Mathematics & Python

    MindriftIE
    Quick Apply
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...Show moreLast updated: 1 day ago