AI Workflow Evaluator - Fully Remote | Upto $190/hr

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: MCP & Plug In Connectors Expert Type: Contract Compensation: $50–$190/hour Location: Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal health , travel , and activity planning . Design realistic prompts for complex personal-life tasks using Google Drive , Expedia , Notion , and other plugins/connectors. Execute tasks while recording your screen and using personal plugins/connectors. Write clear explanations of AI successes and failures to improve AI model performance . Judge AI outputs for practicality, personalization, and reasoning. Identify where models miss context, overreach, or fail to use tools correctly. Qualifications Must-Have US-based only. Strong MCP experience and plugin/connector usage. Experience using LLM plugins/connectors like Google Drive , Expedia , and Notion . Heavy personal usage of LLM products. Active LLM account with 6+ months of history. Willingness to sign a data-share consent form via DocuSign . Experience using AI for multi-step planning, research, and decision-making. Strong written judgment and attention to detail. Ability to evaluate AI outputs and write against rubrics. Preferred Extensive rubric experience with 100+ hours on prior projects. Compensation & Legal Hourly contractor . Experts added to the project will begin in a trial period. Application Process (Takes 20–30 mins to complete) Upload resume AI interview based on your resume Submit form Resources & Support For details about the interview process and platform information, please check: For any help or support, reach out to: PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity. Originally posted on Himalayas