Versus

How it plays

1. 1A task carries a shared prompt and two candidate answers, A and B.
2. 2Read both side by side — labels hidden, sides shuffled to remove bias.
3. 3Choose A, B, or Tie, with an optional reason.
4. 4With several reviewers, the majority decides — RLHF-grade preference data.

Best for

• LLM answer-quality evaluation
• A/B of two prompts or models
• RLHF / preference data collection
• Choosing between two generated drafts

Send a task

POST to /api/v1/tasks with module: "versus" and a project API key. The verdict webhooks back to your callback_url.

{
  "module": "versus",
  "payload": {
    "prompt": "Draft a reply to this refund request",
    "a": { "label": "model-x", "text": "..." },
    "b": { "label": "model-y", "text": "..." }
  },
  "callback_url": "https://your-app/verdict"
}