# auto-arena

Need to compare multiple LLMs or agents but don't have test data ready? This generates test queries from your task description, collects responses from all your endpoints, creates evaluation rubrics on the fly, then runs pairwise comparisons with position-bias swapping to produce win-rate rankings. It's built on OpenJudge's AutoArenaPipeline and handles the...

## Decision Card

- Recommendation score: 39/100 (Review first)
- Fit score: 59/100
- Trust tier: Needs review
- Install readiness: Source guide
- Trial status: not_recorded
- Claim status: Unclaimed
- Verified level: Unverified
- Primary risk: No community evaluation
- Next action: manual_review

## Fit & Capabilities

- Role: Coding & Development
- Protocols: http
- Capabilities: Workflow automation, Multi-agent orchestration, Coding assistance, Reusable skill package, Design assistance

## Evidence

- Source: https://crossaitools.com/skills/agentscope-ai/openjudge/auto-arena
- Source activity: 657 stars
- Community reviews: 0
- External reputation signals: 0
- Known gaps: community review, third-party review/discussion signal, sandbox trial result

## Risk & Safety

- Security verdict: Evidence is thin. Read source, license, and requirements before trial.
- Required checks: Verify install source, Confirm license, Review permissions, Collect user evaluation

## Installation

- Readiness: Source guide
- Primary method: Source guide
- Install JSON: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/install.json

## Trial Plan

- Task: Run a small code review trial
- Expected output: A concise review with reproducible findings, changed-file scope, and test suggestions.
- Trial JSON: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/trial.json

## Profile Data Package

- profile.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/profile.json
- score.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/score.json
- evidence.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/evidence.json
- trust.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/trust.json
- trial.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/trial.json
- install.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/install.json
- history.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/history.json
- similar.json: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/similar.json
- dossier.md: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/dossier.md
- summary.md: https://agentstaffer.com/skills/claudemarketplaces-auto-arena/summary.md
- visibility-report.json: https://agentstaffer.com/vendors/claudemarketplaces-auto-arena/visibility-report.json
- claim.json: https://agentstaffer.com/claim.json
- Vendor visibility report: https://agentstaffer.com/vendors/claudemarketplaces-auto-arena/visibility-report
- Agent index: https://agentstaffer.com/agents.json