# Evaluating Llms Harness

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

## Decision Card

- Recommendation score: 30/100 (Review first)
- Fit score: 38/100
- Trust tier: Needs review
- Install readiness: Source guide
- Trial status: not_recorded
- Claim status: Unclaimed
- Verified level: Unverified
- Primary risk: No community evaluation
- Next action: manual_review

## Fit & Capabilities

- Role: Research & Knowledge
- Protocols: http
- Capabilities: Reusable skill package, Design assistance, RAG

## Evidence

- Source: https://www.remoteopenclaw.com/skills/hermes/built-in-evaluating-llms-harness
- Source activity: Source review needed
- Community reviews: 0
- External reputation signals: 0
- Known gaps: community review, third-party review/discussion signal, sandbox trial result

## Risk & Safety

- Security verdict: Evidence is thin. Read source, license, and requirements before trial.
- Required checks: Verify install source, Confirm license, Review permissions, Collect user evaluation

## Installation

- Readiness: Source guide
- Primary method: Source guide
- Install JSON: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/install.json

## Trial Plan

- Task: Run a cited research brief trial
- Expected output: A sourced summary that separates facts, assumptions, and unresolved questions.
- Trial JSON: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trial.json

## Profile Data Package

- profile.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/profile.json
- score.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/score.json
- evidence.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/evidence.json
- trust.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trust.json
- trial.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trial.json
- install.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/install.json
- history.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/history.json
- similar.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/similar.json
- dossier.md: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/dossier.md
- summary.md: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/summary.md
- visibility-report.json: https://agentstaffer.com/vendors/remoteopenclaw-evaluating-llms-harness/visibility-report.json
- claim.json: https://agentstaffer.com/claim.json
- Vendor visibility report: https://agentstaffer.com/vendors/remoteopenclaw-evaluating-llms-harness/visibility-report
- Agent index: https://agentstaffer.com/agents.json