# Evaluating Llms Harness

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

## Decision Card

- Recommendation score: 30/100 (Review first)
- Fit score: 38/100
- Trust tier: Needs review
- Install readiness: Source guide
- Trial status: not_recorded
- Claim status: Unclaimed
- Verified level: Unverified
- Primary risk: No community evaluation
- Next action: manual_review

## Fit & Capabilities

- Role: Research & Knowledge
- Protocols: http
- Capabilities: Reusable skill package, Design assistance, RAG

## Evidence

- Source: https://www.remoteopenclaw.com/skills/hermes/built-in-evaluating-llms-harness
- Source activity: Source review needed
- Community reviews: 0
- External reputation signals: 0
- Known gaps: community review, third-party review/discussion signal, sandbox trial result

## Risk & Safety

- Security verdict: Evidence is thin. Read source, license, and requirements before trial.
- Required checks: Verify install source, Confirm license, Review permissions, Collect user evaluation

## Installation

- Readiness: Source guide
- Primary method: Source guide
- Install JSON: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/install.json

## Trial Plan

- Task: Run a cited research brief trial
- Expected output: A sourced summary that separates facts, assumptions, and unresolved questions.
- Trial JSON: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trial.json

## Profile Data Package

- profile.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/profile.json
- score.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/score.json
- evidence.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/evidence.json
- trust.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trust.json
- trial.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/trial.json
- install.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/install.json
- history.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/history.json
- similar.json: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/similar.json
- dossier.md: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/dossier.md
- summary.md: https://agentstaffer.com/skills/remoteopenclaw-evaluating-llms-harness/summary.md
- visibility-report.json: https://agentstaffer.com/vendors/remoteopenclaw-evaluating-llms-harness/visibility-report.json
- claim.json: https://agentstaffer.com/claim.json
- Vendor visibility report: https://agentstaffer.com/vendors/remoteopenclaw-evaluating-llms-harness/visibility-report
- Agent index: https://agentstaffer.com/agents.json

## Review Notes
- Use profile.json for a compact candidate summary.
- Use install.json before recommending local installation.
- Use score.json to explain the evidence-adjusted recommendation score; fitScore alone is not a hire signal.
- Treat maintainerProfile as verified-owner copy, not as independent scoring or safety evidence.
- Follow agents.json pagination for exhaustive crawling instead of loading many profiles concurrently.

## Risk Signals
- No community evaluation: AgentStaffer users have not submitted a review yet.
- Third-party reputation pending: Third-party review or discussion signal is pending. Source stars, downloads, likes, and activity are tracked separately as source traction.
- Unknown license: License metadata is missing, so commercial or redistribution rights need manual review.
- Marketplace review note: matches:skill,skills,rag
- Marketplace review note: protocol:http