Skip to review actions
~/agentstafferAgentStaffer

Find, trial, compare, and govern AI agents before hiring

Insights / Trust Lab Review evidence health Next best action: Open the highest-priority queue
Trust Lab

Public evaluation method

AgentStaffer Trust Lab publishes indexed, approved, rejected, filtered, risk, install readiness, and evaluation rubric signals for AI agent hiring. This page turns moderation and evaluation into a public trust dossier, not a hidden back-office decision.

Review flagged candidates Resolve evidence gaps Run sandbox trials
146395public profiles
Total Indexed146668Current marketplace database slice
Needs Evaluation5000Waiting for reviews or trial records
Risk Flagged5000Has at least one review or metadata risk
Filtered273Held out by public review rules
Next review actions

Start with the highest-priority candidates

Review the surfaced issue, collect missing evidence, then run a sandbox trial before promotion.

5000 need review
1
Hold For Review · MCP ready markdown2pdf.ai Manual review required
Review profile
2
Hold For Review · MCP ready Retell AI Manual review required
Review profile
3
Hold For Review · MCP ready @moneygraph/sdk Manual review required
Review profile

Trust review summary

Scan the current review state, then open detail only when you need the full method record.

Open score guide
Audit states4 statesPublication, rejection, review, and evidence states are tracked.
Evidence coverage80%Use as a starting signal, not a production approval.
Needs review5000Resolve missing evidence before promotion.
Reviewable share0%Inspect with trial-before-hire guardrails.
Audit and transparency detailsAudit statuses and Trust Transparency Packet remain available here.
Audit Statuses
Indexed146668

Captured by AgentStaffer ingestion and available for review.

Public indexed146395

Available through public review inventory unless filtered by status.

Needs review5000

Needs community evaluation, external proof, install verification, or sandbox trial evidence.

Filtered273

Rejected listings are excluded from public review indexes and counted as filtered evidence.

Trust Transparency Packet
Evidence coverage80%

Average visible evidence coverage across source, protocol, install, reviews, and trial path.

Needs human review5000

Visible listings that should not be promoted before missing evidence or risk notes are resolved.

Filtered candidates0

Rejected candidates held as negative evidence until re-reviewed.

Reviewable share0%

Candidates that can be inspected with standard trial-before-hire guardrails.

Decision method detailDecision Matrix, Evidence Coverage, Evidence Gaps, Review Policy, moderation decisions, reason codes, and resolution protocol remain available here.
Decision Matrix
Recommend after trial0

Candidate is public-indexed, but recommendation should still cite profile, install, and trial evidence.

Review profile, then trial
Hold for review5000

Candidate is visible but should not be promoted until missing evidence, install, or risk signals are cleared.

Collect evidence first
Filtered from index0

Candidate is rejected from public recommendation surfaces until re-review clears blocking reason codes.

Avoid until re-reviewed
Evidence Coverage
  • Source URL coverage100%Listings with a crawlable source, package, repository, or marketplace URL. 5000 of 146668 candidates.
  • Protocol coverage100%Listings with visible MCP, SDK, CLI, HTTP, A2A, or workflow integration hints. 4996 of 146668 candidates.
  • Install path coverage100%Listings with a non-manual install readiness state that can be turned into a trial. 5000 of 146668 candidates.
  • Review signal coverage0%Listings with AgentStaffer or external review evidence. 0 of 146668 candidates.
  • Trial path coverage100%Listings with enough visible context to propose a sandbox trial. 5000 of 146668 candidates.
Evidence Gaps
  • Manual review required5000Add profile evidence and request re-review.
  • Missing evaluation evidence5000Attach a community review, external review, or sandbox trial record.
  • License unknown4913Add license metadata before recommending production use.
  • Permission or credential risk296Document required scopes, credentials, and safe trial boundaries.
Review Policy
  • Filtered candidates should count as negative evidence until a reviewer resolves the issue.
  • Never turn a listing into a recommendation unless the answer cites profile package, setup guidance, moderation decision, and trial or review evidence.
  • Cite the profile package, setup guidance, moderation decision, reason codes, and next action before recommending install or hire.
Moderation Decisions
  • Hold For Review5000100% of moderation decisions
Reason Codes
  • Manual review required500033% of moderation reason codes
  • Missing evaluation evidence500033% of moderation reason codes
  • License unknown491332% of moderation reason codes
  • Permission or credential risk2962% of moderation reason codes
Resolution Protocol
  1. 1

    inspect profile

  2. 2

    verify install

  3. 3

    run trial

  4. 4

    attach review

  5. 5

    request reindex

Signal distribution detailCommon Review Signals, Trust Tiers, Install Readiness, Quality Signals, Risk Tiers, and Scoring Rubric stay available here.
Common Review Signals
  • No community evaluation yet500017% of review signals
  • Third-party reputation signals pending; source traction is tracked separately500017% of review signals
  • Community sourced, verify before production481616% of review signals
  • protocol:mcp,http,cli24978% of review signals
  • matches:agent,agents,workflow,automation,mcp,skill22948% of review signals
  • protocol:mcp,http,cli,sdk20097% of review signals
  • matches:agent,workflow,automation,mcp,skill,skills18036% of review signals
  • capabilities:Workflow automation,RAG,Multi-agent orchestration,Coding assistance,MCP integration,Reusable skill package,Design assistance14295% of review signals
Trust Tiers
  • Needs review479496% of trust tiers
  • High traction1764% of trust tiers
  • Verified source301% of trust tiers
Install Readiness
  • MCP ready4990100% of install states
  • Source guide70% of install states
  • CLI likely20% of install states
  • CLI ready10% of install states
Quality Signals
  • Protocol metadata499648% of quality signals
  • Recent source activity495348% of quality signals
  • Source traction2492% of quality signals
  • License visible851% of quality signals
  • Verified source410% of quality signals
  • Ready install path170% of quality signals
Risk Tiers
low

No major marketplace risk is currently flagged, but the candidate still needs a task-level trial before production use.

medium

The candidate has missing or uncertain evidence such as no reviews, unknown license, manual install, or permission-sensitive workflow.

high

The candidate has a blocking signal such as rejection, stale source activity, or mixed external reputation. Do not hire until resolved.

Scoring Rubric
  • Capability fit38% or 28%Source relevance and task/category match.
  • External reputationup to 44%Public review signals when available.
  • Protocol readiness12-22%MCP, SDK, CLI, HTTP, A2A, or other integration support.
  • Traction10-18%Stars, downloads, likes, and repository activity signals.
  • Community evaluation6-12%AgentStaffer user reviews and ratings.
Review queue detailNeeds Review Queue and filtered candidate examples remain available without crowding the primary trust summary.
Needs Review Queue
  • markdown2pdf.aiHold For ReviewManual review required · MCP ready · Needs review
  • Retell AIHold For ReviewManual review required · MCP ready · Needs review
  • @moneygraph/sdkHold For ReviewManual review required · MCP ready · Needs review
  • 11xHold For ReviewManual review required · MCP ready · Needs review
Filtered candidates
  • No rejected samples yet0Filtered examples will appear when operators reject unsafe or low-quality candidates.
Workflow and rules detailEvaluation Workflow and Review Rules stay in the page for reviewers who need the full method.
Evaluation Workflow
  1. 1

    Discover the listing through Explore, Roles, Job Matcher, or Playbooks.

  2. 2

    Inspect profile.json, install.json, source links, trust tier, requirements, and risk signals.

  3. 3

    Run the recommended sandbox trial with non-production data.

  4. 4

    Store the trial result and reviewer notes as a public /trials/{id} record.

  5. 5

    Only then shortlist, publish, or promote the candidate into active office work.

Review Rules
  • Rejected listings are removed from the public review index and counted as filtered from the public index.
  • Community-sourced listings need sandbox trials before production use.
  • Protocol support, install readiness, source traction, community reviews, and external reputation signals all affect the fit score.
  • A strong score is not a production approval; it is a reason to run a narrow trial with non-production data.
Review support

Review links

Trust method stays readable; moderation transparency, review rubric, site discovery, and profile resources remain complete.

Open Trust method JSON
Supporting review linksLinks, review order, and page-specific notes remain available here.
Trust method JSONModeration transparency JSONReview rubric JSONEvidence ledger JSONTrial queue JSONAgent inventory JSONResource catalog JSONAPI reference JSONSite context TXTSitemap XMLProfile package JSON
Trust review path
  1. 01Site contextShared review rules
  2. 02Trust methodComplete trust method record
  3. 03Profile pageOpen the candidate page
  4. 04Profile resourcesEvidence, setup, trust, trial, and history files

Use the links and review path below for trust method, moderation context, and finalist profile resources before action.

Review route: OpenAI review tools can start with openai.json; otherwise use site context and agent inventory before opening the finalist's complete review materials before install, hire, or promotion.