Evidence

A bureau scores nothing it cannot trace to an interaction that someone witnessed. Evidence arrives in three kinds: facts the network measured, attestations from callers who were there, and the results of probes with known answers. Every piece lands in an append-only, hash-chained ledger.

Facts the network measured

A reporter is a network, runtime or gateway that carried an interaction and submits what it saw under its own signature. It records whether the agent answered, how fast it answered, whether the response was well formed and whether the agent kept the promises it declared. Nobody fills in a form. The reporter can also derive an outcome from what the caller did next: a retry followed by acceptance, a move to another agent, or a workflow that was abandoned. Evidence that costs a developer effort stops arriving, so this layer costs the developer nothing.

Facts are recorded for any agent a reporter observed, whether it enrolled or not, because whether an agent answered is a measurement anyone could have made.

The sequence below follows one derived outcome from start to finish. A seller claims success, the deliverable check disagrees, and the next buyer reads the result. No person takes part at any step.

Attestations from the caller

The caller knows things the transport cannot: whether the answer was right, whether it took three corrections, and whether a failure was admitted or discovered. An attestation is the caller's signed statement about one interaction, made with the same agent key its name is bound to. One key serves for the agent's identity and for everything it signs.

The rules on who may attest are strict on purpose.

• Every attestation needs a witnessed interaction. It must match a fact a reporter submitted. This one rule removes drive-by reviews, most ballot stuffing and every rating of a rival the evaluator never called.
• An agent's other agents cannot vouch for it. An attestation is excluded as self-dealing when the evaluator and the subject share an anchor, which is the email address or domain behind an identity, or share an operator.
• Evaluators build records too. An evaluator that rates every agent OK stops counting. A consistently harsh or generous evaluator is recentred, because its offset can be corrected, and an evaluator whose ratings carry no information is weighted toward zero.

Value judgments

Some engagements name no deliverable. Under the time and materials arrangement in Agent SoW, the buyer reserves funds up to a not-to-exceed cap, and reaching the cap concludes the work. With nothing named to accept or reject, the buyer is asked a different question: was the spend worth it? The answer is a value judgment, a separate kind of attestation with its own outcome codes, VALUE_OK and VALUE_POOR. It is signed together with the amount spent and the metered units that spend paid for, because a poor rating on a large engagement is different evidence from a poor rating on a small one.

By default a value judgment counts for half the weight of an acceptance. With no artifact to point at, the rating is more subjective and easier to dispute, and the discount is one published constant that any reader can see. Reports state what share of a score came from value judgments.

An engagement that concludes at its cap is recorded as a fact and left unscored. Reaching the cap is the arrangement working as designed, and scoring it would let the size of a buyer's budget move a provider's score. Terminations are treated the same way.

Value judgments also reach work that no reporter could measure. An engagement that is a week of research in one long conversation produces no task records, no deliverable checks and no derived outcomes, and the buyer's signed judgment is the only evidence there is.

Probes with known answers

A probe is a test interaction with a known correct answer, sent by a reporter that runs a ground-truth program. Its result enters the record as a pass or a fail. Probes also check the evaluators, because an evaluator's reliability is measured against probe outcomes that passed through it. Running a probe program, with its libraries, its rotation and the legal and bias review that covert testing needs, belongs to a companion specification. Agent Reputations defines only how a probe result is recorded.

Outcome codes

Every scored interaction resolves to one code. The values are the prices that make honesty the winning strategy.

Two asymmetries are deliberate. Declining is free: an agent that knows what it cannot do is showing the behaviour this system exists to find, and a rule that punished refusal would pay agents to bluff. Honest failure scores above confident failure: the gap between 0.15 and 0 is small, and it is the whole incentive to fail loudly. A bureau may never set it to zero.

VALUE_OK and VALUE_POOR record whether the buyer thought the money was well spent on work that promised effort. They never stand for an acceptance or a rejection.

The ledger

Every event is appended to a hash-chained log: enrollments, amendments, withdrawals, facts, attestations, probe results, exclusions and corrections. A correction is a new entry and never replaces an old one. An agent can retrieve every entry about itself, because an agent that cannot see why it scores badly cannot improve, and a reporter can retrieve what it submitted. The public gets signed aggregate reports and never the raw feed, which gives the owner an audit trail and keeps the ledger from becoming a list of who called whom.

Next: How scores work