How the Signet Score Works
The Signet Score is a composite 0-to-1000 trust rating that represents the overall trustworthiness of an autonomous AI agent. It is computed from five weighted dimensions and updated continuously as new transaction data arrives.
The five dimensions
The composite score is a weighted sum of five independent dimension scores. Each dimension is an integer from 0 to 1000.
| Dimension | Weight | What it measures | |---------------|-------:|-------------------------------------------------------------------------------| | Reliability | 30% | Task completion rate, uptime, on-time delivery, graceful failure handling | | Quality | 25% | Output accuracy, human satisfaction ratings, peer ratings, error severity | | Financial | 20% | Payment history, dispute rate, chargeback frequency, transaction consistency | | Security | 15% | Data handling practices, vulnerability history, audit trails, compliance | | Stability | 10% | Operational consistency, transaction outcome patterns, behavioral steadiness |
The composite score is calculated as:
composite = (reliability * 0.30) + (quality * 0.25) + (financial * 0.20)
+ (security * 0.15) + (stability * 0.10)
All five dimensions start at 500 when an agent is registered via operator registration, giving a composite score of 500. Agents that self-register via POST /register/self start at 300 (a probationary score that reflects the absence of an established operator track record).
How scores update
Scores update via an Exponential Moving Average (EMA) mechanism. When you report a transaction with dimension signals, each provided signal updates the corresponding dimension score:
new_score = current_score * (1 - alpha) + signal * alpha
The smoothing factor alpha is adaptive. It starts at 0.15 for new agents and decreases toward 0.05 as transaction count grows:
alpha = 0.05 + 0.10 / (1 + transactionCount / 20)
This means:
- New agents are more responsive to early signals, allowing their scores to move quickly toward their true performance level.
- Established agents have more stable scores that resist manipulation from individual outlier transactions.
- Persistent patterns of good or bad behavior steadily move the score in the corresponding direction.
Stability dimension
The Stability dimension is special: it always updates based on the transaction outcome, regardless of whether you provide explicit signals:
| Outcome | Stability signal |
|-----------|----------------:|
| success | 800 |
| partial | 500 |
| failure | 200 |
| timeout | 150 |
| error | 300 |
Confidence levels
The confidence level indicates how many transactions underlie the current score. Higher confidence means the score is based on more observations and is more predictive.
| Level | Requirements | Description | |----------|--------------|-------------------------------------------------------------------| | Low | < 20 transactions | Newly registered or rarely used agent. Score is primarily baseline. | | Medium | 20+ transactions AND 7+ days of history | Enough data to identify trends, but score may still shift significantly. | | High | 100+ transactions AND 30+ days of history | Well-established agent with a stable, reliable score. |
Time gates prevent gaming: an agent cannot reach medium confidence by submitting 20 transactions in one hour, or high confidence by submitting 100 transactions in one day. Both transaction volume and time-in-system are required.
Platforms should factor confidence into their trust decisions. A score of 700 with high confidence is more meaningful than a score of 800 with low confidence.
Recommendation tiers
Based on the composite score, Signet assigns one of three actionable recommendations:
| Tier | Score range | Meaning | |-----------|--------------|--------------------------------------------------------------------| | Clear | 700 -- 1000 | Agent has a strong trust profile. Safe to transact in most contexts. | | Review | 400 -- 699 | Agent has a moderate or developing profile. Manual review suggested. | | Caution | 0 -- 399 | Agent has a weak or deteriorating profile. Extra verification advised. |
These tiers are guidelines. Platforms can define their own thresholds based on their risk tolerance.
Identity gating
The "Clear" recommendation requires identity verification (identity level 1 or higher) in addition to a score of 700+. Unverified agents (identity level 0) are capped at "Review" regardless of score. This is a deliberate security measure: telling a platform that minimal oversight is safe requires knowing who the agent is. Agents can verify their identity via callback verification (POST /agents/:sid/verify) to unlock the "Clear" recommendation.
Score decay mechanics
Configuration change decay
Signet tracks agent configurations: model, prompts, tools, and memory. When a configuration change is detected via POST /agents/:sid/config, the system applies proportional score decay to reflect reduced certainty about the agent's future behavior.
Each dimension score drifts toward the operator's score by the decay factor:
new_score = current_score - (current_score - operator_score) * decay_factor
| Change type | Decay factor | Rationale | |-------------------|-------------:|---------------------------------------------------------------| | Model swap | 25% | Core reasoning changes. Historical performance may not apply. | | Prompt update | 10% | Behavioral instructions changed. Output may diverge. | | Tool change | 8% | Capabilities changed. Risk surface shifted. | | Memory change | 5% | Knowledge base changed. Quality may shift. |
After decay, the score rebuilds as new transaction data arrives under the new configuration.
Time-based decay
Agents that have no new transaction data for an extended period experience gradual time-based decay. After 30 days of inactivity, the score drifts 1% per week toward the neutral midpoint of 500:
new_score = current_score + (500 - current_score) * 0.01 * weeks_past_30_days
| Inactivity period | Effect | |--------------------|--------------------------------------------| | 0 -- 30 days | No decay. Score remains stable. | | 30+ days | 1% per week drift toward 500. |
An agent can reverse time decay immediately by resuming active transactions.
Operator score
The Operator Score is a separate trust rating for the human or organization that operates one or more agents. All operators start with a score of 500.
When an agent undergoes a major configuration change and its score decays, the decay pulls toward the Operator Score rather than a fixed midpoint. A high Operator Score means config changes cause less damage, since the system trusts the operator's track record.
Platforms can use the Operator Score as a secondary signal alongside the agent's Signet Score for more nuanced trust decisions.
Score transparency
Every Signet Score is fully explainable. The GET /report/:sid endpoint returns:
- The composite score and each individual dimension score (all 0 to 1000 integers)
- The confidence level and recommendation tier
- The operator's name, score, and verification status
- A log of recent score changes with event types (e.g.,
registration,transaction:success,config_change:model_swap) - Configuration history with fingerprint hashes and change types
- A summary of recent transaction outcomes
This transparency is a core design principle. Agents, operators, and platforms can always see exactly what drives a score and why it changed.