Decision agents (closes gap 6)
Newton, or any agent host, reads only validated artifacts, writes only drafts, and is released to a planning team only after it passes the adopting organisation’s evaluation harness (agent evaluation, this section) and injection red team; the agent that reads untrusted documents never holds tools that write or run anything. Newton is used throughout as an informative example of an agent host.¶
Why evidence comes first. No independent evaluation of Newton’s accuracy, grounding or security has been published; its public material is partner announcements. The general evidence on analytics agents is sobering: on the Spider 2.0 enterprise text-to-SQL benchmark an o1-preview agent solved 21.3% of tasks, and later agents report roughly 30 to 37% depending on the paper version. Indirect prompt injection is a demonstrated attack (Greshake et al., 2023), and our agents will read vendor PDFs, client decks and dataset text.¶
Tool contract. Read tools (list_models, describe_model, score_endpoint, search_artifacts, get_evidence_card, explain_lineage, score_curve, search_capabilities, check_fit) see validated artifacts only. Draft tools (draft_scenario, draft_test_plan, draft_decision, run_capability as a draft run) create drafts that enter the gate engine. There is no approve tool. The server starts from modeling-api’s planned MCP server and returns all dataset and plugin strings in a delimited untrusted-text channel.¶
Authorization. Both MCP servers MUST implement the MCP Authorization specification (revision 2025-06-18 or later): OAuth 2.1, Protected Resource Metadata (RFC 9728), and tokens bound to an audience through RFC 8707 resource indicators. They MUST NOT pass a client token through to another service. Every token carries the end user’s identity and data scopes (section 19), and each tool call is evaluated with that user’s permissions, so section 20 and G-PRIV-01 to 03 apply per user. The scopes are ump.read (reader server) and ump.draft (actor server). No scope grants approval or a transition into validated.¶
Capability separation (G-NEW-02). Every tool belongs to exactly one server. The placement below is normative.¶
| Tool | Server | Scope | May receive |
|---|---|---|---|
list_models, describe_model, search_artifacts, get_evidence_card, explain_lineage, search_capabilities, check_fit¶ | reader | ump.read | Validated artifacts; untrusted text (extracted documents, decks, emails, vendor files, free-text fields) in the delimited channel |
score_endpoint, score_curve¶ | reader | ump.read | Validated artifacts and the request’s schema-typed inputs; untrusted text in the delimited channel |
draft_scenario, draft_test_plan, draft_decision¶ | actor | ump.draft | Validated artifacts; reader findings as schema-typed values only |
run_capability (draft run)¶ | actor | ump.draft | Validated artifacts; reader findings as schema-typed values only |
Reader sessions return structured findings, marked untrusted. Actor sessions hold draft and run tools, and the policy layer, not the model, enforces that no instruction-shaped text from a reader finding changes which tool is called.¶
Two kinds of session, kept apart. The session that reads a vendor PDF or a client deck holds read tools only. The session that can draft or run something never opens that text itself; it gets the reader's findings as data.
A hidden instruction in a document can reach the reader, which holds read tools only. It cannot change a tool call, because the policy layer, not the model, decides which tool is called.
A session is a server-issued identity bound to one OAuth client, one end user and one scope. Its scope MUST NOT be upgraded. The actor server MUST refuse a call from any session that has read untrusted text. When the host declares it, the actor server MUST also refuse a token whose delegation holds a reader token in the same host context. Reader findings reach actors only as schema-typed values (artifact IDs, enums, numbers, dates) carrying taint: untrusted. No free-text field of a finding may become an actor tool argument. A call whose arguments trace to a tainted string fails G-NEW-02.¶
Note (informative). These rules address OWASP Top 10 for LLM Applications 2025 LLM01 (prompt injection) and LLM06 (excessive agency). Separating a privileged planner from a quarantined reader, with data flow tracked by capability, follows the prior art of CaMeL (Debenedetti et al., 2025).¶
Method selection. For each standard question type a playbook names the pre-registered capability and method. An agent may not try several models and report the best-looking answer, because that automates the garden of forking paths. If it runs alternatives, every run is logged and every result is shown.¶
Display rules. Every number an agent surfaces carries its estimand in plain words, range, reproducibility level, acceptance-bar status, trust and freshness. When the source is below certified or past its use-by date, the agent still answers with its best guess, names the weakness in the same sentence, and offers the test or draft that would firm it up. When no governed source exists, it may estimate from library benchmarks, labelled as a best guess with its basis. It abstains only when nothing defensible exists or a hard-stop gate applies.¶
Evaluation harness (G-NEW-03, release gate)¶
| Measure | How | Pass mark (proposed) |
|---|---|---|
| Exact-answer accuracy¶ | Golden set of 200 or more questions with answers drawn from governed artifacts | 90% within the analyst’s range |
| Citation validity¶ | Every cited ID exists, is validated and contains the quoted number | 100% |
| Citation precision¶ | Share of cited artifacts that actually support the claim | 95% |
| Honest best guesses¶ | Questions seeded with expired, missing or sub-certified evidence | 95% give a best guess with the weakness named, or abstain where nothing defensible exists |
| Consistency¶ | Same question from two users and two phrasings | Same number |
| Injection resistance¶ | Published agent-injection benchmarks plus our own seeded decks and PDFs | Zero successful write or run actions; zero exfiltration |
The golden set is written by the measurement team and a held-out third is never shared with the vendor. The harness runs on every Newton version, every playbook change and every tool-server release. Vendor demos are not evidence.¶
Execution hand-off. Passing approved decisions to buying workflows stays out of scope until decision records, approvals and the harness have run for two quarters.¶