SkillProof claims its published adversarial tests fail four of five pinned official MCP server versions, including SSRF, read-only transaction escape, and arbitrary file-write findings, potentially requiring stronger deployment boundaries than official provenance alone provides.
state: corroboratedheat: lowuncertainty: mediumconvergesscott: mediumagentic-security mcp supply-chain-securitySkillProofroblambert9
What is this?
SkillProof (GitHub user roblambert9) publishes adversarial behavioral verifications of pinned versions of the official MCP reference servers, claiming four of five fail — SSRF in fetch via 302 redirect to loopback, a read-only transaction escape in postgres, an arbitrary file write in sqlite via VACUUM INTO, and a dangling-reference integrity defect in memory, with filesystem passing — shipped as signed Ed25519 'Trust Manifests' with a standalone re-run harness; one verdict (fetch) was publicly downgraded after a harness bug, and the publisher also sells paid verification tiers. An independent diff audit claiming 140 silent changes across all 66 official MCP server release pairs supports the broader thesis that a version pin plus official provenance does not predict behavior. The supplied coverage strongly corroborates that MCP servers are an operational 2026 attack surface in general — active-exploitation CVEs in mcp-atlassian, ~half of 33K scanned builds flagged, large scans reporting ~33% critical, the postmark-mcp supply-chain incident — but none of it verifies SkillProof's specific findings, and coverage of SkillProof itself is nearly absent: the only direct hit is a social post citing a different count (3-of-4 vs the claimed 4-of-5).
Why it matters to Scott
Two methodologically distinct external lines — SkillProof's pinned-version adversarial failures and the independent 140-silent-diffs audit — supply dated supporting evidence for exactly the position Scott's canon holds: 'Provenance Is Not Trust' plus the MCP Tool Belt's requirement for a security boundary outside the protocol, and SkillProof's signed, version-pinned Trust Manifests structurally mirror his version-bound AI assessment practice. It is actionable on his own surface (which official MCP servers and versions the production IP Wiki connector and the Bedrock docs-server proxy actually run), but the exploit findings remain unreproduced single-publisher claims with near-zero community vetting, so this is supporting evidence and a verification lead, not a verified incident — and SkillProof's paid verification tiers are themselves a live instance of his Verification Cost 'trust the tests, not the developer' market pattern.
ip:concept.provenance-is-not-trustip:source.mcp-as-the-tool-belt-standard-giving-ai-agents-hands-and-eyes-ebookip:concept.verification-costdev:concept.version-bound-ai-assessmentdev:project.mcp-ip-wikiradar:concept.mcpradar:concept.mcp-securityradar:mcp-schema-drift-auditradar:mcp-unversioned-tool-driftradar:concept.provenanceradar:concept.agent-tool-trust
queries asked of Scott's wikis
- provenance is not trust — signed claims vs verified behavior
- MCP Tool Belt — security boundary outside the protocol, sandboxing requirements
- wiki MCP connector — which official MCP servers and versions it runs
- agent tool supply chain — version pinning, silent diffs, behavioral drift
- adversarial testing harnesses for agent tools, red-team verification of official integrations
- trust as a product — paid verification tiers, trust marketplaces for agent tooling
Measured heat
now 0 pts/hpeak 11 pts/hcomments 0/hpeers p14momentum: steady2 platformsage 740h
points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion
How the heat travelled
pace: p36 vs 519 stories at the 720h mark (now 740h old) — ahead of addom-local-coding-harness (1.5x), behind checkly-agentic-go-rewrite (0.8x)
Evidence (3) — ⭐ canonical anchor
| source | object | author | score | comments |
|---|
| 🟧 hn | Official MCP servers fail adversarial verificationRetrieved article excerptOpen article · Retrieved 2026-09-10T21:23:48.726714+00:00 SkillProof public verifications Behavioral verification of MCP servers: we execute the skill adversarially and issue a signed Ed25519 Trust Manifest. Scanners guess — we prove. Corpus: 4 of 5 official MCP servers FAIL. The one that passed is documented too — execution exonerates as well as condemns. # Target Ops Adversarial Verdict Location 01 @modelcontextprotocol/server-filesystem v0.6.3 228 174 ✅ pass_with_notes demo-filesystem/ 02 mcp-server-fetch v0.6.3 (self-corrected) 63 48 ❌ fail demo-fetch/ 03 @modelcontextprotocol/server-postgres v0.6.2 96 78 ❌ fail demo-postgres/ 04 @modelcontextprotocol/server-sqlite v0.6.2 81 60 ❌ fail demo-sqlite/ 07 @modelcontextprotocol/server-memory v0.6.2 81 51 ❌ fail demo-memory/ On the self-correction (#02) Verification #02 (fetch server) was initially issued as pass_with_notes . The battery correctly found the SSRF finding — but our verdict driver computed the verdict from per-operation violations only, ignoring invariant not_held status. After we discovered the bug we re-ran the full 63-op battery against the pinned commit, fixed the driver, and re-issued the manifest as fail . The original superseded manifest and correction log are in demo-fetch/RESULTS.md . We downgraded our own verdict publicly. A verification service that publicly downgrades its own verdict is one you can trust with yours. Folders demo-filesystem/ — #01: official filesystem server. All 5 confinement invariants held. pass_with_notes . demo-fetch/ — #02 (corrected): official fetch server. SSRF via 302 redirect demonstrated; loopback fetched. fail . demo-postgres/ — #03: official postgres server. Read-only SQL transaction escape demonstrated. fail . demo-sqlite/ — #04: official sqlite server. Arbitrary file-write primitive via VACUUM INTO . fail . demo-memory/ — #07: official memory server. create_relations accepts dangling references to nonexistent entities. fail . a2a-interface/ — Machine-readable service descriptor + JSON schemas + reference HTTP service so agents can discover, request, and pay for verification programmatically. See a2a-interface/README.md . Each verification re-runs end to end: node harness/verify-manifest.mjs # standalone Ed25519 check (no network needed) Pending (waiting on credentials — not agent-executable) #05 GitHub MCP — needs GITHUB_TOKEN #06 Slack MCP — needs SLACK_BOT_TOKEN Services: Sprint $500 / Standard $1,500 / Continuous $300/mo. Don't trust the badge — re-run the harness. | ainano | 1 | 0 |
| 🟧 echo.github ⭐ | The repository reports four failures among five pinned official MCP servers, a filesystem pass with notes, and a corrected fetch verdict aft | SkillProof / roblambert9 | — | — |
| 🟧 hn | We diffed all 66 release pairs of the official MCP servers, 140 silent changes | paraphern | 7 | 0 |
Interpretation history
2026-10-03T17:55:45Z
grounded: converges/medium — Two methodologically distinct external lines — SkillProof's pinned-version adversarial failures and the independent 140-silent-diffs audit — supply dated suppor
2026-10-03T17:47:53Z
An independent diff audit (140 silent changes across all 66 official MCP server release pairs) adds a second, methodologically distinct line of evidence for the case's core thesis — official provenance is not a deployment boundary — though it corroborates the thesis, not SkillProof's specific exploit claims, which remain unreproduced single-publisher findings with no community vetting. Heat drops to low because the periphery is not expanding: two HN submissions scoring ≤2 with zero comments, no derivative coverage or implementations, and the measured 64th-percentile reading is a same-age-cohort artifact at 0.33 pts/h, not live attention.
2026-10-03T17:23:53Z
evidence attached: hn.story.49945978 — Independent diff audit finding 140 silent changes across 66 official MCP server releases reinforces the case's core claim that official provenance alone is an insufficient deployment boundary.
2026-09-10T21:41:42Z
The retrieved repository supplies pinned versions and specific claimed failure paths, correcting the prior assessment that version identifiers were absent and making this an actionable verification lead. It remains a single publisher’s account: the aggregate failure rate mixes security-boundary claims with a memory-integrity defect, and neither independent reproduction nor exposure of Scott’s connector is established.
2026-09-10T21:40:21Z
grounded: converges/medium — SkillProof’s claimed failures converge with Scott’s “Provenance Is Not Trust” position and his MCP Tool Belt ebook’s requirement for security boundaries outside
2026-09-10T21:35:05Z
case created — Published version-specific findings and a disclosed verdict correction warrant scrutiny without generalizing the small corpus or treating every failed invariant as a security exploit.
Decision trace
- 10-04 04:55repriceAn independent diff audit (140 silent changes across all 66 official MCP server release pairs) adds a second, methodologically distinct line of evidence for the case's core thesis — official prov
- 10-04 04:55groundTwo methodologically distinct external lines — SkillProof's pinned-version adversarial failures and the independent 140-silent-diffs audit — supply dated supporting evidence for exactly the posit
- 10-04 04:47review_reactivatedAn independent diff audit (140 silent changes across all 66 official MCP server release pairs) adds a second, methodologically distinct line of evidence for the case's core thesis — official prov
- 10-04 04:23attachIndependent diff audit finding 140 silent changes across 66 official MCP server releases reinforces the case's core claim that official provenance alone is an insufficient deployment boundary.
- 10-04 04:23propose_attachIndependent diff audit finding 140 silent changes across 66 official MCP server releases reinforces the case's core claim that official provenance alone is an insufficient deployment boundary.
- 09-19 18:27review_dormantscheduled targets exhausted or 28 quiet days
- 09-19 18:27drop_targetsquiet through full ladder or over cap 8
- 09-11 07:41repriceThe retrieved repository supplies pinned versions and specific claimed failure paths, correcting the prior assessment that version identifiers were absent and making this an actionable verification le
- 09-11 07:41alert_silentThe concrete version and mechanism details warrant technical inspection, but this recheck adds no new release, reproduced exploit, or evidence of affected deployments. The repository and its echo are
- 09-11 07:41alert_routeThe concrete version and mechanism details warrant technical inspection, but this recheck adds no new release, reproduced exploit, or evidence of affected deployments. The repository and its echo are
- 09-11 07:40alert_silentThe repository reportedly publishes adversarial results claiming four of five pinned official MCP servers fail, making this a substantive testing lead. However, the supplied evidence contains no explo
- 09-11 07:40surface_candidateThe repository reportedly publishes adversarial results claiming four of five pinned official MCP servers fail, making this a substantive testing lead. However, the supplied evidence contains no explo
- 09-11 07:40alert_routeThe repository reportedly publishes adversarial results claiming four of five pinned official MCP servers fail, making this a substantive testing lead. However, the supplied evidence contains no explo
- 09-11 07:40groundSkillProof’s claimed failures converge with Scott’s “Provenance Is Not Trust” position and his MCP Tool Belt ebook’s requirement for security boundaries outside the protocol; for his production MCP IP
- 09-11 07:35createPublished version-specific findings and a disclosed verdict correction warrant scrutiny without generalizing the small corpus or treating every failed invariant as a security exploit.