Independent verification and ecosystem response will determine whether roughly one in ten published Claude Code skills fail to load and require stronger validation or packaging safeguards.

state: expiredheat: lowuncertainty: highconvergesscott: mediumagent-harnesses coding-agents agent-skillsAnthropicKynth Studio

What is this?

The case concerns a reported dataset finding that 9.70% of published Claude Code skill artifacts fail to load, raising questions about validation and packaging in the coding-agent skill ecosystem. The supplied snippets describe skills as folders containing a `SKILL.md` plus optional scripts or templates, and distinguish load or activation failures from failures during execution; they also recommend testing skills in a fresh session. However, the search results do not independently surface the cited Zenodo dataset, its methodology, Kynth Studio’s role, or Anthropic’s response, so the one-in-ten figure remains unverified from the provided web evidence.

Why it matters to Scott

The reported failure rate independently supports Scott’s position that reusable agent skills are production artifacts that need repeatable evaluation and binding release gates, rather than informal Markdown dropped into a folder. If independently reproduced, it could sharpen his skills-and-workflows doctrine into concrete cold-start loading checks and packaging safeguards; for now, the dataset and methodology remain unverified in the supplied evidence.

ip:concept.skills-and-workflowsip:concept.evaluation-driven-developmentip:framework.12-factor-agents-frameworkradar:concept.agent-skillsradar:shared-agent-skills-standard

queries asked of Scott's wikis

agent skill packaging and validation
coding-agent harness extension reliability
executable skills as software dependencies
agent capability registries and package managers
cold-start testing for agent instructions
supply-chain safeguards for agent plugins

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (5) — ⭐ canonical anchor

source	object	author	score	comments
🟧 hn	Nearly one in ten published Claude Code skills does not load	kyisaiah47	1	0
🟧 echo.other ⭐	The primary artifact is the versioned Zenodo dataset, not the HN post or census page. It reports: “9.70% of the published Claude Code artefa	Kynth Studios	—	—
🟠 reddit	28 of my 43 installed Claude Code skills have never fired once ClaudeAI	karanb192	1	0
🟧 hn	Show HN: SkillWorks – every Claude Code skill, scored on whether it loads	kyisaiah47	2	0
🟠 reddit	I scored every Claude Code skill I could find on whether it actually loads — 46k of them do not ClaudeAI	TillDramatic1	0	5

Interpretation history

2026-08-25T20:35:13Z
The case has produced no independent runtime reproduction, disclosed validation advance, or Anthropic/ecosystem response after repeated checks. The packaging concern remains plausible, but the specific one-in-ten estimate has faded as an active developing signal.
2026-08-23T20:32:03Z
New discussion sharpens the unresolved methodological issue: static checks of frontmatter and directory shape may not measure whether a skill loads in a real Claude Code installation. This adds useful scrutiny but no reproduction, ecosystem response, or evidence supporting the one-in-ten prevalence estimate.
2026-08-23T14:30:35Z
The newly attached corpus-wide claim is another promotion of the SkillWorks census, not an independent reproduction, so it does not corroborate the roughly one-in-ten estimate. The practical packaging failure modes remain plausible, but prevalence and classification methodology still need outside validation or an Anthropic response.
2026-08-23T14:22:58Z
evidence attached: reddit.post.1vw78dt — Independent corpus-wide scoring reports that a large share of published Claude Code skills fail to load, materially corroborating the open packaging-reliability hypothesis.
2026-08-22T12:29:04Z
SkillWorks exposes per-skill scoring and makes the originating census more inspectable, but it is a product of the same organization rather than an independent reproduction. The 9.70% estimate therefore remains uncorroborated, with no Anthropic or ecosystem response yet.
2026-08-22T12:22:49Z
evidence attached: hn.story.49398601 — Independent SkillWorks scoring directly tests the reported Claude Code skill loading failure rate and could corroborate the open case.
2026-08-20T20:38:58Z
The transcript audit adds an independent practical signal that installed skills often go unused or undiscovered, broadening the reliability concern beyond packaging. It does not reproduce the census’s 9.70% load-failure claim or distinguish malfunction from lack of need, so the central estimate remains uncorroborated.
2026-08-20T16:24:01Z
evidence attached: reddit.post.1vtnzzb — The user's transcript-based measurement independently supports concerns about Claude Code skill discovery or activation failures.
2026-08-20T12:46:03Z
No independent reproduction, implementation response, or Anthropic acknowledgement has appeared; the one-in-ten failure claim remains a single-source census rather than a corroborated ecosystem reliability finding.
2026-08-20T12:42:15Z
grounded: converges/medium — The reported failure rate independently supports Scott’s position that reusable agent skills are production artifacts that need repeatable evaluation and bindin
2026-08-20T12:39:36Z
origin walked (codex/luna, conf 0.97): anchor hn.story.49373129 -> echo.other.090fd96f73 by Kynth Studios
2026-08-20T12:37:50Z
case created — The linked census presents a bounded, testable reliability claim about the published Claude Code skill ecosystem.

Decision trace

08-26 06:35expireThe case has produced no independent runtime reproduction, disclosed validation advance, or Anthropic/ecosystem response after repeated checks. The packaging concern remains plausible, but the specifi
08-26 06:35alert_silentThe only delta is elapsed staleness; no new consequential fact occurred, so there is nothing to surface before the next briefing.
08-26 06:35alert_routeThe only delta is elapsed staleness; no new consequential fact occurred, so there is nothing to surface before the next briefing.
08-24 06:32repriceNew discussion sharpens the unresolved methodological issue: static checks of frontmatter and directory shape may not measure whether a skill loads in a real Claude Code installation. This adds useful
08-24 06:32alert_silentThe refreshed comments only question the census methodology and introduce no verified event or consequential new finding. The case can wait for independent runtime reproduction, fuller validation deta
08-24 06:32alert_routeThe refreshed comments only question the census methodology and introduce no verified event or consequential new finding. The case can wait for independent runtime reproduction, fuller validation deta
08-24 06:21sensor_dirtycomment_update
08-24 00:30repriceThe newly attached corpus-wide claim is another promotion of the SkillWorks census, not an independent reproduction, so it does not corroborate the roughly one-in-ten estimate. The practical packaging
08-24 00:30alert_silentThis delta repeats the originating organization’s census through another self-published post and adds neither independent verification nor an ecosystem response. It can wait for routine briefing while
08-24 00:30alert_routeThis delta repeats the originating organization’s census through another self-published post and adds neither independent verification nor an ecosystem response. It can wait for routine briefing while
08-24 00:23alert_silentThis is another self-published promotion of the existing SkillWorks census, not independent reproduction or a materially new finding. The claimed load-failure rate and useful packaging lessons can rem
08-24 00:23alert_routeThis is another self-published promotion of the existing SkillWorks census, not independent reproduction or a materially new finding. The claimed load-failure rate and useful packaging lessons can rem
08-24 00:22attachIndependent corpus-wide scoring reports that a large share of published Claude Code skills fail to load, materially corroborating the open packaging-reliability hypothesis.
08-24 00:22propose_attachIndependent corpus-wide scoring reports that a large share of published Claude Code skills fail to load, materially corroborating the open packaging-reliability hypothesis.
08-22 22:29repriceSkillWorks exposes per-skill scoring and makes the originating census more inspectable, but it is a product of the same organization rather than an independent reproduction. The 9.70% estimate therefo
08-22 22:29alert_silentThe new interface is established but does not independently validate the underlying failure-rate claim or create a time-sensitive decision; it can wait for routine briefing while independent replicati
08-22 22:29alert_routeThe new interface is established but does not independently validate the underlying failure-rate claim or create a time-sensitive decision; it can wait for routine briefing while independent replicati
08-22 22:23alert_silentSkillWorks is a newly exposed interface from the same organization that produced the census, not independent reproduction or an ecosystem response. Its launch is established, but the reported 9.70% lo
08-22 22:23surface_candidateSkillWorks is a newly exposed interface from the same organization that produced the census, not independent reproduction or an ecosystem response. Its launch is established, but the reported 9.70% lo
08-22 22:23alert_routeSkillWorks is a newly exposed interface from the same organization that produced the census, not independent reproduction or an ecosystem response. Its launch is established, but the reported 9.70% lo
08-22 22:22attachIndependent SkillWorks scoring directly tests the reported Claude Code skill loading failure rate and could corroborate the open case.
08-22 22:22propose_attachIndependent SkillWorks scoring directly tests the reported Claude Code skill loading failure rate and could corroborate the open case.
08-21 06:38repriceThe transcript audit adds an independent practical signal that installed skills often go unused or undiscovered, broadening the reliability concern beyond packaging. It does not reproduce the census’s
08-21 06:38alert_silentA single-user utilization audit is useful supporting evidence but establishes neither widespread load failures nor an ecosystem change requiring action today; independent reproduction or an Anthropic/
08-21 06:38alert_routeA single-user utilization audit is useful supporting evidence but establishes neither widespread load failures nor an ecosystem change requiring action today; independent reproduction or an Anthropic/
08-21 02:24alert_silentThe audit introduces a useful utilization and discovery measurement tool, but it does not reproduce the reported skill load-failure rate or distinguish failed discovery from skills that were simply un
08-21 02:24surface_candidateThe audit introduces a useful utilization and discovery measurement tool, but it does not reproduce the reported skill load-failure rate or distinguish failed discovery from skills that were simply un
08-21 02:24alert_routeThe audit introduces a useful utilization and discovery measurement tool, but it does not reproduce the reported skill load-failure rate or distinguish failed discovery from skills that were simply un
08-21 02:24attachThe user's transcript-based measurement independently supports concerns about Claude Code skill discovery or activation failures.
08-21 02:22propose_attachThe user's transcript-based measurement independently supports concerns about Claude Code skill discovery or activation failures.
08-20 22:46repriceNo independent reproduction, implementation response, or Anthropic acknowledgement has appeared; the one-in-ten failure claim remains a single-source census rather than a corroborated ecosystem reliab
08-20 22:46alert_silentThis look adds no consequential evidence beyond the existing self-published dataset, so routine monitoring for an independent replication or ecosystem response is sufficient.
08-20 22:46alert_routeThis look adds no consequential evidence beyond the existing self-published dataset, so routine monitoring for an independent replication or ecosystem response is sufficient.
08-20 22:43alert_silentKynth Studios has published a versioned dataset and methodology claiming that 43,199 of 445,348 Claude Code skill listings fail to load, but this is a self-published ecosystem census whose sampling, f
08-20 22:43surface_candidateKynth Studios has published a versioned dataset and methodology claiming that 43,199 of 445,348 Claude Code skill listings fail to load, but this is a self-published ecosystem census whose sampling, f
08-20 22:43alert_routeKynth Studios has published a versioned dataset and methodology claiming that 43,199 of 445,348 Claude Code skill listings fail to load, but this is a self-published ecosystem census whose sampling, f
08-20 22:42groundThe reported failure rate independently supports Scott’s position that reusable agent skills are production artifacts that need repeatable evaluation and binding release gates, rather than informal Ma
08-20 22:39promote_anchororigin walk conf 0.97
08-20 22:37createThe linked census presents a bounded, testable reliability claim about the published Claude Code skill ecosystem.