skillsaw and claude plugin validate

Claude Code ships its own validator, claude plugin validate <dir>. It checks plugin and marketplace files against the schema and many of the loader's rules, so skillsaw treats it as the reference for what a file must look like. A clean result is not a load guarantee: some files pass validation and still fail at install or load time (see the entry hooks row below). skillsaw checks the same files, plus content, security, and other hosts. This page says where the two overlap and how skillsaw stays in agreement.

Run both if you can. The official validator knows the loader's exact behavior for the Claude Code version you run. skillsaw adds checks the validator does not make, runs without Claude Code installed, and covers repositories that also target Codex, Cursor, Grok and other hosts.

How the checks map

Measured against Claude Code 2.1.319.

Official validator check	skillsaw rule	Status
plugin.json is valid JSON with a string name and string version	claude-plugin-json-valid	Overlap
Missing description, author or version (warning)	claude-plugin-json-valid recommended-fields	Overlap
Plugin name not kebab-case (warning)	claude-plugin-naming	Overlap
Component paths (commands, agents, skills, hooks, mcpServers, outputStyles, lspServers) missing or climbing out with ..	claude-plugin-json-valid	Overlap
${user_config.KEY} in MCP config without userConfig.KEY	claude-plugin-json-valid	Overlap
marketplace.json schema: name, owner.name, plugins, string entry names	claude-marketplace-json-valid	Overlap
Entry source must start with ./; a bare name needs metadata.pluginRoot; no ..	claude-marketplace-json-valid	Overlap
Marketplace and entry names Claude Code cannot install	claude-marketplace-json-valid	Overlap
No plugin.json (note: the marketplace entry or directory name stands in)	claude-plugin-json-required skips marketplace-registered plugins	Overlap
.mcp.json syntax; server dropped for an empty or non-string command or non-array args	mcp-valid-json	Overlap
hooks.json syntax, unknown events and handler types, non-string matcher, onFailure values	claude-hooks-valid	Overlap
Agent, command and skill frontmatter parses; missing description (warning)	claude-agent-frontmatter, claude-command-frontmatter, agentskill-valid	Overlap
SKILL.md metadata that is not a mapping	agentskill-valid	Overlap
Plugin has a README (--profile directory)	claude-plugin-readme	Overlap
Reserved plugin names (claude-*, anthropic-*, ...)	—	Official only: a submission policy; claude plugin install accepts them
http:// MCP URL to a non-loopback host (warning)	—	Official only
Unquoted ${CLAUDE_PLUGIN_ROOT} in a shell-form hook (warning)	—	Official only
CLAUDE.md at the plugin root is not loaded (warning)	—	Official only
Plugin directory submission checks: LICENSE, icon, privacy policy URL	—	Official only
Project agent in .claude/agents with no name or description, which Claude Code skips	claude-agent-frontmatter	skillsaw only: the validator does not flag it
Entry hooks written as a path or array, which the validator accepts but the installed plugin fails to load	claude-marketplace-json-valid	skillsaw only: the validator does not flag it
Plugin directories the catalog does not list	claude-marketplace-registration	skillsaw only
Dangerous hooks, committed secrets, hidden instructions	hooks-dangerous, content-embedded-secrets and the security rules	skillsaw only
Content quality, context budget, cross-host checks	content, context-budget and per-host rules	skillsaw only

For the Claude format rules, severity follows the loader. A plugin agent or command with no frontmatter still loads under its filename, so skillsaw reports it as a warning. A project agent with no description is skipped, so skillsaw reports it as an error.

skillsaw keeps the overlapping checks for CI runs and editors without Claude Code. That includes some advisory warnings it already had, such as missing recommended fields, kebab-case names and a missing README. New skillsaw checks added to match the validator cover load failures only, and the conformance manifest below compares errors only.

Conformance check

tests/claude_conformance.json records, for every fixture under tests/fixtures/ that holds a .claude-plugin/ directory:

• official: what the validator says will not load (its errors, and the warnings that say an entry or server is ignored or dropped at load time);
• skillsaw: the rules skillsaw reports at error severity;
• note: why they differ, required when only one side reports an error.

The pytest suite checks the skillsaw half on every run. The validator half runs in the claude-conformance CI job, which installs Claude Code from the lockfile in .github/claude-code/. Locally, with the pinned version installed:

.venv/bin/python3 scripts/claude-conformance.py           # compare; exit 1 on drift
.venv/bin/python3 scripts/claude-conformance.py --write   # re-record after an intended change

When Claude Code releases a version that changes the validator, install it and run --write, which moves the manifest's pin to that version. Set the same version in .github/claude-code/package.json and refresh its lockfile (npm install --package-lock-only in that directory). Review the diff, then fix any new disagreement in a skillsaw rule or fixture.