Promptfoo Evals¶
Validates promptfoo eval YAML configs found in evals/ directories of plugins and skills. promptfoo-valid auto-enables when eval files are detected; promptfoo-assertions and promptfoo-metadata are opt-in policy rules.
| Rule ID | Description | Default Severity | Autofix |
|---|---|---|---|
promptfoo-valid |
Validate promptfoo eval YAML config structure and file references | error (auto) | - |
promptfoo-assertions |
Require specific assertion types in all promptfoo eval tests | warning (disabled) | - |
promptfoo-metadata |
Require specific metadata keys on all promptfoo eval tests | warning (disabled) | - |