Promptfoo
GuardrailsOpen-source framework for testing LLM vulnerabilities and agent tool abuse.
Visit websiteWhat it solves
Before an agent goes near production tools, someone should test whether it can be talked into misusing them. Promptfoo lets you red-team LLM behavior — including prompt injection and tool-abuse scenarios — as repeatable test suites instead of ad-hoc manual probing.
How it works
You define test cases and adversarial prompts in config files, run them from the CLI against your model or agent, and get a report of which attacks succeeded. Suites live in your repo and run in CI, so regressions in agent behavior fail the build.
Pros & cons
Pros
- Open source with an active community and broad model provider support.
- CI/CD integration makes agent safety a repeatable check, not a one-off review.
- Red-team templates cover common LLM attack patterns out of the box.
Considerations
- It tests model and agent behavior — it does not enforce anything at runtime.
- Writing meaningful tool-abuse evals requires understanding your own MCP setup well.
- Enterprise features (team dashboards, SSO) sit behind the paid tier.
When to choose it
Choose Promptfoo when you want pre-deployment and CI testing of agent behavior, paired with a runtime control like a gateway rather than replacing one.
Disclaimer: Tool capabilities and pricing change. Verify current details on the vendor's official website before making procurement decisions.