Better LLM applications
Toor is an open-source TypeScript toolkit for experimentation, LLM-as-a-judge evaluation, prompt optimization, structured outputs, datasets, and benchmarks.
Flexible by Design
Build AI workflows in TypeScript or define them with simple configuration file — choose the approach that fits your project.
·
TypeScript code
·
YAML or JSON configuration
Evaluate and Experiment
Build, run, and automate reproducible LLM evaluations with LLM-as-a-judge, benchmarks, datasets, and experiments to compare prompts, validate improvements, and ship reliable AI systems faster.
·
LLM-as-a-judge evaluators for binary and scalar outcomes
·
Experiments with reasoning and structured evaluation
Fully Customizable
Adapt every part of the evaluation pipeline to your project — from judge prompts and scoring metrics to completely custom evaluators.
·
Multiple built-in scoring scales
·
User-defined metrics
·
Custom LLM judge prompts
·
Pluggable evaluators
What's Next
The foundation is here. Next, we're expanding Toor to support increasingly capable AI systems — from prompt optimization to agents that use tools and external services.
·
GEPA for automatic prompt optimization
·
Tool calling support
·
MCP server support
Ready to Build?
Explore the documentation to get started, or dive into the source code to see how Toor works under the hood.