Manomenu /

suwalski-agent-eval

Evaluation framework for AI agents: scenarios as YAML in the agent's repo (including simulated users), checks on tool calls, events…

A new way for AI to search for data
View on GitHub
1GitHub stars

C#, TypeScript

What XERJ indexed

Snapshot

Whole repository

The whole repository at commit 6b39b8deb7 (2026-10-19).

Selected work

Piotr Gugnowski (Maniumek)
@Manomenu

“[feature,refactor,test,docs] added CLI commands for agents, scenarios…”. 78 files, 5,027 lines in Suwageval.Web, Suwageval.Cli, Suwageval.Cli.Tests.

Licence

The repository

MIT

Checked by a person at the pinned commit.

The XERJ corpus

MIT, with credit

The author agreed by email on 2026-10-23. Same terms as the repository. Answers name the repo, the commit and the file:line.

in licence reviewnot in the hub yetsuggested use: adapt with attribution

agent-evaluationai-agentsaspnetcoreclaude-codedotnetevaluation-frameworkllmllm-as-a-judge

Example questions

The prompt

Paste it into your AI agent. It clones Manomenu/suwalski-agent-eval at the pinned commit and indexes it on your machine.

Search the GitHub repository Manomenu/suwalski-agent-eval with XERJ (https://hub.xerj.org/corpus/manomenu--suwalski-agent-eval).
Run these in a shell, in order, and check each result before the next:
1. xerj --help  (success: it prints usage). If xerj is missing: curl -fsSL https://xerj.org/get | sh
2. git clone https://github.com/Manomenu/suwalski-agent-eval suwalski-agent-eval && git -C suwalski-agent-eval checkout 6b39b8deb7b1cdf4489137475e8f453d4a05b7bf
3. Start a local node and leave it running: xerj --insecure --data-dir ./xerj-data
4. xerj autoindex ./suwalski-agent-eval --url http://127.0.0.1:9200 --prefix suwalski-agent-eval --progress plain
5. xerj autoindex map --url http://127.0.0.1:9200
Then answer my question from the indexed files and cite the ax_path of every hit. Query syntax: https://xerj.org/llms.txt
Licence: MIT. Keep the copyright notice and credit the authors.

Related

Piotr Gugnowski (Maniumek)AI, prompts and agents corporaxerj-blogpostsanything-llmprompt-engineering-guidericfanin/netwrightGrit-77/done-is-a-claimntatschner/loadout-cli

Selected work and consent: the XERJ registry.