crawlingo
Generate validated datasets from web pages with a Rust engine, Python and Node.js SDKs, exact deduplication, and resumable JSONL/CSV…
What XERJ indexed
Selected work
“validate research datasets against a public directory”. 10 files, 779 lines in measurements, scripts, src/parser.
Licence
The repository
Checked by a person at the pinned commit.
The XERJ corpus
The author agreed by email on 2026-10-20. Same terms as the repository. Answers name the repo, the commit and the file:line.
in licence reviewnot in the hub yetsuggested use: adapt with attribution
nodejscsvdata-extractiondataset-generationetljsonlpythonrust
Example questions
- How does crawlingo implement nodejs?
- Where does crawlingo handle “validate research datasets against a public directory”?
- What does measurements do in crawlingo?
The prompt
Paste it into your AI agent. It clones Vamshavardhan50/crawlingo at the pinned commit and indexes it on your machine.
Search the GitHub repository Vamshavardhan50/crawlingo with XERJ (https://hub.xerj.org/corpus/vamshavardhan50--crawlingo). Run these in a shell, in order, and check each result before the next: 1. xerj --help (success: it prints usage). If xerj is missing: curl -fsSL https://xerj.org/get | sh 2. git clone https://github.com/Vamshavardhan50/crawlingo crawlingo && git -C crawlingo checkout b258d6236c2deacd6ed5f8ed463000548e7e9358 3. Start a local node and leave it running: xerj --insecure --data-dir ./xerj-data 4. xerj autoindex ./crawlingo --url http://127.0.0.1:9200 --prefix crawlingo --progress plain 5. xerj autoindex map --url http://127.0.0.1:9200 Then answer my question from the indexed files and cite the ax_path of every hit. Query syntax: https://xerj.org/llms.txt Licence: MIT. Keep the copyright notice and credit the authors.
Related
Chintha Vamsha vardhanWeb and frontend platform corporachromadelta-rsforyemiliano-go/rypipeloribel-labs/tabalystNagareWorks/metricraft
Selected work and consent: the XERJ registry.