Open SourceLocal-first RAG

RAG, Smaller
and Closer to Hand.

A local-first RAG toolkit that runs entirely on one PC.

mrag is an open-source command-line tool for building and operating a small RAG knowledge base on your own PC. Ingest business documents such as internal rules, manuals and specifications, build an index, and make them searchable from generative AI such as Claude Code and Dify. Documents, index and retrieval all stay local. Nothing is sent to the cloud.

View on GitHub Talk to us about adoption

LicenseMIT
RuntimePython 3.11+
ModelsOllama (bge-m3 and others)
IndexSQLite FTS5 + Qdrant
InputsMarkdown / Text
InterfacesCLI · HTTP API · MCP

mrag — my-kb

Create a project, add documents, build the index, search. Four commands.

Turn just what your work needs into knowledge.

You don't have to wait for a company-wide platform to build a knowledge base from the documents you work with today.
mrag is a tool for turning the documents you need, and only those, into a small knowledge base on your own PC, fast.

Before With mrag

You have documents worth indexing, but first you need a vector database contract and an embedding API, and it takes a while before you can even try.

mrag init → add → index → searchFour commands, and you are searching the same day.

Loading a department's small set of documents onto a large company-wide RAG platform is overkill, and you end up waiting in line.

mrag init kb-manual / kb-spec / kb-rulesBuild small knowledge bases per department, team or person, with only the documents you need. Add more and bundle them later.

"Why did it answer that?" is a black box, with no clues for improving accuracy.

mrag inspect chunk <id> / mrag eval "query"See the chunks, the added context and why something matched, then tune.

Small, and quick to build

Four commands to learn. A folder of manuals or specifications for your own work becomes a searchable knowledge base the same day.

One PC, nothing leaves the building

Runs on Python, SQLite, Qdrant and Ollama alone. Qdrant runs in-process by default, so there is no server or container to set up. Documents, index and retrieval all stay on your PC, and embeddings and context generation use local models.

Grow it and bundle it when you need to

Check how documents were split, what context was added and why a result matched with mrag inspect and mrag eval, and switch retrieval strategies and chunking in YAML. When knowledge bases multiply, bundle them with mrag registry and let agents pick.

A RAG pipeline that runs entirely on your machine.

Ingest, split, add context, embed, search.
Everything inside the dashed boxes (document conversion and the agents that use the results) is external; everything else is mrag and Ollama.

Documents PDF / Office / Markdown / Text

docling / MarkItDown Convert to Markdown (external tools)

mrag add Register documents

mrag index Chunk → add context → embed

SQLite FTS5 Source of truth · keyword index

Qdrant Vector index (in-process)

mrag search Hybrid retrieval → rerank

CLI / REST API / Dify API / MCP mrag serve · mrag mcp

AI agents Claude Code · Codex · Dify · LM Studio

Ollamabge-m3 (embeddings) · gemma4 (context generation) · runs locally

SQLite is the source of truth; Qdrant is a rebuildable vector index. In the default local mode Qdrant runs in-process, so no separate server is needed.

Four commands to a knowledge base.

Create a project, add documents, build the index, search. That's all.

mrag init my-kb --non-interactive
cd my-kb
mrag add /path/to/documents --recursive --include '**/*.md'
mrag index
mrag search "your query"

1
init — create a projectGenerates mrag.yaml, a retrieval profile, kb_information.yaml and mrag.db. The Japanese tokenizer is detected automatically.
2
add — register documentsRegister Markdown or text files one at a time or by directory. Preview the selection with --dry-run. Nothing is searchable yet at this point.
3
index — build the indexSplits documents into chunks, computes embeddings with Ollama and writes them to Qdrant and SQLite FTS5. Later runs process only what was added.
4
search — query itHybrid retrieval, fusing keyword and vector search, is the default. Switch with --strategy, and hand results to an agent with --json.

Convert PDF and Office documents to Markdown with docling or MarkItDown before adding them. See an integration example here. For installation, see the README on GitHub; to let an agent such as Claude Code do it, point it at the bundled SETUP.md.

Accuracy grows with configuration.

Choose how documents are split and how they are searched to fit the content and the use. Settings live in a profile (YAML), so you can switch and compare.

Four chunking strategies

General-purpose recursive, heading-aware markdown_recursive, block_aware to keep tables and code intact, and parent_child to match on children and return parents.

Four retrieval strategies

Keyword (BM25), vector (semantic), hybrid fusing the two (default), and parent_child. Switch per query with --strategy.

Contextual retrieval

Each chunk gets document-level context from a local LLM before embedding, so searches that a bare fragment would miss are found. Follows Anthropic's Contextual Retrieval approach.

Reranking

A CrossEncoder reorders the top candidates. Helps when the right passage is retrieved but ranked too low.

Evaluation and inspection

Measure retrieval quality with mrag eval, and look at chunk bodies, generated context and section hierarchy with mrag inspect.

Strong in Japanese

Supports a Japanese morphological tokenizer. The multilingual bge-m3 model is the default for embeddings.

Built by agents, used by agents.

mrag is designed to be operated by AI agents.

Agents build it, guided by SKILL.md

Point Claude Code or a similar agent at the bundled SKILL.md and SETUP.md and it handles installation, knowledge-base creation and accuracy tuning on its own.

# Example prompt for Claude Code
> Create an mrag project at ./my-kb from the
  Markdown files under ./documents/

Expose it as an MCP server

mrag mcp turns a project into a read-only MCP server, so Claude Code, LM Studio and other clients can call search as a tool. Supports stdio and Streamable HTTP.

# inside an indexed project
mrag mcp
# tools: search / list_documents / inspect_*

External knowledge for Dify

mrag serve provides an endpoint compatible with the Dify External Knowledge API, so a Dify chatbot can query a local knowledge base. A native REST API is also available.

mrag serve --host 0.0.0.0 --port 8000
# POST /retrieval  (Dify External Knowledge API)

Bundle multiple knowledge bases

mrag registry gathers several knowledge bases into knowledge_registry.yaml. Agents decide which base to search before searching. A foundation for agentic RAG.

mrag registry generate ./knowledges
# knowledges/kb-manual, kb-spec, kb-rules ...

Connects to

Claude Code
Codex
Gemini CLI
LM Studio
Dify
Any MCP client

Where mrag fits

Knowledge for a department or team
When the manuals, rules and specifications of your own work are worth indexing but not worth a company-wide platform.
Trying it right now
When you want to run RAG on the documents in a folder today and see whether it helps, without waiting for contracts or infrastructure.
Documents that can't leave the company
When confidential documents must stay on a PC or an on-premises server while still being usable by generative AI.
Giving agents domain knowledge
When you want Claude Code, Dify, LM Studio or another agent to have knowledge specific to your work, delivered over MCP or an API.

What you can do with it

Questions about internal rules and manuals

"What is the cap on travel expenses?" — answered by searching the rules, with the supporting passage.

rules .md → mrag → Claude Code

Q&A over product specs and design docs

Give your development agent your own product's specifications as the basis for implementation and support answers.

specs → mrag mcp → agent

Maintenance and operations procedures

Turn field procedures into a knowledge base and let an operator's agent pull up the right step.

procedures → mrag → LM Studio

Accumulating minutes and notes

Keep adding and indexing daily Markdown. Retire outdated documents from search with exclusions while keeping the originals.

add → index → exclusions

Local knowledge for a Dify chatbot

Use mrag as the external knowledge of an existing Dify bot and answer with documents that stay in-house.

mrag serve → Dify External Knowledge

A RAG test bench

Switch chunking and retrieval strategies and compare accuracy on your own documents. Evidence for designing your production RAG.

profiles → eval → inspect

Run it where it fits

mrag is not a platform you build big from day one. Start small on your own machine and extend to sharing and system integration as the need arises.

Personal

On your own PC first

Turn your research notes or the documents of your own work into a knowledge base and use it from Claude Code or LM Studio.

Connect viamrag mcp (stdio) / mrag search --json

Team

Share with your team

Keep mrag running on an in-house server or a small PC and let agents and Dify on the LAN query it.

Connect viamrag serve / mrag mcp (Streamable HTTP)

System

Embed in business systems

Add search to your own applications through the REST API, bundle several knowledge bases with the registry, and design update and exclusion operations.

Connect viaREST API / registry / operations design

The same project layout grows with you wherever it runs. From team sharing onward is where BathTimeFish's services come in.

Only from BathTimeFish

Because it's OSS, rely on pros only where you need to

mrag is free to use as open source. Serious in-house use, though, calls for judgment specific to your documents and your work: choosing and converting source documents, designing chunking and retrieval, evaluating accuracy, connecting agents and existing systems, and operating updates and exclusions. BathTimeFish helps companies adopt and systematize in-house RAG knowledge with mrag.

Free · OSS

What you can do yourselves

Use mrag (MIT License)
Build and search knowledge bases on your PC
Switch chunking and retrieval strategies via profiles
Publish over MCP, the REST API or the Dify-compatible API
Let agents build it with SKILL.md

Paid · BathTimeFish

What you can ask the developers for

Proof of concept — selecting and converting source documents, building the knowledge base, evaluating retrieval quality
Accuracy tuning — designing chunking, retrieval strategies, contextual augmentation and reranking
Integration with business systems — connecting agents such as Claude Code, Dify integration, application access over MCP or the REST API
Operations design — document update and exclusion workflows, multi-knowledge-base layout, running on-premises or on in-house servers
Internal rollout — user guides and hands-on sessions for your staff

Contact BathTimeFish

As the developer of mrag, BathTimeFish provides adoption support, accuracy design, system integration and operations design. We work with you from building small and confirming accuracy through to embedding it in your work.

FAQ

Yes. mrag is open-source software under the MIT License, and commercial use is allowed.

Not for normal use. Embeddings and context generation run on local models through Ollama. A network connection is needed only to download the models the first time.

mrag ingests Markdown and plain text. Convert PDFs with docling and Office documents with MarkItDown or a similar tool to Markdown first.

Python 3.11 or later and Ollama (with ollama serve running). Qdrant runs in-process by default, so no separate server is required.

It supports a Japanese-capable embedding model (bge-m3) and a Japanese morphological tokenizer (sqlite-vaporetto). Accuracy depends on the documents and the configuration, so tune while checking with mrag eval.

Through mrag mcp, any MCP client such as Claude Code, Codex or LM Studio can search it. Dify connects through the External Knowledge API. A REST API is also provided.

mrag is designed for operating small knowledge bases locally. For larger document sets we recommend splitting them into several knowledge bases by business area or topic and bundling them with mrag registry.

Yes. BathTimeFish provides consulting for adopting and systematizing in-house RAG with mrag. Please reach out through the contact form.