Building a Fast Local Document Search CLI with Python and ChromaDB

Why Build a Local Semantic Search Tool?

Developers accumulate hundreds of markdown notes, READMEs, and technical runbooks. While grep is great for exact strings, semantic search allows you to ask conceptual questions like "how do I reset database permissions" and surface relevant documentation instantly.

Step 1: Setting Up the Virtual Environment

Create a dedicated directory and install ChromaDB with sentence-transformers:

mkdir ~/doc-search && cd ~/doc-search
python3 -m venv .venv
source .venv/bin/activate
pip install chromadb sentence-transformers

Step 2: Building the Indexer Script

Create a script named index_notes.py to embed text files into a persistent local vector store:

import os
import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection(name="linux_notes")

def index_folder(folder_path):
    documents = []
    ids = []
    metadatas = []

    for filename in os.listdir(folder_path):
        if filename.endswith(".md") or filename.endswith(".txt"):
            filepath = os.path.join(folder_path, filename)
            with open(filepath, "r", encoding="utf-8", errors="ignore") as f:
                content = f.read().strip()
                if content:
                    documents.append(content)
                    ids.append(filename)
                    metadatas.append({"filename": filename})

    if documents:
        collection.upsert(documents=documents, ids=ids, metadatas=metadatas)
        print(f"Indexed {len(documents)} documents successfully.")

if __name__ == "__main__":
    index_folder("./my_notes")

Step 3: Creating the Query CLI

Create a script named search.py to query the vector database from your terminal:

import sys
import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_collection(name="linux_notes")

query = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else "backup database"
results = collection.query(query_texts=[query], n_results=3)

print(f"\n=== Results for: '{query}' ===")
for i, (doc, meta) in enumerate(zip(results["documents"][0], results["metadatas"][0])):
    print(f"\n[{i+1}] Source: {meta['filename']}")
    print(doc[:250] + "...")

You can now index and query your internal technical documentation directly from your shell in milliseconds.