Skip to content

repo-guide

repo-guide is a command-line tool designed to analyze codebase structures and automatically generate detailed, static documentation sites using Large Language Models (LLMs). By extracting repository hierarchies, tracking metadata through Git, and prompting models (primarily Google Gemini) with structured contextual layouts, it programmatically drafts Markdown guides. It then compiles and hosts these guides locally or prepares them for remote static deployment (such as on GitHub Pages) using MkDocs.

This automated guide serves to complement, rather than replace, human-authored documentation by keeping deep subsystem details and directory structures continuously accurate and easy to navigate.

Dependencies

The project relies on a robust ecosystem of libraries to safely handle Git metadata, perform LLM calls, handle text tokenization, and generate/sanitize HTML output:

  • llm & llm-gemini: Manage the connections, prompts, and interface with LLM providers (defaulting to Google Gemini models).
  • click: Provides the structural command-line framework, parameter parsing, and execution controls.
  • gitpython: Inspects local Git repositories to track versioned files and map paths directly to remote blob URLs.
  • tiktoken: Counts tokens locally to respect user-configured token budgets, preventing accidental execution costs.
  • mkdocs & mkdocs-material: Configures, builds, and hosts the generated documentation in a responsive web format.
  • bleach & bleach-allowlist: Functions within MkDocs hooks to sanitize raw HTML generated by the LLM, protecting against cross-site scripting (XSS) or malformed structures.
  • tqdm: Renders clear progress bars during the multi-step generation phase.
  • magika (optional): Performs deep, deep-learning-based file type classification to safely exclude binary files.

Core Architecture and Execution Flow

The utility coordinates actions across multiple layers, passing context down to nested components to produce clean output.

                  ┌──────────────────────┐
                  │      CLI Entry       │ (cli.py / click)
                  └──────────┬───────────┘
                             │ (runs)
                             ▼
                  ┌──────────────────────┐
                  │    DocGenerator      │ (Git inspection &
                  └──────────┬───────────┘  token budgeting)
                             │
            ┌────────────────┴────────────────┐
            ▼                                 ▼
┌──────────────────────┐            ┌──────────────────────┐
│  Prompt Builder      │            │   LLM Generation     │ (exponential
│ (Hierarchical context│            │     & Token Check    │  backoff retry)
│   from directories)  │            └─────────┬────────────┘
└──────────────────────┘                      │ (writes MD output)
                                              ▼
                                    ┌──────────────────────┐
                                    │    MkDocs Build      │ (creates mkdocs.yml,
                                    │    & Local Server    │  serves/deploys site)
                                    └──────────────────────┘

1. Command-Line Entry

When run, the program enters via main.py within src/repo_guide, invoking the CLI defined in cli.py. Users specify options such as token budgets, ignore paths, target LLM models, or direct-to-deployment options.

2. Workspace & Git Inspection

The execution engine (cli.py) initializes its DocGenerator class. It uses Git metadata to identify relevant, tracked files. It filters out non-text configurations, binary assets, and user-ignored paths (optionally utilizing magika for advanced MIME-type evaluation).

3. Hierarchical Prompt Engineering

To help the LLM contextualize large file structures without overflowing the context window, DocGenerator uses a bottom-up/top-down logic. For any directory, it compiles file pathways and text contents alongside previously processed summaries of nested directories. This structures prompts in clear XML envelopes that are easy for the model to parse.

4. Model Orchestration & Backoff

DocGenerator dispatches raw context maps to the configured LLM API. It implements token verification through tiktoken to guard against cost overruns and wraps API invocations in an exponential backoff loop to gracefully handle rate-limit situations.

5. Compiling & Sanitizing the Site

Once written to disk, the utility writes a mkdocs.yml configuration and runs an asynchronous background thread to serve the live-reloading site. Custom hooks intercept the rendering pipeline, employing bleach to ensure any HTML inside generated Markdown is completely sanitized before the final output is compiled.

Repository Structure

Source Package

The src directory acts as the root of the source code. The core logic lives inside src/repo_guide, structured as follows:

  • init.py: Standard package initialization.
  • main.py: Script entry point that delegates execution straight to the CLI logic.
  • cli.py: Houses CLI flag definitions and the orchestrating DocGenerator engine. It handles token calculations, sub-process spawning, prompt generation, and MkDocs integration.

Test Suite

The tests folder verifies behavior without relying on active, paid internet requests. Its primary file is test_repo_guide.py. The test suite focuses on:

  • Mock Repositories: Spawning isolated temporary Git repos with nested directory structures, corrupted or unreadable byte files, and simulated GitHub remotes.
  • Prompt Validation: Asserting that internal prompt-builder methods construct correct XML contextual envelopes.
  • Mocking the LLM: Patching llm.get_model with local mock models that supply predictable replies.
  • Integration Tests: Validating CLI commands against end-to-end setups to confirm output markdown structure and file linkage.

GitHub Workflows

The .github directory controls continuous integration, testing, package publishing, and self-documentation inside .github/workflows:

  • test.yml: Automatically builds a test matrix (Python 3.11, 3.12, and 3.13) and runs pytest via uv on every push or pull request.
  • publish.yml: Uses secure Trusted Publishers (OIDC) authentication to upload release assets to PyPI.
  • publish_site.yml: Executes a "dogfooding" run. It spins up the repo-guide CLI itself on its own codebase, querying Gemini to rewrite this very documentation, compiling it into a clean layout, and publishing the live site onto GitHub Pages.

Key Root Configuration Files

  • pyproject.toml: Defines package metadata, version rules, command-line entry scripts (repo-guide = "repo_guide.cli:cli"), development tools, dependencies (such as click, gitpython, mkdocs, and optional magika), and Ruff configurations.
  • .gitattributes: Configures line-ending normalization for script environments across platforms (critical for preventing Git diff mutations during cross-platform execution).
  • .gitignore: Prevents tracking of Virtual Environments (.venv), temporary python caches (__pycache__), unit testing artifacts (.pytest_cache), or workspace-specific configurations (.vscode/).
  • .pre-commit-config.yaml: Automates syntax quality enforcement using ruff and basedpyright statically on pre-commits.
  • README.md: Serves as the developer and user onboarding manual, containing installation guides, setup guides, troubleshooting steps, and local testing configurations.