Table of Contents

Architecture

Overview
Module Map
Base Scan Data Flow
AI Pipeline (--ai flag)
Cache
Key Constraints
AI Model

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Architecture

This page is the high-level map. For a code-level walkthrough with file:line references — how the dir loop actually works, where to add a tool, what the cache really stores — read Internals.

Overview

Luminos is a zero-dependency Python CLI at its base. The --ai flag layers an agentic investigation on top using the Claude API. The two layers are strictly separated — the base scan never requires pip packages.

Entry point: luminos.py — argument parsing, scan orchestration, output routing.

Module Map

Module	Purpose	External commands
`luminos.py`	Entry point — arg parsing, scan(), main()	None
`luminos_lib/tree.py`	Recursive directory tree with file sizes	None (os)
`luminos_lib/filetypes.py`	Classifies files into 7 categories	`file --brief`
`luminos_lib/code.py`	Language detection, LOC counting, large file flagging	`wc -l`
`luminos_lib/recency.py`	Finds N most recently modified files	`find -printf`
`luminos_lib/disk.py`	Per-directory disk usage	`du -b`
`luminos_lib/report.py`	Formats report dict as terminal output	None
`luminos_lib/watch.py`	Continuous monitoring loop with snapshot diffing	None
`luminos_lib/capabilities.py`	Optional dependency detection, cache cleanup	None
`luminos_lib/cache.py`	AI investigation cache — read/write/clear/flush	None
`luminos_lib/ast_parser.py`	tree-sitter code structure parsing	tree-sitter
`luminos_lib/prompts.py`	System prompt templates for AI loops	None
`luminos_lib/ai.py`	Multi-pass agentic analysis via Claude API	anthropic, python-magic

Base Scan Data Flow

scan(target)
    build_tree()           → report["tree"], report["tree_rendered"]
    classify_files()       → report["file_categories"], report["classified_files"]
    detect_languages()     → report["languages"], report["lines_of_code"]
    find_large_files()     → report["large_files"]
    find_recent_files()    → report["recent_files"]
    get_disk_usage()       → report["disk_usage"], report["top_directories"]
    └── returns report dict

AI Pipeline (--ai flag)

analyze_directory(report, target)
    │
    └── _run_investigation()
            │
            ├── _get_investigation_id()    new UUID, or resume an existing one
            │
            ├── _discover_directories()    find all dirs, sort leaves-first
            │
            ├── _run_survey()              single short loop, max 3 turns
            │       inputs: survey_signals + 2-level tree preview
            │       Tools: submit_survey
            │       output: shared description, approach, relevant_tools,
            │       skip_tools, domain_notes, confidence
            │       (skipped via _default_survey() on tiny targets)
            │
            ├── _filter_dir_tools(survey)  remove skip_tools (if confidence ≥ 0.5)
            │
            ├── per-directory loop (each uncached dir, up to max_turns=14)
            │       _build_dir_context()    list files + sizes + MIME
            │       _get_child_summaries()  read cached child summaries
            │       _format_survey_block()  inject survey context into prompt
            │       _run_dir_loop()         agent loop with budget check on
            │                               every iteration; flushes a partial
            │                               cache entry on budget breach
            │       Tools: read_file, list_directory, run_command,
            │              parse_structure, write_cache, think, checkpoint,
            │              flag, submit_report
            │
            ├── _run_synthesis()            single loop, max 5 turns
            │       reads all "dir" cache entries
            │       produces brief (2-4 sentences) + detailed (free-form)
            │       Tools: read_cache, list_cache, flag, submit_report
            │       fallback: _synthesize_from_cache() if out of turns
            │
            └── returns (brief, detailed, flags)

Token usage and the context budget are tracked by _TokenTracker in ai.py. The budget check uses the most recent call's input_tokens, not the cumulative sum across turns — see #44 and the Internals page §4.4 for why.

Cache

Location: /tmp/luminos/<investigation_id>/

Layout:

meta.json              investigation metadata
files/<sha256>.json    one JSON file per cached file entry
dirs/<sha256>.json     one JSON file per cached directory entry
flags.jsonl            JSONL — appended on every flag tool call
investigation.log     JSONL — appended on every tool call

File and dir entries are stored as one sha256-keyed JSON file per entry (not as JSONL) so that has_entry(path) is an O(1) os.path.exists() check rather than a file scan. Only flags.jsonl and investigation.log are JSONL.

File entries (files/<sha256>.json):

{path, relative_path, size_bytes, category, summary, cached_at,
 [confidence], [confidence_reason], [notable], [notable_reason]}

Dir entries (dirs/<sha256>.json):

{path, relative_path, child_count, dominant_category, summary, cached_at,
 [confidence], [confidence_reason], [notable_files],
 [partial], [partial_reason]}

partial: true marks a dir entry written by the budget-breach early-exit path — the agent didn't reach submit_report and the summary was synthesized from already-cached file entries.

Flags (flags.jsonl):

{path, finding, severity}   severity: info | concern | critical

Investigation IDs are persisted in /tmp/luminos/investigations.json keyed by absolute target path. Cache is reused across runs for the same target. --fresh mints a new investigation ID. --clear-cache deletes the entire cache root.

Key Constraints

Base tool: no pip dependencies. tree, filetypes, code, disk, recency, report, watch use only stdlib and GNU coreutils.
AI deps are lazy. anthropic, tree-sitter, python-magic imported only when --ai is used. Missing packages produce a clear install error.
Subprocess for OS tools. LOC counting, file detection, disk usage, and recency shell out to GNU coreutils. Do not reimplement in pure Python.
Graceful degradation everywhere. Permission denied, subprocess timeouts, missing API key — all handled without crashing.

AI Model

claude-sonnet-4-20250514

Context budget. 70% of 200,000 tokens (140,000) — Sonnet 4's real context window with a 30% safety margin. The budget is checked against the latest per-call input_tokens reading (the actual size of the context window in use), not the cumulative sum across turns. Early exit flushes partial cache on budget breach. See #44.

Per-loop turn cap. Each dir loop runs for at most max_turns = 14 turns. This is a sanity bound separate from the context budget — even on small targets the agent should produce a submit_report long before exhausting 14 turns. The cap exists to prevent runaway loops when the agent gets stuck (e.g. repeatedly retrying a failing tool call). If we observe legitimate investigations consistently hitting 14, raise the cap; do not raise it speculatively.

Per-loop message history growth. Tool results are appended to the message history and never evicted, so per-turn input_tokens grows roughly linearly across a loop (~1.5–2k per turn observed on codebase targets). At the current max_turns=14 cap this stays well under 200k. Raising max_turns significantly (e.g. via Phase 3 dynamic turn allocation) would expose this — see #51.

Pricing tracked and reported at end of each run.