Commit graph

75 commits

Author SHA1 Message Date
Jeff Smith
ea8c07a692 refactor: extract system prompts into luminos_lib/prompts.py
Moves _DIR_SYSTEM_PROMPT and _SYNTHESIS_SYSTEM_PROMPT from ai.py into
a dedicated prompts module. Both are pure template strings with .format()
placeholders — no runtime imports needed in prompts.py. Prompt content
is byte-for-byte identical to the original.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 14:44:45 -06:00
Jeff Smith
5c6124a715 merge: extract AST parser module 2026-03-30 14:34:06 -06:00
Jeff Smith
0c49da23ab refactor: extract AST parsing into luminos_lib/ast_parser.py
Moves all tree-sitter parsing logic from ai.py into a dedicated module.
Replaces the if/elif language chain with a _LANGUAGE_HANDLERS registry
mapping language names to handler functions.

Extracted: _tool_parse_structure body, _get_ts_parser, _child_by_type,
_text, and all per-language helpers (_py_func_sig, _py_class, etc.).
ai.py retains a thin wrapper for path validation.

Public API: parse_structure(path) -> JSON string

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 14:34:02 -06:00
Jeff Smith
8aa6c713db merge: post-cache-extraction cleanup 2026-03-30 13:52:43 -06:00
Jeff Smith
dceff144b6 chore: remove dead clear_cache from ai.py, deduplicate CACHE_ROOT
- Delete unused clear_cache() from ai.py (luminos.py imports it from
  capabilities.py)
- Remove CACHE_ROOT import from ai.py (was only used by dead function)
- Replace local CACHE_ROOT constant in capabilities.py with import
  from cache.py (single source of truth)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 13:52:38 -06:00
Jeff Smith
811fe3514c merge: extract cache management module 2026-03-30 13:12:42 -06:00
Jeff Smith
bbd04f41a7 refactor: extract cache management into luminos_lib/cache.py
Moves investigation ID persistence and _CacheManager class from ai.py
into a dedicated cache module. No behavior changes.

Moved: _load_investigations, _save_investigations, _get_investigation_id,
_CacheManager (all methods), _sha256_path, CACHE_ROOT, INVESTIGATIONS_PATH.

Also added a local _now_iso() in cache.py to avoid a circular import
(ai.py imports from cache.py).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 13:12:37 -06:00
Jeff Smith
a7546fa1e8 merge: chain-of-thought observability tools 2026-03-30 13:02:25 -06:00
Jeff Smith
f324648c10 feat: add chain-of-thought observability tools
Adds think, checkpoint, and flag tools for agent reasoning visibility:
- think: records observation/hypothesis/next_action before investigation
- checkpoint: summarizes learned/unknown/next_phase after file clusters
- flag: marks notable findings to flags.jsonl with severity levels

Additional changes:
- Step numbering in investigation system prompt
- Text blocks from agent now printed to stderr (step labels visible)
- flag tool available in both investigation and synthesis passes
- analyze_directory() returns (brief, detailed, flags) three-tuple
- format_flags() in report.py renders flags sorted by severity
- Per-directory max_turns increased from 10 to 14

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 13:02:19 -06:00
Jeff Smith
dd58a4fd3a merge: fix token budget early exit cache flush 2026-03-30 12:17:33 -06:00
Jeff Smith
2e2c64386f fix: flush partial directory cache on context budget early exit
When the 70% context budget is hit mid-directory, the early exit now
writes a partial directory cache entry from whatever file summaries
the agent cached in prior turns, instead of discarding the work.

If file entries exist: concatenates their summaries into a directory
entry marked partial=true. If no files were cached: writes a minimal
entry noting the budget was reached before processing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:17:28 -06:00
Jeff Smith
faccf28dd8 chore: document git workflow conventions in CLAUDE.md
Adds branch naming, commit message format, and merge procedure.
All future changes must start on a branch and merge to main with --no-ff.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:14:34 -06:00
Jeff Smith
da387289f3 chore: add venv setup script and update CLAUDE.md for optional deps
- setup_env.sh creates ~/luminos-env venv and installs all AI packages
- CLAUDE.md updated to reflect the new dependency model: base tool is
  zero-dep, --ai requires packages installed via venv
- Documents the capabilities module and updated ai.py architecture

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:14:13 -06:00
Jeff Smith
0412a8c0cb feat: add --fresh, --clear-cache, and --install-extras CLI flags
- --install-extras: prints status of all optional AI packages
- --clear-cache: wipes /tmp/luminos/ investigation cache
- --fresh: forces a new investigation ID, ignoring cached results
- AI import is now lazy (only when --ai is used) so the base tool
  never touches optional dependencies
- target argument is optional when using --install-extras

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:14:05 -06:00
Jeff Smith
907dcf0a37 refactor: replace single-shot API with multi-pass agentic investigation
Rewrites ai.py from a single Claude API call into a multi-pass,
cache-driven agent architecture:

- Per-directory isolated agent loops (max 10 turns each) with context
  discarded between directories
- Leaves-first processing order so child summaries inform parents
- Disk cache (/tmp/luminos/{uuid}/) persists across runs for resumability
- Investigation ID persistence keyed by target realpath
- Separate synthesis pass reads only directory-level cache entries
- Replaces urllib with Anthropic SDK (streaming, automatic retries)
- Token counting with 70% context budget threshold for early exit
- parse_structure tool via tree-sitter (Python, JS, Rust, Go)
- python-magic integration for MIME-aware directory listings
- Cost tracking printed at end of investigation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:13:55 -06:00
Jeff Smith
2ba07f34a2 feat: add capabilities detection module for optional AI dependencies
Introduces luminos_lib/capabilities.py as the single source of truth for
optional package availability. Detects anthropic, tree-sitter, python-magic
and their grammar packages. Provides check_ai_dependencies() for gating
--ai mode and print_status() for --install-extras. Also hosts clear_cache()
to avoid pulling heavy AI imports for cache cleanup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:13:40 -06:00
Jeff Smith
bcf7d12b4a feat: add AI-powered directory analysis via Claude API
Adds --ai flag that sends the directory tree, file categories, and
sampled file contents to Claude for analysis. Produces a brief
summary at the top of the report and a detailed breakdown at the
end. Requires ANTHROPIC_API_KEY env var; degrades gracefully without it.
Uses only stdlib (urllib) to keep the zero-dependency constraint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 10:03:48 -06:00
Jeff Smith
d6f36ecea5 feat: add --watch mode with change diffing
Re-scans every 30 seconds and shows new files, deleted files, and
size changes between scans.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:39 -06:00
Jeff Smith
2c0a9cf872 feat: add formatted report output with --json and --output flags
Human-readable terminal report with clear sections, plus JSON output
mode and file output support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:35 -06:00
Jeff Smith
9cba2be54e feat: add disk usage summary
Uses du to show per-directory disk usage and highlights the top 5
largest directories.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:31 -06:00
Jeff Smith
ae891357cc feat: add recency analysis
Finds the 10 most recently modified files using find with printf
and shows human-readable timestamps.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:25 -06:00
Jeff Smith
07d96faf90 feat: add code detection and line counting
Detects programming languages, counts lines of code per language via
wc -l, and flags unusually large files (>1000 lines or >10MB).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:22 -06:00
Jeff Smith
610fb55367 feat: add file type intelligence
Classifies files by category (source, config, data, media, document,
archive, unknown) using extension mapping and the `file` command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:18 -06:00
Jeff Smith
35ededc06b feat: add directory tree visualization
Renders a visual tree with file sizes, configurable depth, and
hidden file filtering.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:14 -06:00
Jeff Smith
461bdc404e chore: initial project scaffold
Set up Python project structure with entry point and library package.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:57:11 -06:00