archeious/marchwarden

Fork 0

Table of Contents

Development Guide

Setup

Prerequisites
Installation
Environment Setup

Project Structure
Running Tests
Running the CLI
Development Workflow

1. Create a branch
2. Make changes
3. Commit
4. Test before pushing
5. Push and create PR

Debugging

Viewing trace logs
Debug logging
Interactive testing

Common Tasks

Adding a new tool to the researcher
Changing the research contract
Running cost analysis

Development Guide

Setup

Prerequisites

Python 3.10+
pip (with venv)
Tavily API key (free tier available at https://tavily.com)

Installation

git clone https://forgejo.labbity.unbiasedgeek.com/archeious/marchwarden.git
cd marchwarden

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in dev mode
pip install -e ".[dev]"

Environment Setup

Create a .env file in the project root:

TAVILY_API_KEY=<your-tavily-api-key>
ANTHROPIC_API_KEY=<your-claude-api-key>
MARCHWARDEN_TRACE_DIR=~/.marchwarden/traces

Test that everything works:

python -c "from anthropic import Anthropic; print('OK')"
python -c "from tavily import TavilyClient; print('OK')"

Project Structure

marchwarden/
├── researchers/
│   ├── __init__.py
│   └── web/                    # V1: Web search researcher
│       ├── __init__.py
│       ├── server.py           # MCP server entry point
│       ├── agent.py            # Inner research agent
│       ├── models.py           # Pydantic models (ResearchResult, Citation, etc)
│       └── tools.py            # Tavily integration, URL fetch
├── orchestrator/               # (V2+) PI agent
│   ├── __init__.py
│   └── pi.py
├── cli/                        # CLI shim (ask, replay)
│   ├── __init__.py
│   ├── main.py                 # Entry point (@click decorators)
│   └── formatter.py            # Pretty-print results
├── tests/
│   ├── __init__.py
│   ├── test_web_researcher.py
│   └── fixtures/
├── docs/
│   └── wiki/                   # You are here
├── README.md
├── CONTRIBUTING.md
├── pyproject.toml
└── .gitignore

Running Tests

# Run all tests
pytest tests/

# Run with verbose output
pytest tests/ -v

# Run a specific test file
pytest tests/test_web_researcher.py

# Run with coverage
pytest --cov=. tests/

All tests are unit + integration. We do not mock the database or major external services (only Tavily if needed to avoid API costs).

Running the CLI

# Ask a question
marchwarden ask "What are ideal crops for a garden in Utah?"

# With options
marchwarden ask "What is X?" --depth deep --budget 25000

# Replay a trace
marchwarden replay <trace_id>

# Show help
marchwarden --help

The first run will take a few seconds (agent planning + searches + fetches).

Development Workflow

1. Create a branch

git checkout -b feat/your-feature-name

Branch naming: feat/, fix/, refactor/, chore/ + short description.

2. Make changes

Edit code, add tests:

# Run tests as you go
pytest tests/test_your_feature.py -v

# Check formatting
black --check .
ruff check .

# Type checking (optional, informational)
mypy . --ignore-missing-imports

3. Commit

git add <files>
git commit -m "Brief imperative description

- What changed
- Why it changed
"

Commits should be atomic (one logical change per commit).

4. Test before pushing

pytest tests/
black .
ruff check . --fix

5. Push and create PR

git push origin feat/your-feature-name

Then on Forgejo: open a PR, request review, wait for CI/tests to pass.

Once approved:

Merge via Forgejo UI (not locally)
Delete remote branch via Forgejo
Locally: git checkout main && git pull --ff-only && git branch -d feat/your-feature-name

Debugging

Viewing trace logs

# Human-readable trace
marchwarden replay <trace_id>

# Raw JSON
cat ~/.marchwarden/traces/<trace_id>.jsonl | jq .

# Pretty-print all lines
cat ~/.marchwarden/traces/<trace_id>.jsonl | jq . -s

Debug logging

Set MARCHWARDEN_DEBUG=1 for verbose logs:

MARCHWARDEN_DEBUG=1 marchwarden ask "What is X?"

Interactive testing

Use Python REPL:

python
>>> from researchers.web import WebResearcher
>>> researcher = WebResearcher()
>>> result = researcher.research("What is X?")
>>> print(result.answer)

Common Tasks

Adding a new tool to the researcher

Define the tool in researchers/web/tools.py
Register it in the agent's tool list (researchers/web/agent.py)
Add test coverage in tests/test_web_researcher.py
Update docs if it changes the contract

Changing the research contract

If you need to modify the research() signature:

Update researchers/web/models.py (ResearchResult, Citation, etc)
Update researchers/web/agent.py to produce the new fields
Update docs/wiki/ResearchContract.md
Add a migration guide if breaking
Tests must pass with new signature

Running cost analysis

See how much a research call costs:

marchwarden ask "Q" --verbose
# Shows: tokens_used, iterations_run, wall_time_sec

For batch analysis:

import json
import glob
for trace_file in glob.glob("~/.marchwarden/traces/*.jsonl"):
    for line in open(trace_file):
        event = json.loads(line)
        # Analyze cost_metadata

FAQ

Q: How do I add a new researcher?
A: Create researchers/new_source/ with the same structure as researchers/web/. Implement research(), expose it as an MCP server. Test with the CLI.

Q: Do I need to handle Tavily failures?
A: Yes. Catch TavilyError and fall back to what you have. Document in gaps.

Q: What if Anthropic API goes down?
A: The agent will fail. Retry logic TBD. For now, it's a blocker.

Q: How do I deploy this?
A: V1 is CLI-only, local use only. V2 will have a PI orchestrator with real deployment needs.