archeious/marchwarden
1
0
Fork 0
Code Issues 14 Pull requests Projects Releases Packages Wiki Activity Actions

M5.2 Contract validation across two researchers #50

New issue
Open
opened 2026-04-08 23:24:44 +00:00 by claude-code · 0 comments
claude-code commented 2026-04-08 23:24:44 +00:00
Collaborator
Copy link

Phase 5 — Second Researcher, milestone 2. Closes Phase 5.

Goal

Empirically validate that the v1 ResearchContract composes cleanly across researcher types now that we have two of them (web + arxiv-rag).

Process

  1. Pick 5–10 questions that should be answerable by both researchers (ML/AI topics where blog posts and arxiv papers both have good coverage)
  2. Run each through --researcher web and --researcher arxiv independently
  3. Compare side-by-side:
    • Do the contracts have the same shape? (yes — they should, by construction)
    • Do the answers complement or contradict?
    • Could a hypothetical PI orchestrator merge them mechanically?
  4. Identify any contract changes needed — backward-compatible additions only (no field renames or type changes)
  5. Document findings in the wiki

Deliverable

  • A side-by-side comparison table for the test queries
  • A "lessons learned" section in the ResearchContract wiki page
  • Any proposed contract additions filed as separate issues (do not merge them in this milestone)

Blocked by

M5.1.6 (cost ledger fields close out the arxiv-rag researcher)

Phase 5 — Second Researcher, milestone 2. Closes Phase 5. ## Goal Empirically validate that the v1 ResearchContract composes cleanly across researcher types now that we have two of them (web + arxiv-rag). ## Process 1. Pick 5–10 questions that should be answerable by *both* researchers (ML/AI topics where blog posts and arxiv papers both have good coverage) 2. Run each through `--researcher web` and `--researcher arxiv` independently 3. Compare side-by-side: - Do the contracts have the same shape? (yes — they should, by construction) - Do the answers complement or contradict? - Could a hypothetical PI orchestrator merge them mechanically? 4. Identify any contract changes needed — **backward-compatible additions only** (no field renames or type changes) 5. Document findings in the wiki ## Deliverable - A side-by-side comparison table for the test queries - A "lessons learned" section in the [ResearchContract](https://forgejo.labbity.unbiasedgeek.com/archeious/marchwarden/wiki/ResearchContract) wiki page - Any proposed contract additions filed as separate issues (do not merge them in this milestone) ## Blocked by M5.1.6 (cost ledger fields close out the arxiv-rag researcher)
archeious added this to the Phase 5: Second Researcher milestone 2026-04-08 23:25:14 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/step-durations
chore/makefile
feat/depth-presets
feat/per-step-logging
fix/budget-display-clarity
feat/costs-command
feat/cost-ledger
feat/structured-logging
fix/mcp-env-propagation
fix/budget-enforcement
fix/model-default-id
fix/synthesis-truncation
chore/docker-test-env
feat/cli-replay
feat/cli-ask
feat/mcp-server
feat/open-questions
feat/agent-loop
feat/trace-logger
feat/search-fetch-tools
feat/contract-models
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Phase 5: Second Researcher
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: archeious/marchwarden#50
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?