watchman/plan/plan.md

# Watchman — Product Plan and Roadmap

Last updated: 2025-09-17 10:54 local

This document lays out where Watchman is today, the features we will add next, and a pragmatic, incremental path to get there. It complements the existing planning notes in:
- plan/features.md — initial Free vs. Paid feature thinking
- plan/monetization.md — high-level monetization comparison

The focus here is on technical scope, sequence, and acceptance criteria.


## 1) Vision and Principles

Watchman provides multi-signal host monitoring and automated response to help detect and mitigate unauthorized access or suspicious changes on a system. Key principles:
- Defense-in-depth: multiple independent signals with clear provenance.
- Actionable by default: every detection can notify and/or trigger a safe, reversible response.
- Secure by default: hardened runtime, least-privilege, safe config parsing.
- Observability: clear logs, metrics, and health checks.
- Cross-platform where feasible; optimize for Linux first.


## 2) Current State (v0.2.x)

Already in the repository (per README and codebase):
- Monitors
  - USB activity: detect increases in connected USB devices; optional unmount/reboot/notify.
  - File system integrity: hash-based monitoring for specified directories (e.g., /etc, /bin, $PATH) using blake3.
  - Network: detect network issues/failures.
  - SSH “burn file”: remote path check that can trigger protective actions.
- Notification: Discord webhook alerts.
- Config: Settings.toml with intervals, toggles, directories, webhook URL.
- Runtime: Rust async, uses tokio/actix; prometheus crate is present.

Gaps
- Metrics exist in codebase structure but not fully surfaced/standardized.
- Limited notification channels (Discord only).
- Validation of config values and schema evolution not formalized.
- Tests are limited. No fuzzing or integration test harness.
- Cross-platform support unclear (Linux focused). 
- Service management, packaging, and upgrade story can be improved.


## 3) Near-Term Goals (security and reliability first)

- Hardening: minimize privileges, sanitize command execution, robust error handling.
- Deterministic behavior: clear state machine for each monitor, idempotent actions.
- Observability: consistent metrics and structured logs for every monitor and action.
- Safer configuration: validation, defaults, and deprecation path.


## 4) Feature Roadmap (Phased)

The roadmap uses small, shippable milestones. Each item lists key deliverables and acceptance criteria.

### Milestone A — 0.3.0: Observability & Config Hygiene
- Prometheus metrics v1
  - Counters for events by monitor (usb_events_total, fs_events_total, net_failures_total, burn_events_total)
  - Gauges for current connected USB count, last_success_timestamp per monitor
  - Histogram for monitor latency and action execution duration
  - /metrics endpoint gated by bind address (default: localhost) and optional auth token
  - Acceptance: metrics documented in README; basic dashboard example provided.
- Structured logging
  - Add consistent, machine-parseable logs (e.g., JSON via env-controlled format)
  - Include correlation IDs per tick/incident
  - Acceptance: toggle via config; example jq commands in README
- Config validation
  - Strongly-typed Settings with explicit defaults
  - Startup validation with actionable error messages
  - Acceptance: invalid config causes non-zero exit with guidance; sample Settings.toml updated

### Milestone B — 0.3.1: Notification Channels v2
- Add Slack and generic webhook support
- Pluggable notifier trait; Discord refactored to use it
- Basic rate limiting and de-dup window to avoid alert storms
- Acceptance: e2e tests that simulate events and assert notifier behavior

### Milestone C — 0.3.2: Filesystem Integrity v2
- Baseline management (init, refresh, compare)
- Exclude/include patterns; per-path policies
- Optional immutable baseline storage (append-only file) and checksum manifest
- Acceptance: CLI subcommands watchman fs init|refresh|diff and docs

### Milestone D — 0.3.3: USB Monitor v2
- Trusted device allowlist (vendor:product IDs) and policy actions (alert-only, unmount, power-cycle where supported)
- Cooldown and backoff to avoid flapping
- Acceptance: simulator/test harness to inject fake USB events

### Milestone E — 0.3.4: Network Monitor v2
- Multiple targets, success quorum, and jittered intervals
- Optional traceroute-on-failure (best effort)
- Acceptance: integration tests with a local dummy target

### Milestone F — 0.4.0: Response Actions Framework
- Action registry (unmount, reboot, kill-process, rotate-keys [stub], run-script)
- Pre- and post-conditions; dry-run mode; per-action timeouts
- Acceptance: action plans expressed in config; unit tests for each action

### Milestone G — 0.4.1: Health and Control API
- HTTP control plane (bind-local by default):
  - GET /healthz, /readyz, /metrics
  - POST /actions/{name} with dry-run and audit
- Acceptance: integration tests; example curl commands in README

### Milestone H — 0.4.2: Packaging & Service
- Systemd unit hardening (CapabilityBoundingSet, NoNewPrivileges, ProtectSystem, etc.)
- Deb/RPM packaging; Homebrew Tap formula for macOS CLI-only
- Acceptance: reproducible binaries, install docs, and service hardening checklist

### Milestone I — 0.5.x: Cross-Platform & Extensibility
- macOS: fs events via FSEvents; limited USB visibility; launchd plist
- Windows: filesystem via USN Journal (scoped); service integration; USB capabilities evaluated
- Plugin interface (WASM or dynamic dispatch) for third-party monitors
- Acceptance: CI builds for Linux/macOS; Windows nightly artifacts

Note: If monetization becomes relevant later, see plan/features.md and plan/monetization.md for an initial split. We will keep the open-source core useful and secure.


## 5) Architecture Notes

- Monitors
  - Each monitor is an independent component with a common trait: poll(), emit events, and register metrics.
  - Monitors should be stateless across ticks where possible; persisted state is explicit (e.g., fs baseline).
- Event Bus
  - Internally, use a channel-based event bus to decouple detection from actions/notifications.
  - Events carry severity, source, timestamp, and correlation ID.
- Actions
  - Declarative action specifications (conditions + operations). Support dry-run.
- Notifiers
  - Implement a notifier trait with backoff, jitter, and rate-limiting wrappers.
- Configuration
  - Strongly typed, documented, validated; feature flags guarded with clear defaults.


## 6) Configuration Evolution (draft)

Add or refine keys in config/Settings.toml:
- [general]
  - tick_delay_seconds, fs_tick_delay_seconds
  - log_format = "text"|"json"
  - metrics_bind_addr = "127.0.0.1:9898"
  - metrics_auth_token = ""  # optional
- [notifications]
  - discord_webhook_url = "..." (existing)
  - slack_webhook_url = "..."
  - generic_webhook_url = "..."
  - notify_dedupe_window_secs = 30
- [fs]
  - enabled = true
  - dirs = ["/etc", "/bin", "$PATH"]
  - exclude = ["/etc/ssl/**"]
  - baseline_path = "/var/lib/watchman/fs-baseline.json"
  - hash = "blake3"
- [usb]
  - enabled = true
  - allowlist = ["abcd:1234", "1d6b:0002"]
  - action_on_unauthorized = "alert|unmount|reboot|none"
  - cooldown_secs = 10
- [net]
  - enabled = false
  - targets = ["1.1.1.1", "8.8.8.8"]
  - quorum = 1
  - interval_jitter_pct = 20
- [burn]
  - enabled = false
  - ssh_host = "hostname"
  - ssh_user = "root"
  - ssh_key = "/home/user/.ssh/id_rsa"
  - path = "/root/.config/burn"
  - check_interval_secs = 30


## 7) Security & Hardening Checklist

- Run as dedicated user; least filesystem permissions.
- Systemd hardening options enabled by default.
- Validate and sanitize all external inputs (paths, webhook URLs, command args).
- Cryptographic hashing via blake3; check for algorithm agility hooks.
- Protect secrets in memory where possible; avoid logging sensitive values.
- Optional signed baseline and config (future).


## 8) Observability & Diagnostics

- Prometheus metrics as in Milestone A.
- Structured logs with correlation IDs.
- Debug bundle command: watchman diag bundle → collects versions, last logs, metrics snapshot (no secrets).


## 9) CLI and UX

- watchman run — start monitors with current config.
- watchman fs init|refresh|diff — manage filesystem baseline.
- watchman validate — validate config and report errors.
- watchman install-service webhook=<url> — keep for quick start; document flags.


## 10) Testing Strategy

- Unit tests per monitor and notifier.
- Integration tests spawning the binary with temp configs.
- Property-based tests for config parsing/validation.
- Fuzz critical parsers (USB sysfs parsing, config loader).
- Golden tests for logs/metrics outputs.
- CI: run clippy, fmt, tests; produce artifacts for Linux (and later macOS/Windows).


## 11) Performance Targets

- Idle memory: < 150 MB typical on Linux with all monitors enabled.
- CPU: < 2% average on a modern 2-core VM at default intervals.
- Baseline diff for 100k files: < 30s on SSD; incremental checks amortized.


## 12) Risks and Mitigations

- False positives driving destructive actions → default to alert-only; require explicit opt-in for destructive actions; dry-run support.
- Platform differences (USB, FS APIs) → feature gating; per-OS implementations.
- Metric/telemetry exposure → bind-local by default; optional token; document firewalling.


## 13) Release Cadence and Versioning

- Minor iterations every 2–4 weeks; patch releases as needed.
- Semantic versioning: 0.x minor bumps may include breaking changes, documented in CHANGELOG.


## 14) Immediate Next Steps (for maintainers)

1. Define metric names and add minimal /metrics endpoint (A).
2. Add config validation with friendly errors (A).
3. Introduce notifier trait and migrate Discord (B).
4. Draft fs baseline CLI skeleton (C).

Track progress via issues and link them back to this plan.