Bluetooth-based management interface #71

Merged
albert merged 39 commits from u/albert/65/ble-management into main 2026-07-04 14:34:41 +00:00
Owner

Bluetooth LE management interface (#65)

Closes #65 — "Management interface when network is down".

Why

The existing HTTP management API only works when the phone can actually reach the box: it needs a live LAN, plus either mDNS autodiscovery or a static IP the admin remembers. In practice DHCP, AP isolation, and captive portals break that often enough that "I need to change a setting and can't reach the device" is a recurring problem.

This PR adds Bluetooth LE as the primary admin transport. BLE removes the IP-discovery / static-IP friction entirely and works regardless of network state. HTTP stays available but secondary. Design doc: docs/ai/history/2026-06-20 002 ble-management.md.

Architecture

The load-bearing refactor is extracting a transport-agnostic service so HTTP and BLE share one business-logic implementation:

  • shepherd-management (new) — defines the ManagementService trait covering every admin operation (list entries, launch/stop/extend sessions, overrides, usage, volume, brightness, reload, windows, event subscription). DefaultManagementService composes the existing CoreEngine/Store/HostAdapter/controllers the daemon already wires up. The HTTP handlers in shepherd-http are reduced to wire-format translation — sessions.rs alone drops ~250 lines.
  • shepherd-ble (new) — BleServer, the BLE counterpart to HttpServer, serving the same trait over a custom GATT service. Modules: protocol (UUIDs + JSON-RPC envelope), framing (length-prefix + chunked reassembly), outbox (read-poll FIFO), rpc (method → trait dispatch), admin (TOML admin record + reset sentinel), claim (Unclaimed→Claimed state machine + per-request auth gate), agent (BlueZ pairing agent), server (advertising, GATT registration, per-client loop).
  • shepherd-pairing-display (new) — wlr-layer-shell Sway overlay sidecar that shows the 6-digit pairing code full-screen on the TV.
  • companion-android (new) — the Android companion app (Jetpack Compose + Kable), the primary admin client. Pairs, claims, and drives the full RPC catalog over the bonded link.

Schema-driven dispatch (follow-on)

The initial split still left three per-transport dispatchers duplicating the trait signatures — BLE's rpc.rs (~390 lines of match-and-parse boilerplate), IPC's handle_command in shepherdd/main.rs (~900 lines of tagged-Command-enum handling), and HTTP's ten REST handler files. Adding a method touched seven places: the trait, both server dispatchers, the HTTP handler + route, the IPC Command enum, the Kotlin client, and the TypeScript client.

A #[management_rpc] proc-macro (crates/shepherd-management-macros) reads the trait and emits both a dispatch_json(svc, method, params) -> Result<Value, RpcDispatchError> function next to it and a machine-readable RPC_SCHEMA_JSON const. Per-method attributes (#[rpc(default(...))], #[rpc(wrap_result = "field")]) cover the small handful of quirks the wire format needs (defaulting at/date, wrapping bool/Option/usize in a named field for forward-compat). A companion rpc-codegen binary consumes the schema and writes three checked-in files: docs/rpc-schema.json, companion-android/.../ble/RpcMethods.kt (typed method-name constants + a wrapField() lookup), and shepherd-webui/src/api/rpc-methods.generated.ts (union type + wrap-field map). A drift-check test in crates/shepherd-management/tests/rpc_codegen_drift.rs fails CI if the checked-in outputs no longer match the current trait.

Every transport now sits on the generated dispatcher:

  • BLEcrates/shepherd-ble/src/rpc.rs shrinks 388 → 116 lines, just an adapter mapping RpcDispatchError onto the wire's ErrorCode.
  • IPC — the 900-line handle_command match in shepherdd/src/main.rs is gone. Wire format changes from tagged Command / ResponsePayload enums to JSON-RPC ({method, params} request, Value-or-error response) on the same NDJSON socket. Every consumer (launcher-ui, hud, e2e) now speaks typed helpers on IpcClient::{service_state, launch, get_volume, ...}. ~830 net lines deleted.
  • HTTP — all ten REST handler modules (entries.rs, sessions.rs, volume.rs, overrides.rs, ...) removed. The router is two routes: POST /api/v1/rpc (dispatch_json pass-through with ManagementError → HTTP status mapping) and GET /api/v1/events (SSE stays — push shape doesn't fit RPC). shepherd-webui/src/api/client.ts collapses ~180 lines of hand-written REST wrappers into a typed call<T>(method, params) plus per-method one-liners.

Adding a new RPC method is now a single trait line. The macro re-emits dispatch_json, cargo run -p shepherd-management --bin rpc-codegen regenerates the schema + Kotlin + TypeScript mirrors, and every transport picks it up automatically — the CI drift test forces the regeneration into the same commit as the trait change. The IPC-migration commit alone extended the trait with volume_up/volume_down/toggle_mute/brightness_up/brightness_down/ping and had all three transports serving them without touching per-transport code.

Security model (two layers)

  • Link layer — BLE pairing + bonding gives confidentiality and peer identity. Pairing uses Numeric Comparison (LESC) or Passkey Entry (legacy 4.0/4.1 controllers) — both MITM-protected; the agent handles both paths and the overlay copy is controller-aware. Just Works is rejected (MITM-vulnerable for a device controlling system policy).
  • App layerTOFU, single admin (v1): the first phone to pair + send the claim RPC becomes the sole admin; later claims get already_claimed. The claim mints a bearer token that is also valid for the HTTP API — clearing the admin record (or factory reset) invalidates both transports in one step (AdminAuthority trait unifies the two).

Recovery

A factory-reset sentinel file (<data_dir>/.factory-reset-ble by default): touch it and restart shepherdd to wipe the admin record and bond. Also surfaced as shepherd bluetooth clear to wipe a logged-out user's BLE state.

Transport: read-poll, not notify

The most significant evolution on the branch. BLE notify proved unfixable for bonded reconnects on this Android/BlueZ stack pair — CCCD writes are cached at the bond level, so the server's notify task never re-armed on a reopened app and responses went nowhere ("activity list blank on every subsequent open"). Full investigation: docs/ai/history/2026-06-28 001 ble-read-poll-replaces-notify.md.

The fix drops notify entirely. Response and Events become read characteristics backed by a server-side Outbox (ordered, framed FIFO that never tears a frame across reads); the companion polls with adaptive backoff (25 ms after data → 300 ms idle, woken instantly on RPC dispatch via a conflated channel, sub-50 ms latency). Two live-hardware bugs caught and fixed along the way:

  • Session boundaries are now keyed on id==1 (each connection restarts its RPC counter) instead of a 5 s write-gap heuristic that wiped responses mid-delivery when the 15 s client timeout fired.
  • 512-byte read cap — Android silently truncates readCharacteristic to GATT_MAX_ATTRIBUTE_VALUE, dropping bytes 513–516 at MTU 517 and desyncing the reassembler. The outbox now hands out ≤512 bytes per read.

ClaimMachine::authorize was also relaxed for v1: the encrypted link is the security boundary and TOFU guarantees a single bond, so reconnects no longer fail on BlueZ identity-address drift.

Config

New optional [service.ble_management] block (enabled, device_name, admin_record_path, reset_sentinel_path) — see config.example.toml. Disabled by default. Requires a working BlueZ adapter, the bluez package, and membership in the bluetooth group (startup now requires and diagnoses this).

Android companion app

com.armeafamily.shepherd.companion — Compose/Material 3, single activity, one ShepherdViewModel. Kable for GATT (coroutine-native, Apache-2.0); bonding is OS-driven. Multi-device selector on Home, hand-rolled usage bar chart, EncryptedSharedPreferences for the token at rest. Constraints honoured: no telemetry/phone-home, no background BLE (link lives only in foreground), GPL-3.0-compatible deps only. Spec: docs/ai/history/2026-06-21 001 ble-companion-android-spec.md. Build via ./scripts/shepherd deps install android then ./gradlew :app:assembleDebug in companion-android/.

Testing & CI

  • Rust unit tests (run in the existing Test job, cargo test --all-targets): shepherd-ble covers framing, outbox, claim-machine, and protocol in isolation, plus the server.rs write-path invariants the read-poll rewrite depends on — the id==1 session-boundary outbox wipe and the FrameReader reset on peer change / framing error (the logic behind the field bugs in the read-poll doc). The MockSvc test double is shared via a crate-level testsupport module. shepherd-http API tests drive DefaultManagementService/AdminAuthority end-to-end, so the shared service the BLE transport calls into is integration-covered.
  • Android unit tests now run in CI via a new android-companion job: FramingTest (codec) + WireTest (decode of every sampled spec payload), 17 tests guarding the phone side of the wire protocol against drifting from shepherd-ble. Hardware-independent JVM tests only — no emulator.
  • CI infrastructure: a dedicated Android CI image (.ci/Dockerfile.android, built by an image-android job) layers the JDK + ~1 GB Android SDK onto the base image so the Gradle job gets a ready SDK; the 8 Rust-only jobs keep the lean base image. Storage is split by volatility — the static SDK is baked into the image, while the churnier Gradle/Maven dep cache rides actions/cache.
  • On-device: pair → claim → exercise each screen, validated end-to-end on a Pixel 10a against the kiosk hardware (Marvell HCI 4.0 controller). The live pair/claim path stays a manual smoke test; emulating it (vhci) was scoped out of CI.

Deferred to v2

Multi-admin / approve-new-device (the per-bond record schema already accommodates it), OOB-QR pairing as a display-down escape hatch, and tracking the IRK explicitly in the admin record rather than relying on the single-bond assumption.

## Bluetooth LE management interface (#65) Closes #65 — "Management interface when network is down". ### Why The existing HTTP management API only works when the phone can actually reach the box: it needs a live LAN, plus either mDNS autodiscovery or a static IP the admin remembers. In practice DHCP, AP isolation, and captive portals break that often enough that "I need to change a setting and can't reach the device" is a recurring problem. This PR adds **Bluetooth LE as the primary admin transport**. BLE removes the IP-discovery / static-IP friction entirely and works regardless of network state. HTTP stays available but secondary. Design doc: [`docs/ai/history/2026-06-20 002 ble-management.md`](docs/ai/history/2026-06-20%20002%20ble-management.md). ### Architecture The load-bearing refactor is extracting a transport-agnostic service so HTTP and BLE share one business-logic implementation: - **`shepherd-management`** (new) — defines the `ManagementService` trait covering every admin operation (list entries, launch/stop/extend sessions, overrides, usage, volume, brightness, reload, windows, event subscription). `DefaultManagementService` composes the existing `CoreEngine`/`Store`/`HostAdapter`/controllers the daemon already wires up. The HTTP handlers in `shepherd-http` are reduced to wire-format translation — `sessions.rs` alone drops ~250 lines. - **`shepherd-ble`** (new) — `BleServer`, the BLE counterpart to `HttpServer`, serving the same trait over a custom GATT service. Modules: `protocol` (UUIDs + JSON-RPC envelope), `framing` (length-prefix + chunked reassembly), `outbox` (read-poll FIFO), `rpc` (method → trait dispatch), `admin` (TOML admin record + reset sentinel), `claim` (Unclaimed→Claimed state machine + per-request auth gate), `agent` (BlueZ pairing agent), `server` (advertising, GATT registration, per-client loop). - **`shepherd-pairing-display`** (new) — `wlr-layer-shell` Sway overlay sidecar that shows the 6-digit pairing code full-screen on the TV. - **`companion-android`** (new) — the Android companion app (Jetpack Compose + Kable), the primary admin client. Pairs, claims, and drives the full RPC catalog over the bonded link. ### Schema-driven dispatch (follow-on) The initial split still left three per-transport dispatchers duplicating the trait signatures — BLE's `rpc.rs` (~390 lines of match-and-parse boilerplate), IPC's `handle_command` in `shepherdd/main.rs` (~900 lines of tagged-`Command`-enum handling), and HTTP's ten REST handler files. Adding a method touched seven places: the trait, both server dispatchers, the HTTP handler + route, the IPC `Command` enum, the Kotlin client, and the TypeScript client. A **`#[management_rpc]` proc-macro** (`crates/shepherd-management-macros`) reads the trait and emits both a `dispatch_json(svc, method, params) -> Result<Value, RpcDispatchError>` function next to it and a machine-readable `RPC_SCHEMA_JSON` const. Per-method attributes (`#[rpc(default(...))]`, `#[rpc(wrap_result = "field")]`) cover the small handful of quirks the wire format needs (defaulting `at`/`date`, wrapping `bool`/`Option`/`usize` in a named field for forward-compat). A companion `rpc-codegen` binary consumes the schema and writes three checked-in files: `docs/rpc-schema.json`, `companion-android/.../ble/RpcMethods.kt` (typed method-name constants + a `wrapField()` lookup), and `shepherd-webui/src/api/rpc-methods.generated.ts` (union type + wrap-field map). A drift-check test in `crates/shepherd-management/tests/rpc_codegen_drift.rs` fails CI if the checked-in outputs no longer match the current trait. Every transport now sits on the generated dispatcher: - **BLE** — `crates/shepherd-ble/src/rpc.rs` shrinks 388 → 116 lines, just an adapter mapping `RpcDispatchError` onto the wire's `ErrorCode`. - **IPC** — the 900-line `handle_command` match in `shepherdd/src/main.rs` is gone. Wire format changes from tagged `Command` / `ResponsePayload` enums to JSON-RPC (`{method, params}` request, `Value`-or-error response) on the same NDJSON socket. Every consumer (`launcher-ui`, `hud`, `e2e`) now speaks typed helpers on `IpcClient::{service_state, launch, get_volume, ...}`. ~830 net lines deleted. - **HTTP** — all ten REST handler modules (`entries.rs`, `sessions.rs`, `volume.rs`, `overrides.rs`, ...) removed. The router is two routes: `POST /api/v1/rpc` (`dispatch_json` pass-through with `ManagementError` → HTTP status mapping) and `GET /api/v1/events` (SSE stays — push shape doesn't fit RPC). `shepherd-webui/src/api/client.ts` collapses ~180 lines of hand-written REST wrappers into a typed `call<T>(method, params)` plus per-method one-liners. Adding a new RPC method is now a single trait line. The macro re-emits `dispatch_json`, `cargo run -p shepherd-management --bin rpc-codegen` regenerates the schema + Kotlin + TypeScript mirrors, and every transport picks it up automatically — the CI drift test forces the regeneration into the same commit as the trait change. The IPC-migration commit alone extended the trait with `volume_up`/`volume_down`/`toggle_mute`/`brightness_up`/`brightness_down`/`ping` and had all three transports serving them without touching per-transport code. ### Security model (two layers) - **Link layer** — BLE pairing + bonding gives confidentiality and peer identity. Pairing uses **Numeric Comparison** (LESC) or **Passkey Entry** (legacy 4.0/4.1 controllers) — both MITM-protected; the agent handles both paths and the overlay copy is controller-aware. Just Works is rejected (MITM-vulnerable for a device controlling system policy). - **App layer** — **TOFU, single admin (v1)**: the first phone to pair + send the `claim` RPC becomes the sole admin; later claims get `already_claimed`. The claim mints a bearer token that is **also valid for the HTTP API** — clearing the admin record (or factory reset) invalidates both transports in one step (`AdminAuthority` trait unifies the two). ### Recovery A factory-reset sentinel file (`<data_dir>/.factory-reset-ble` by default): `touch` it and restart `shepherdd` to wipe the admin record and bond. Also surfaced as `shepherd bluetooth clear` to wipe a logged-out user's BLE state. ### Transport: read-poll, not notify The most significant evolution on the branch. BLE notify proved unfixable for bonded reconnects on this Android/BlueZ stack pair — CCCD writes are cached at the bond level, so the server's notify task never re-armed on a reopened app and responses went nowhere ("activity list blank on every subsequent open"). Full investigation: [`docs/ai/history/2026-06-28 001 ble-read-poll-replaces-notify.md`](docs/ai/history/2026-06-28%20001%20ble-read-poll-replaces-notify.md). The fix drops notify entirely. Response and Events become **read** characteristics backed by a server-side `Outbox` (ordered, framed FIFO that never tears a frame across reads); the companion polls with adaptive backoff (25 ms after data → 300 ms idle, woken instantly on RPC dispatch via a conflated channel, sub-50 ms latency). Two live-hardware bugs caught and fixed along the way: - **Session boundaries** are now keyed on `id==1` (each connection restarts its RPC counter) instead of a 5 s write-gap heuristic that wiped responses mid-delivery when the 15 s client timeout fired. - **512-byte read cap** — Android silently truncates `readCharacteristic` to `GATT_MAX_ATTRIBUTE_VALUE`, dropping bytes 513–516 at MTU 517 and desyncing the reassembler. The outbox now hands out ≤512 bytes per read. `ClaimMachine::authorize` was also relaxed for v1: the encrypted link is the security boundary and TOFU guarantees a single bond, so reconnects no longer fail on BlueZ identity-address drift. ### Config New optional `[service.ble_management]` block (`enabled`, `device_name`, `admin_record_path`, `reset_sentinel_path`) — see `config.example.toml`. Disabled by default. Requires a working BlueZ adapter, the `bluez` package, and membership in the `bluetooth` group (startup now requires and diagnoses this). ### Android companion app `com.armeafamily.shepherd.companion` — Compose/Material 3, single activity, one `ShepherdViewModel`. Kable for GATT (coroutine-native, Apache-2.0); bonding is OS-driven. Multi-device selector on Home, hand-rolled usage bar chart, `EncryptedSharedPreferences` for the token at rest. Constraints honoured: **no telemetry/phone-home, no background BLE** (link lives only in foreground), GPL-3.0-compatible deps only. Spec: [`docs/ai/history/2026-06-21 001 ble-companion-android-spec.md`](docs/ai/history/2026-06-21%20001%20ble-companion-android-spec.md). Build via `./scripts/shepherd deps install android` then `./gradlew :app:assembleDebug` in `companion-android/`. ### Testing & CI - Rust unit tests (run in the existing `Test` job, `cargo test --all-targets`): `shepherd-ble` covers framing, outbox, claim-machine, and protocol in isolation, plus the `server.rs` write-path invariants the read-poll rewrite depends on — the `id==1` session-boundary outbox wipe and the `FrameReader` reset on peer change / framing error (the logic behind the field bugs in the read-poll doc). The `MockSvc` test double is shared via a crate-level `testsupport` module. `shepherd-http` API tests drive `DefaultManagementService`/`AdminAuthority` end-to-end, so the shared service the BLE transport calls into is integration-covered. - Android unit tests now run in CI via a new `android-companion` job: `FramingTest` (codec) + `WireTest` (decode of every sampled spec payload), 17 tests guarding the phone side of the wire protocol against drifting from `shepherd-ble`. Hardware-independent JVM tests only — no emulator. - CI infrastructure: a dedicated **Android CI image** (`.ci/Dockerfile.android`, built by an `image-android` job) layers the JDK + ~1 GB Android SDK onto the base image so the Gradle job gets a ready SDK; the 8 Rust-only jobs keep the lean base image. Storage is split by volatility — the static SDK is baked into the image, while the churnier Gradle/Maven dep cache rides `actions/cache`. - On-device: pair → claim → exercise each screen, validated end-to-end on a Pixel 10a against the kiosk hardware (Marvell HCI 4.0 controller). The live pair/claim path stays a manual smoke test; emulating it (vhci) was scoped out of CI. ### Deferred to v2 Multi-admin / approve-new-device (the per-bond record schema already accommodates it), OOB-QR pairing as a display-down escape hatch, and tracking the IRK explicitly in the admin record rather than relying on the single-bond assumption.
BLE-based primary admin transport: Numeric Comparison pairing + TOFU
single-admin claim + unified HTTP/BLE bearer-token identity + filesystem
reset sentinel. Records rejected alternatives (hotspot fallback, Just Works,
OOB-QR, pre-shared codes) with rationale, and the GATT / state-machine /
work-breakdown specifics for the implementation pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prerequisite for the planned BLE management transport. New crate
shepherd-management owns DefaultManagementService, which composes the
daemon's existing engine, store, host, volume, brightness, hidpi, and
broadcast collaborators. shepherd-http handlers become thin adapters that
translate Axum extractors into trait calls and ManagementError into
ApiError (or, for the historical launch/stop/config wire shapes, into
their existing custom envelopes). HTTP wire format is unchanged; all 43
shepherd-http integration tests pass without modification beyond the
fixture rebuild.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The BLE management transport that the design doc in
docs/ai/history/2026-06-20 002 ble-management.md specifies. Implements
the full surface but is not yet wired into shepherdd (that's the next
commit). The server's GATT integration speaks bluer + BlueZ; everything
else (protocol envelopes, framing, admin record TOML, claim machine,
RPC dispatch) is unit-tested in isolation.

Modules:

- protocol: GATT service/characteristic UUIDs, RpcRequest/RpcResponse
  envelopes, ErrorCode mapping from ManagementError.
- framing: u16-LE length-prefix reader and MTU chunk encoder for ATT
  writes and notifies that exceed one packet.
- admin: AdminRecord (identity_address, http_token, role) persisted as
  atomic TOML, plus the factory-reset sentinel check.
- claim: Unclaimed/Claimed state machine with TOFU single-admin
  semantics and the per-RPC authorize() gate.
- rpc: name-keyed dispatch from RpcRequest into the
  ManagementService trait.
- agent: bluer pairing agent configured for Numeric Comparison (IO
  capability DisplayYesNo); calls a PairingDisplay trait so the
  daemon's Sway overlay plugs in without bringing Wayland into this
  crate.
- server: BleServer that brings up the adapter, registers the agent
  and GATT application, advertises the management service, and
  routes incoming writes through the claim machine before the RPC
  dispatcher.

System dependencies: libdbus-1-dev (build) and bluez (runtime) added
to scripts/deps/{build,run}.pkgs. CI hashes those files so the image
will rebuild automatically.

Tests: 34 unit tests in shepherd-ble (protocol, framing, admin, claim,
rpc, agent). cargo check / test / clippy / fmt all clean across the
workspace.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the [service.ble_management] config section, constructs a single
DefaultManagementService that both HttpServer and BleServer share when
either transport is enabled, and spawns BleServer on the same shutdown
watch as HttpServer with a matching drain timeout.

PairingDisplay is plumbed through but uses NoopPairingDisplay for now;
the Sway overlay sidecar is a follow-up. Advertising and the GATT
application still come up — only the on-device visual is missing.

config.example.toml gains an example ble_management block; default
admin_record_path is &lt;data_dir&gt;/admin.toml and the reset sentinel is
&lt;data_dir&gt;/.factory-reset-ble. `./scripts/shepherd config validate
config.example.toml` is clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The BLE claim flow mints an http_token on the AdminRecord; HTTP's
auth middleware now accepts that token in addition to the static
config-time auth_token. The two transports share one admin identity,
so revoking the BLE bond (or factory-reset sentinel) also revokes
HTTP access in a single step.

- shepherd-management gains AdminAuthority: trait current_http_token()
  returns the current admin's token (or None when unclaimed).
- ClaimMachine implements AdminAuthority; its state moved from
  tokio::sync::RwLock to std::sync::RwLock so the sync middleware can
  read without blocking on an async lock. Lock-held work is brief and
  sync (in-memory update + an atomic file write on claim/reset).
- HTTP gains AuthSources (static_token + Option<Arc<dyn
  AdminAuthority>>) and a new HttpServer::with_admin_authority builder.
  Middleware passes a request through if either source matches; if
  neither is configured it falls back to the legacy open-API
  behaviour.
- shepherdd now constructs BleServer before HttpServer when both are
  configured, then hands BleServer::claim_machine() to HttpServer as
  the admin authority.
- Two new integration tests in shepherd-http cover the
  admin-token-only and static+admin combinations.

45 shepherd-http tests pass; full workspace cargo test/clippy/fmt
clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New shepherd-pairing-display binary renders the 6-digit BLE Numeric
Comparison passkey as a full-screen wlr-layer-shell overlay during
pairing, so the user has something to compare against the number
their phone displays. Without this the pairing was technically MITM-
resistant only if the user trusted whatever the phone showed.

- crates/shepherd-pairing-display: gtk4 + gtk4-layer-shell binary
  patterned on shepherd-hud. Single overlay window, dark background,
  220px monospace passkey, short instructions. Takes --passkey and
  --device CLI args; lives until killed.
- crates/shepherdd/src/pairing_display.rs: SwayPairingDisplay
  implements shepherd_ble::PairingDisplay by spawning / killing the
  binary as a child process. Show replaces any previous overlay
  first; Drop also tears it down.
- main.rs swaps NoopPairingDisplay for SwayPairingDisplay.
- scripts/lib/build.sh adds the new binary to the install list so
  ./scripts/shepherd install bins picks it up.

cargo workspace check / test / clippy / fmt all clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-contained spec for an agent in a different environment to pick up
and build the Android companion. Covers the full BLE protocol (UUIDs,
length-prefix framing, JSON-RPC envelope, error codes), the Numeric
Comparison pairing flow as it surfaces on Android 12+, every RPC
method with sample request/response JSON, the event-stream payload
shapes, opinionated app architecture (Kotlin + Compose + Nordic
Android-BLE-Library), persistence + multi-device support, the
non-negotiable parent-project constraints (no telemetry / no DRM /
GPL-compatible), and a verification approach against a real device.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the Android companion app from the spec at
docs/ai/history/2026-06-21 001 ble-companion-android-spec.md, living in
companion-android/. Kotlin + Jetpack Compose (Material 3), single
activity, min SDK 31 / target 35.

BLE via Kable (per instruction, not the spec's Nordic suggestion):
Kable drives GATT, while bonding/Numeric-Comparison goes through the
raw Android createBond + bond-state broadcast. The wire layer mirrors
crates/shepherd-ble and shepherd-api exactly — u16-LE length-prefix
framing, the JSON-RPC envelope, the full error-code enum, every RPC
method, and the event stream (snake_case via JsonNamingStrategy, with a
custom serializer for the externally-tagged LaunchOutcome).

UI covers pairing (scan -> compare digits -> claim), home (device chips
+ entries), entry detail (launch / live countdown / extend / stop /
today's override editor / 7-day usage chart), device controls
(volume / brightness / reload / logout), and settings (factory reset,
forget-all, nickname). Constraints honoured: no telemetry, no
background BLE (link bound to the foreground lifecycle), encrypted
token storage with backup/transfer disabled, GPL-compatible deps only.

Tooling: new `deps install android` set installs JDK 21 + the Android
SDK into /opt/android-sdk (scripts/lib/deps.sh, scripts/deps/
android.pkgs). The license step toggles pipefail off so `yes |
sdkmanager --licenses` (SIGPIPE -> 141) doesn't abort the script.

Tests: FramingTest (codec round-trips, split frames, oversize guard)
and WireTest (decodes every sample payload from the spec). All pass;
:app:assembleDebug produces a sideload-friendly APK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsKhrrk2ZeD3dcCR6dS1DA
shepherd-kiosk was missing the `bluetooth` group on a live install, so
BlueZ refused our agent registration and set_pairable calls via the
polkit rules that ship with bluez. The daemon then logged a generic
adapter error and never advertised, and the phone — finding the
controller only over classic Bluetooth — got bluetoothd's default
PIN-entry fallback instead of the Numeric Comparison flow.

Two changes:

- scripts/lib/install.sh: SHEPHERD_REQUIRED_GROUPS gains `bluetooth`
  so `./scripts/shepherd install groups --user <kiosk>` (and the full
  install) wires it up automatically. Existing installs need to run
  install groups again and re-login.

- crates/shepherd-ble/src/server.rs: wrap each BlueZ startup call
  (session, default adapter, set_powered, set_pairable, register
  agent) with a contextual error message that names the likely root
  cause. The bluetooth-group hint is repeated on the two calls most
  likely to fail that way; the agent-registration message also
  explains the user-visible consequence (PIN fallback on the phone,
  no overlay on the TV) so the journal points straight at the fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dev box's controller is HCI 4.0 (Marvell), which predates LE
Secure Connections (added in BT 4.2). SMP on this hardware therefore
falls back to LE Legacy Pairing, and with our DisplayYesNo IO
capability plus the phone's KeyboardDisplay + MITM-required, the
selected method is Passkey Entry, responder-displays — we display the
6-digit passkey, the phone prompts the user to type it.

Our agent only had `request_confirmation` (Numeric Comparison),
nothing for Passkey Entry display. BlueZ called display_passkey, our
agent had no handler so it rejected, pairing stalled, and the phone's
"Enter PIN to pair with leibniz" dialog waited indefinitely for input
that we never displayed. btmon trace of an attempt:

  SMP: Pairing Request   (phone)  IO=KeyboardDisplay   auth=Legacy MITM
  SMP: Pairing Response  (us)     IO=DisplayYesNo      auth=Legacy MITM
  MGMT: Passkey Notify   passkey=0x000a693a            ← display this!

Fix: add `display_passkey` to the agent that drives the same
PairingDisplay show/hide as `request_confirmation`. The on-device UX
is unchanged (a big number on the TV); only the phone-side prompt
differs ("type this" vs "match this"). Security property is the same
MITM-protected exchange.

Adding `display_passkey` does NOT promote our IO capability away from
DisplayYesNo — bluer's capability table groups display+yes_no
callbacks under DisplayYesNo. So Numeric Comparison still gets
selected on 4.2+ controllers, and Passkey Entry on 4.0/4.1.

PASSKEY_DISPLAY_HOLD bumped from 30s to 60s — Passkey Entry needs
the user to actually type the number on the phone, which takes
longer than confirming a match.

Design doc updated to reflect the controller-dependent method.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Numeric Comparison and Passkey Entry ask the user to do different
things (compare vs type). The overlay shipped Numeric Comparison copy
unconditionally, which was wrong on the leibniz dev box and on any
BT 4.0/4.1 controller that lands in Legacy Passkey Entry.

- shepherd-ble: PairingDisplay::show grows a PairingMethod parameter
  (Compare | Enter). request_confirmation passes Compare;
  display_passkey passes Enter.
- shepherd-pairing-display: required --method <compare|enter> CLI flag
  switches the instruction label.
- shepherdd: SwayPairingDisplay maps the enum to the CLI arg and logs
  which method spawned each overlay.

Copy reads:
  compare → "If this number matches the one on your phone, tap PAIR
             (or MATCH) on the phone. If it does not match, tap
             CANCEL (or DON'T MATCH) and tell whoever is in charge…"
  enter   → "Type this number into the prompt on your phone to
             complete pairing. If you did not initiate this pairing,
             tap CANCEL on the phone and tell whoever is in charge…"

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shepherd-pairing-display silently exited 0 before its activate
handler fired, so no overlay ever showed even though shepherdd
logged a successful spawn. The cause was GLib's option parser:
gtk4::Application::run() defaults to processing std::env::args(),
sees --passkey / --device / --method (which we already consumed
with clap), and rejects them with the standard "Unknown option
--passkey" GOptionContext message. Exit code 0 because GLib treats
unknown options as a request to exit cleanly.

Fix: call `app.run_with_args(&empty)` so GTK never sees our argv.
If we later need to pass GTK debug flags through, they'll need to
be split off from argv before clap runs — none today.

Also drop the Stdio::null() on shepherdd's child spawn so the next
sidecar failure surfaces in shepherdd's journal instead of being
silently swallowed; the previous suppression cost us this debug
cycle.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Request characteristic was declared with both
encrypt_authenticated_write and secure_write. `secure_write: true`
means "LE Secure Connections required", which a BT 4.0 controller
(the leibniz dev box) doesn't support. After Legacy Passkey Entry
bonding the link is MITM-protected but not LESC, so BlueZ silently
rejected every Request write — the companion's claim() RPC vanished
and the app sat on its spinner forever.

Drop secure_write; keep encrypt_authenticated_write, which is
satisfied by Legacy MITM pairing and gives us the property we
actually need (encrypted + authenticated link). LESC-capable
controllers still get LESC; we just no longer demand it.

Also promote three RPC-layer log lines to INFO so the next time a
frame disappears, the journal points at the chain step:

  BLE request chunk arrived              (debug; one per ATT chunk)
  BLE RPC received                       (info; per logical frame)
  BLE RPC response queued                (info)
  BLE Response notify subscribed         (info; per subscribe)
  BLE Response notify sending payload    (info; per response)

The previous version logged none of these at INFO, so a missing
response looked identical to a successful one in the journal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The overlay was sitting on screen for 60 seconds after every pairing,
even when bonding completed in 5 seconds — PASSKEY_DISPLAY_HOLD was
the only signal for hide().

Now the hide watcher races a Device1.Paired=true property change
from BlueZ against the 60s fallback. Successful pairings close the
overlay immediately; cancel/failure cases still time out so it never
sits indefinitely.

- build_agent now takes a bluer::Session so it can resolve the peer
  device on demand and subscribe to its property events. The session
  is cheap to clone (internal Arc), captured per-callback.
- spawn_hide_when_paired_or_timeout wraps wait_for_paired in a
  tokio::time::timeout; either branch terminates the spawned task
  through the same hide-lock + display.hide() path.
- wait_for_paired checks device.is_paired() once before subscribing
  to events so the tiny race between read and subscribe doesn't
  miss an already-set bond.
- Removed the two handler unit tests that previously round-tripped
  through display.show with a fake address — they'd now require a
  live BlueZ session to construct the handler call. Replaced with
  a CountingDisplay round-trip test that's strictly about the
  display trait, and made the agent-capability test gracefully skip
  when no D-Bus is available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After a kiosk user logs out (or is being decommissioned) shepherdd is
gone, but BlueZ still holds the bond and the user's admin.toml still
points the next shepherdd at the same admin. The factory-reset
sentinel handles this on the next startup, but operators sometimes
want to clear the state immediately — e.g. before handing the device
to a new user.

`sudo shepherd bluetooth clear --user USER` does the full cleanup in
one shot:

  1. Refuses if the user has an active login session (override with
     --force; documented as racing with the live shepherdd).
  2. Reads ~USER/.local/share/shepherdd/admin.toml to find the bonded
     peer's identity address.
  3. `bluetoothctl disconnect <addr>` then `bluetoothctl remove <addr>`
     for each recorded peer — both no-ops if the state is already
     clean.
  4. Deletes admin.toml so the next shepherdd boot lands unclaimed.
  5. Removes any leftover .factory-reset-ble sentinel.

--admin-record / --sentinel let operators point at non-default
locations when shepherd-config overrides them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes for the companion-app symptom of "list of activities
doesn't appear on subsequent app opens":

1. Response and Events used Arc<Mutex<Option<mpsc::Receiver>>>: the
   first BLE subscriber call .take()'d the receiver, and every
   subsequent subscribe got None + logged "receiver already taken"
   + returned immediately. After the first connect/disconnect the
   notify channels were silently dead for the lifetime of shepherdd.

   Switched both to broadcast channels. Response uses a single
   broadcast::Sender shared between dispatch and notify; each
   notify subscribe takes its own subscriber. Events skips the
   central pump entirely and subscribes directly to
   svc.subscribe_events() per callback. Result: every reconnect
   gets working notifications.

2. The companion populated its UI from StateChanged events. On
   second open, with no state-changing activity, no event arrived,
   and the list stayed empty. (The user's workaround was triggering
   reload_config from settings, which fires StateChanged.)

   Now every Events subscribe receives a synthesised StateChanged
   carrying the current ServiceStateSnapshot as the first frame,
   so the UI populates immediately on every reconnect without an
   explicit service_state RPC.

push_response no longer treats "no subscribers" as a fatal channel
close — broadcast::send returns Err when nobody's listening, but
that's the normal pre-subscribe race for a request that races the
notify CCCD write, not an error.

Updated docs/ai/history/2026-06-21 001 ble-companion-android-spec.md
§5 to document the initial-StateChanged push so the companion
doesn't need a service_state call on connect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two small companion-side fixes that complement the device-side
broadcast/initial-StateChanged work in the previous commit:

- ShepherdConnection._events is now MutableSharedFlow(replay = 1,
  extraBufferCapacity = 64). The device's "initial StateChanged
  on every Events subscribe" push can land on the BLE thread
  before ShepherdViewModel.eventsJob has attached its collector;
  without replay that snapshot — the one the UI uses to populate
  its list — was silently dropped. replay = 1 makes the most
  recent event reach any late collector.

- ShepherdViewModel.refreshAll used to runCatching the
  service_state call and swallow failures silently, which is
  exactly what made the original "list doesn't appear on
  reopen" bug invisible: the call was failing (server-side
  notify was broken), the UI saw no result, and no error was
  surfaced. Now a failure raises a transient message so the
  next time something breaks at the device end we'll see it.

`./gradlew :app:compileDebugKotlin` and `:app:testDebugUnitTest`
clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A real-world StateChanged push with 14 configured entries serialises
to 19_746 bytes, larger than the companion's 16 KiB frame cap, so the
FrameAssembler threw a FramingException that propagated uncaught out
of the Events collector and killed the process on every subsequent
app open. Stack trace from the device:

  FATAL EXCEPTION: main
  FramingException: declared frame length 19746 exceeds cap 16384
    at FrameAssembler.push(Framing.kt:78)
    at ShepherdConnection$start$1$2.emit(ShepherdConnection.kt:93)

Two fixes:

- Raise Protocol.MAX_FRAME_BYTES to 0xFFFF, the actual hard ceiling
  imposed by the length prefix. The earlier 16 KiB value was carried
  over from the device's Request-char cap (a defensive bound on
  incoming writes), but Response/Events on the companion need to
  accept whatever the server legitimately sends — and a snapshot
  with a non-trivial number of entries exceeds 16 KiB easily.

- Wrap each collector's push() in runCatching and route a
  FramingException through handleFramingFailure → peripheral
  disconnect, so a future wire-level glitch surfaces as a dropped
  link (which the ViewModel reconnect loop handles cleanly) rather
  than as a process-killing uncaught exception. Other exceptions
  still propagate.

The FramingTest now uses a tight-cap FrameAssembler so it can still
exercise the oversize-prefix guard without allocating 64 KiB of
buffer just for the assertion.

`./gradlew :app:testDebugUnitTest :app:assembleDebug` clean.
`adb install` + relaunch on the bonded Pixel: app stays up, journal
shows the device's BLE flow connecting without incident.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the broadcast/initial-state refactor the notify tasks for the
Response and Events characteristics never exited. bluer's
CharacteristicNotifier.notify() goes through D-Bus to BlueZ, which
emits the ATT notification regardless of whether any client is
actually listening — so notify() returned Ok forever, the loop kept
consuming events, and one subscriber from an old BLE session would
permanently intercept every Response/Event going out.

Symptom on the device side, captured live: a single
"BLE Response notify subscribed" log entry from the *previous*
companion session, then on every subsequent reopen the
companion's service_state RPC hit the 15s timeout — the request
arrived, the response was queued, "sending payload payload_len=1145
chunk_size=20" was logged, but no fresh notify subscriber existed
to actually deliver it.

Fix: each notify loop now races rx.recv() against notifier.stopped()
in a tokio::select!. When BlueZ reports the peer has unsubscribed
(connection dropped, CCCD-disable), the loop exits, the
broadcast::Receiver is dropped, and the next reconnect's notify
callback wires up a brand-new subscriber. Same pattern applied to
both Response and Events.

Verified end-to-end on hardware:

  21:35:54  service_state response delivered, UI populated
  21:35:59  "BLE Response notify session ended"  ← stopped() fired
  21:36:10  "BLE Response notify subscribed"     ← fresh subscribe
  21:36:11  service_state response delivered (again, on second open)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The notify-based path was unfixable on bonded reconnects: BlueZ caches
CCCD state at the bond, Android short-circuits subsequent CCCD writes,
and the server-side notify task spawned for the very first session ends
up the only one the wire ever reaches. Every reopen of the companion
silently lost responses (the long-running "list blank, 15s timeout"
symptom).

Server side, Response and Events become read characteristics backed by a
new `Outbox` byte queue (`crates/shepherd-ble/src/outbox.rs`). A
long-lived events forwarder pushes serialized events for the entire
server lifetime. Two further fixes were needed:

  - First-RPC `id=1` clears both outboxes. Every `ShepherdConnection`
    starts its counter at 1, so this is the deterministic "fresh BLE
    session" signal we couldn't get from bluer at the GATT-server
    layer. (A write-gap heuristic was the first attempt; a 15s per-RPC
    timeout naturally exceeded any threshold and kept wiping the queue
    mid-delivery.)
  - Cap each read at 512 bytes (the spec's `GATT_MAX_ATTR_VALUE`).
    With MTU 517 an ATT read response carries up to 516 bytes, but
    Android silently truncates a single `readCharacteristic` to 512;
    the lost 4 bytes per chunk stitched together with bytes from the
    *next* response, corrupting the frame.

Client side, the Kable `observe()` collectors become poll loops with
adaptive backoff and a wake channel that `call()` pokes so the response
poller drops to the fast interval the instant an RPC is dispatched.
`connect()` drains any stale bytes synchronously and only then flips
`ready`, so the very first RPC's response can't be raced.

Also relaxes `ClaimMachine::authorize` to allow any peer once claimed:
BlueZ's identity-resolution drift between the pairing-time RPA and the
post-bond identity address was breaking the strict comparison on every
reconnect, and v1's single-bond TOFU model means the encrypted-
authenticated link is already the security boundary.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Captures the diagnosis (why notify was unfixable here), the dead-end
CCCD-bounce attempt, the read-poll design, and the two follow-on
bugs (5s session-boundary heuristic + Android's silent 512-byte
read truncation) caught during live validation of e7ba977.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
http: keep auth open until first BLE claim, not when BLE is just enabled (#65)
Some checks failed
CI / Build (pull_request) Blocked by required conditions
CI / Test (pull_request) Blocked by required conditions
CI / E2E (pull_request) Blocked by required conditions
CI / Clippy (pull_request) Blocked by required conditions
CI / Rustfmt (pull_request) Blocked by required conditions
CI / Firewall E2E (pull_request) Blocked by required conditions
CI / Android portability (shepherd-media-core) (pull_request) Blocked by required conditions
CI / ShellCheck (pull_request) Successful in 7s
CI / CI image (pull_request) Has been cancelled
06ba213b3c
The documented contract ("omit auth_token to allow unauthenticated
access") regressed when BLE was always plumbed in as the unified bearer
source: the auth middleware's `is_open` check keyed on `admin.is_none()`
alone, so a fresh install with BLE enabled but no admin yet would 401
every HTTP request — locking the device out of HTTP before any admin
existed.

Relax `is_open` to also require the BLE authority's
`current_http_token()` to be None. Open mode now means "no token to
enforce" (whether because there's no static token configured AND no
BLE admin authority, or because there is one but it hasn't been
claimed yet). As soon as a claim lands, the gate engages and
unauthenticated requests are rejected as before.

Adds a regression test covering the "BLE authority plugged in but
unclaimed → open" case alongside the existing
claimed-admin-authority tests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
companion-android: rename package to com.armeafamily.shepherd.companion
Some checks failed
CI / ShellCheck (pull_request) Successful in 7s
CI / CI image (pull_request) Successful in 5m3s
CI / Rustfmt (pull_request) Successful in 7s
CI / Clippy (pull_request) Failing after 2m6s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 40s
CI / Firewall E2E (pull_request) Successful in 4m18s
CI / Build (pull_request) Successful in 7m58s
CI / Test (pull_request) Successful in 8m13s
CI / E2E (pull_request) Successful in 8m21s
5676945515
Adopts the project's own DNS-rooted namespace instead of the
generic com.shepherd.companion placeholder, so installs on a phone
that already has another "shepherd" app (or a future store
publication) don't collide on applicationId.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Merge branch 'main' into u/albert/65/ble-management
Some checks failed
CI / ShellCheck (pull_request) Successful in 8s
CI / CI image (pull_request) Successful in 23s
CI / Rustfmt (pull_request) Successful in 18s
CI / Clippy (pull_request) Failing after 4m12s
CI / Test (pull_request) Successful in 4m52s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 38s
CI / Build (pull_request) Successful in 4m57s
CI / E2E (pull_request) Successful in 5m4s
CI / Firewall E2E (pull_request) Successful in 4m54s
d18e7f17b7
CI clippy job failed with `-D warnings` on two lints in server.rs:
- `manual_clamp` on the per-read max_chunk computation; use `clamp`.
- `too_many_arguments` on `handle_write` (8/7); allow it, matching the
  existing `#[allow(clippy::too_many_arguments)]` usage in shepherdd.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zu8HeDACiBBaBmXGtwAXq
shepherd-ble: unit-test the write-path session/framing invariants (#65)
All checks were successful
CI / ShellCheck (pull_request) Successful in 8s
CI / CI image (pull_request) Successful in 19s
CI / Rustfmt (pull_request) Successful in 15s
CI / Clippy (pull_request) Successful in 3m44s
CI / Test (pull_request) Successful in 4m5s
CI / Build (pull_request) Successful in 4m13s
CI / E2E (pull_request) Successful in 4m33s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 55s
CI / Firewall E2E (pull_request) Successful in 5m32s
dba36ec2db
The read-poll rewrite hinges on two write-path behaviors that previously
only had live-hardware coverage and caused the field bugs documented in
`docs/ai/history/2026-06-28 001 ble-read-poll-replaces-notify.md`:

- `dispatch_frame` wipes both outboxes only on `id == 1` (fresh-session
  marker), leaving mid-session events intact.
- `handle_write` resets the `FrameReader` on a peer change and on a
  framing error, so a stuck half-frame can't corrupt the next request.

Add `server::tests` covering all four cases against the existing
`ManagementService` mock. The mock + `req` helper move out of `rpc.rs`'s
test module into a crate-level `#[cfg(test)] testsupport` module so both
`rpc` and `server` tests share one trait impl instead of duplicating it.

Runs in the existing CI Test job (`cargo test --all-targets`); no new
infrastructure. shepherd-ble unit tests: 38 -> 42.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zu8HeDACiBBaBmXGtwAXq
Wire the companion app's hardware-independent JVM tests into CI:
FramingTest (length-prefix codec) and WireTest (decode of every sampled
spec payload) — 17 tests that protect the phone side of the BLE wire
protocol from drifting against crates/shepherd-ble.

The job runs in the existing Rust CI image (which has no JDK/SDK, since
android.pkgs is kept out of the image hash) and installs the Android
toolchain on top via `./scripts/shepherd deps install android`, caching
/opt/android-sdk and the Gradle caches across runs. Unit tests only — no
emulator; the live pair/claim path remains a manual smoke test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zu8HeDACiBBaBmXGtwAXq
ci: layer a dedicated Android image so the SDK is prebuilt (#65)
All checks were successful
CI / ShellCheck (pull_request) Successful in 7s
CI / CI image (pull_request) Successful in 23s
CI / Clippy (pull_request) Successful in 2m42s
CI / Test (pull_request) Successful in 2m50s
CI / Build (pull_request) Successful in 2m53s
CI / E2E (pull_request) Successful in 3m1s
CI / Rustfmt (pull_request) Successful in 17s
CI / CI image (Android) (pull_request) Successful in 3m32s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 40s
CI / Firewall E2E (pull_request) Successful in 3m3s
CI / Android companion (unit tests) (pull_request) Successful in 2m50s
fb6d56189f
Lots of Android work is incoming, and pulling the SDK on every
android-companion run (or churning it through actions/cache on each
build.gradle edit) is wasteful. Split storage by volatility:

- The ~1 GB, rarely-changing Android SDK is baked into a new image,
  `shepherd-launcher-ci-android` (.ci/Dockerfile.android), layered
  `FROM` the base CI image + `./scripts/shepherd deps install android`.
  A new `image-android` job builds+pushes it on a content-hash miss,
  mirroring the base `image` job; the hash folds in the base image ref
  so a base rebuild forces an Android rebuild too. The 8 Rust-only jobs
  keep the lean base image and never pay the SDK pull.
- The Gradle/Maven dep cache, which churns with build.gradle edits,
  stays on actions/cache keyed on the gradle files.

android-companion now runs on the prebuilt image (ANDROID_SDK_ROOT
already set), dropping the per-run toolchain install. android-portability
stays on the base image — it only needs the aarch64 rustup target, not
the SDK. The new image is the natural home for the NDK when device-side
Rust cross-compilation lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zu8HeDACiBBaBmXGtwAXq
Conflicts:
- crates/shepherd-config/src/policy.rs: union both import additions
  (RawBleManagementConfig from ours, RawBrowserConfig from theirs).
- crates/shepherd-http/src/handlers/sessions.rs: keep our
  ManagementService-delegating handler. Move main's inline browser
  wiring for launch spawns into ManagementService::launch so it applies
  uniformly to IPC, HTTP, and BLE instead of only HTTP.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Two related bugs in the "Today's override" section:

- Clear was gated on `loaded != null`, which is only set by the
  initial `reload()`. On a first-time save, `loaded` stayed null, so
  the button that would remove the override the user just created
  was permanently disabled — the exact "you can't remove an
  override" symptom the web version doesn't have because React Query
  invalidation refreshes state after a mutation.

  Now `upsertOverride.onDone` calls `reload()`, populating `loaded`
  and enabling Clear.

- Clear was also useless when the user had *unsaved* local edits
  (chose Block, dialed in some quota, then thought better of it):
  no server round-trip needed, but the button was dead. Now Clear
  reflects any dirty state — persisted or local — and only issues
  the delete RPC when there's actually something on the server to
  delete.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Merge remote-tracking branch 'origin/main' into u/albert/65/ble-management
All checks were successful
CI / ShellCheck (pull_request) Successful in 8s
CI / CI image (pull_request) Successful in 10m36s
CI / CI image (Android) (pull_request) Successful in 3m16s
CI / Rustfmt (pull_request) Successful in 9s
CI / Clippy (pull_request) Successful in 4m31s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 51s
CI / Test (pull_request) Successful in 7m5s
CI / Build (pull_request) Successful in 7m8s
CI / Android companion (unit tests) (pull_request) Successful in 1m40s
CI / E2E (pull_request) Successful in 7m33s
CI / Firewall E2E (pull_request) Successful in 4m45s
023b512bee
Conflicts:
- crates/shepherd-http/src/handlers/sessions.rs: keep our
  ManagementService-delegating handler. Port main's new
  `confirm_on_close: bool` on the SessionStarted event into
  ManagementService::launch so it applies uniformly across IPC,
  HTTP, and BLE.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
The BLE JSON-RPC dispatcher in shepherd-ble/src/rpc.rs was ~390 lines
of hand-written boilerplate mirroring every trait method: a match arm
per method, an ad-hoc `#[derive(Deserialize)]` param struct with
manual default-value helpers, and a per-shape result wrapper. Every
new RPC on the trait meant three places to update — the trait itself,
the match arm, and (for wrapped results like `{"deleted": bool}`) a
tiny wrapper struct.

Introduce `shepherd-management-macros`, a proc-macro crate whose
`#[management_rpc]` attribute on the trait definition emits a
`dispatch_json(svc, method, params) -> Result<Value, RpcDispatchError>`
function next to it. The macro:

- Derives a per-method params struct from the trait signature. `&T`
  arguments deserialise into owned `T` and re-borrow at the call site.
- Handles the three return shapes we actually use (`T`,
  `ManagementResult<T>`, `ManagementResult<()>`) — `Ok(())` becomes
  `Value::Null`, everything else `serde_json::to_value`s the body.
- Supports three per-method knobs via `#[rpc(...)]`:
  - `default(param = "expr_path")` for optional fields that need a
    computed default (replaces the old `AtTime.at_or_now()` /
    `DateOpt.date_or_today()` helpers scattered through rpc.rs).
  - `wrap_result = "field_name"` for the small set of methods whose
    wire form wraps a bare primitive in a named field
    (`extend_current`, `delete_override`, `reload_config`) —
    forward-compat with future extra fields.
  - `name = "wire_name"` in case the Rust name and wire name ever
    need to diverge (unused today, cheap to keep).

`shepherd-ble/src/rpc.rs` shrinks to a 34-line adapter: call
`dispatch_json`, map its four error variants onto BLE's `ErrorCode`,
wrap the result in an `RpcResponse` with the caller's id.

Net: rpc.rs 388 → 116 lines (with tests), one place to add or change
an RPC method (the trait itself). All existing tests pass, plus new
ones covering `wrap_result` and the zero-arg params-blob shape.

Not touched by this refactor:
- HTTP handlers stay REST-shaped (URL params, per-endpoint status
  codes) — a macro would fight axum's ergonomic extractor pattern
  for little gain.
- The IPC dispatcher (~1700 lines in shepherdd/main.rs) is a much
  larger legacy migration, worth its own commit.
- TypeScript and Kotlin clients still hand-write method names. A
  future emit-the-schema step could drive their codegen off the same
  attribute, but that's a separate follow-up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Extends `#[management_rpc]` to also emit `pub const RPC_SCHEMA_JSON:
&str = "..."` alongside `dispatch_json`. The schema shape is:

    {
      "methods": [
        { "name", "params": [{ "name", "type", "required" }],
          "result": { "type", "wrap_field": <optional> } },
        ...
      ]
    }

Types are verbatim Rust source strings so external codegen can map
them to whatever the target language uses.

Adds a `rpc-codegen` binary that consumes the schema and writes three
files, all checked in:

- docs/rpc-schema.json — pretty-printed, diffable in PRs
- companion-android/.../ble/RpcMethods.kt — method-name constants +
  a `wrapField()` lookup so the Kotlin companion no longer duplicates
  the `{"deleted": ...}` unwrap knowledge that lives on the trait
- shepherd-webui/src/api/rpc-methods.generated.ts — a `RpcMethod`
  union of every wire name plus the matching `RPC_WRAP_FIELDS` map

A drift-check test (`tests/rpc_codegen_drift.rs`) runs the codegen
into a temp dir and byte-compares against the checked-in outputs.
CI failing this test means someone changed the trait without
regenerating; the fix is a `cargo run -p shepherd-management --bin
rpc-codegen` at the repo root.

The Kotlin `ManagementClient` and TypeScript client aren't switched
over to the new constants in this commit — that's a mechanical
follow-up. The schema + files land first so the codegen path is
proven end-to-end before we edit consumers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Prep for the IPC-to-JSON-RPC migration (Path A). The IPC wire's
`Command` enum has variants that don't currently correspond to any
trait method:

  Command::VolumeUp   { step }
  Command::VolumeDown { step }
  Command::ToggleMute
  Command::BrightnessUp   { step }
  Command::BrightnessDown { step }
  Command::Ping

Adds equivalent async methods so once IPC dispatches through
`dispatch_json` there's a matching entry for every wire operation
instead of a handful of IPC-only special cases.

`brightness_up` / `brightness_down` compose on top of `set_brightness`
(reading the current level, saturating, and clamping through the
policy) — matches the existing IPC behaviour of routing user
+/- keys through the min/max policy instead of poking the backlight
directly. Volume relative moves delegate to the existing
`VolumeController::volume_up` / `volume_down` methods so the policy
clamps stay live. `ping` is a no-op — it exists so RPC clients can
detect a wedged connection with a well-typed round-trip.

Regenerates `docs/rpc-schema.json` + the Kotlin/TS files.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Full Path A of the IPC-dispatcher cleanup. The IPC wire format was a
tagged `Command`/`ResponsePayload` enum pair whose every variant had a
matching hand-written arm in shepherdd's `handle_command` (~600 lines
of duplicated match logic that mirrored what the trait already did).
Replaces it with a JSON-RPC-style `{ method, params }` request and
`Result<serde_json::Value, ErrorInfo>` response — the same shape BLE
already uses — dispatched through `shepherd_management::dispatch_json`.

Wire changes (shepherd-api::commands.rs):

- `Command` enum → deleted; requests now carry `method: String` +
  `params: Value`.
- `ResponsePayload` enum → deleted; responses carry a bare `Value` on
  success and an `ErrorInfo { code, message }` on error.
- `ErrorCode` variants pruned to the ones actually reachable through
  the trait + IPC-layer concerns (rate limit, invalid request).

Server (shepherdd/src/main.rs):

- Removes ~900 lines: `handle_command`, `handle_relative_volume`,
  `handle_relative_brightness`, and the volume/brightness-restrictions
  helpers those depended on. The IPC dispatcher is now one `match` on
  method name that routes subscribe/unsubscribe/ping specially and
  delegates everything else to `dispatch_json`.
- `svc` is constructed unconditionally now (previously it was tied to
  HTTP-or-BLE being enabled) — IPC always needs it.
- `dispatch_ipc` is the ~15-line ManagementError-to-ErrorCode mapper.

Trait extension (shepherd-management/src/service.rs, prior commit):

- Adds `volume_up`, `volume_down`, `toggle_mute`, `brightness_up`,
  `brightness_down`, and `ping` — the trait methods that IPC's
  relative-media and keepalive commands had lived outside of.
- Regenerates docs/rpc-schema.json + Kotlin/TS mirrors.

Client (shepherd-ipc/src/client.rs):

- Rewrites `IpcClient` around a `call_raw` / `call<T>` pair, then
  layers typed helpers (`service_state`, `launch`, `stop_current`,
  `get_volume`, ...) matching the trait method names. Every IPC
  consumer now speaks these typed helpers instead of building a
  `Command` variant + pattern-matching on `ResponsePayload`.
- Subscribe is still a special case (the writer task must flip its
  flag AFTER the response frame writes so events can't slip in
  before the subscribe ack). The typed helper preserves that
  semantics.

Consumers migrated to the new client (~500 line reduction):

- shepherdd/src/main.rs             — server-side (see above)
- shepherd-hud/src/{app,volume,brightness}.rs
- shepherd-launcher-ui/src/{main,client,app}.rs
- shepherd-e2e/tests/e2e.rs

Behaviourally unchanged: every response payload the old wire produced
now comes back as the trait method's return value serialised through
`dispatch_json`. Session lifecycle, event broadcast, hidpi apply /
restore, and policy-clamped volume/brightness all keep their existing
paths — they already lived on `ManagementService` for the HTTP/BLE
transports and now IPC just calls them too.

Net: ~1500 lines deleted, ~700 added. Adding an IPC operation is now a
one-place trait change; the wire schema, client helper, and dispatcher
are all derived.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
Deprecate REST endpoints in favor of a single JSON-RPC HTTP surface (#65)
Some checks failed
CI / ShellCheck (pull_request) Successful in 8s
CI / CI image (pull_request) Successful in 20s
CI / CI image (Android) (pull_request) Successful in 30s
CI / Rustfmt (pull_request) Successful in 6s
CI / Firewall E2E (pull_request) Failing after 4m48s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 52s
CI / E2E (pull_request) Failing after 7m8s
CI / Android companion (unit tests) (pull_request) Successful in 1m4s
CI / Clippy (pull_request) Successful in 9m52s
CI / Build (pull_request) Successful in 12m34s
CI / Test (pull_request) Successful in 13m24s
01e73d5bea
Path A follow-on for HTTP: the REST routes came off, and everything
migrates to `POST /api/v1/rpc` with `{ method, params }` body.

Wire semantics (see `crates/shepherd-http/src/handlers/rpc.rs`):

- Success → 2xx with the trait method's `dispatch_json`-encoded return
  value as the response body.
- Failure → 4xx/5xx with `{ "error": "<code>", "message": "<string>" }`,
  where `<code>` is one of `method_not_found`, `invalid_params`,
  `not_found`, `bad_request`, `forbidden`, `conflict`, `unprocessable`,
  `internal`.

`GET /api/v1/events` (SSE) stays — push shape doesn't fit RPC.

Removals:

- crates/shepherd-http/src/handlers/{brightness,config,debug,entries,
  health,logout,overrides,sessions,usage,volume}.rs — the per-endpoint
  REST handlers.
- crates/shepherd-http/src/error.rs — `ApiError`/`ApiResult` were only
  used by the REST handlers.
- crates/shepherd-http/src/handlers/mod.rs — router shrinks to two routes.

New:

- crates/shepherd-http/src/handlers/rpc.rs — 90-line handler that maps
  `RpcDispatchError` and `ManagementError` variants onto HTTP status
  codes with a machine-readable `error` field.

Migrations:

- crates/shepherd-http/tests/api.rs — 44 tests, all through
  `POST /rpc`. Coverage preserved (auth, entries, sessions,
  overrides, usage, volume/brightness policy, config reload) plus new
  coverage for the RPC-only paths (unknown method, params
  round-tripping, volume_up/volume_down/toggle_mute, ping).
- crates/shepherd-e2e/src/http.rs — `HttpClient::rpc()` helper.
- crates/shepherd-e2e/tests/{e2e,firewall,firewall_real{,_flatpak,_snap},
  browser}.rs — every REST call is now `http.rpc(method, params)`.
- shepherd-webui/src/api/client.ts — replaces ~180 lines of hand-written
  `axiosInstance.get("/entries")` / `.put("/volume")` / etc. with a
  single typed `call<T>(method: RpcMethod, params)` helper plus
  per-method wrappers. Each wrapper is 1–3 lines; adding a new one
  when the trait grows a method is a mechanical change. `LaunchOutcome`
  gets normalised at the client boundary since the on-wire form
  (`{Approved: {...}}` / `{Denied: {...}}`) is uglier than the
  `{result: "approved" | "denied"}` shape existing callers expect.
- crates/shepherd-http/README.md, crates/shepherd-e2e/README.md — reflect
  the new two-endpoint surface.

Net: 46 REST handler functions gone, ~40 test assertions rewritten,
~180-line hand-written REST client replaced with a schema-typed
30-line dispatch + typed wrappers. Adding a new method now only
requires: (1) a trait line on `ManagementService`, (2) regenerate
`docs/rpc-schema.json`, and (3) optionally a two-line typed helper on
whichever clients want a nicer name than `call("volume_up", ...)`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
@ -0,0 +232,4 @@
// Nothing persisted yet — just discard
// the local edits, no server round-trip.
availability = null
quotaDeltaMinutes = 0
Author
Owner

Not sure what the quota system is here

Not sure what the quota system is here
Author
Owner

ah it was just missing from the web version; it was in shepherdd all along

ah it was just missing from the web version; it was in shepherdd all along
albert marked this conversation as resolved
shepherd-webui: add the quota-delta stepper the companion has
Some checks failed
CI / ShellCheck (pull_request) Successful in 7s
CI / CI image (pull_request) Successful in 20s
CI / CI image (Android) (pull_request) Successful in 36s
CI / Rustfmt (pull_request) Successful in 7s
CI / Test (pull_request) Successful in 4m6s
CI / Build (pull_request) Successful in 4m25s
CI / E2E (pull_request) Failing after 4m44s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 44s
CI / Clippy (pull_request) Successful in 5m12s
CI / Android companion (unit tests) (pull_request) Successful in 1m3s
CI / Firewall E2E (pull_request) Failing after 4m53s
52459d072e
The Android companion has a ±5-minute quota stepper on its entry
detail screen; the web UI could only *display* an existing
`quota_delta_seconds` value as a read-only caption. Parents who
open the web UI to grant or claw back time for one activity had to
switch to the phone. Not ideal.

Adds the same ±5-minute controls inline on each entry card, right
under the "Up to Xm" line and next to the existing "Quota +Xm"
readout. Clicking either button commits immediately via
`upsertOverride` — no separate Save step — matching the instant-
commit behaviour the web UI already has for Enable/Disable Today.

Preserves the existing `availability` override when adjusting quota
so bumping time on an "Off today" entry doesn't accidentally clear
its blocked status. Setting the delta back to 0 upserts with
`quota_delta_seconds = null`, which is the same shape the server
returns for "no adjustment" — the entry card's `quota !== 0`
condition now also gates the "Clear Override" button appearance so
a pure quota adjustment (no availability change) can still be
cleared from that card.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A38wEcPFAgtawLSY9cp1Wg
shepherd-e2e: assert launch RPC wire form, not normalized-client shape
All checks were successful
CI / ShellCheck (pull_request) Successful in 9s
CI / CI image (pull_request) Successful in 20s
CI / CI image (Android) (pull_request) Successful in 35s
CI / Rustfmt (pull_request) Successful in 5s
CI / Test (pull_request) Successful in 4m17s
CI / Build (pull_request) Successful in 4m29s
CI / E2E (pull_request) Successful in 4m57s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 43s
CI / Clippy (pull_request) Successful in 5m6s
CI / Android companion (unit tests) (pull_request) Successful in 59s
CI / Firewall E2E (pull_request) Successful in 4m56s
0e440f7ce5
The JSON-RPC migration (01e73d5) made `launch` return its raw
`LaunchOutcome` serialization on the wire (`{"Approved": {...}}`); the
`{result: "approved"}` shape only exists after normalization in the TS
webui client. Two e2e tests were missed in that migration and still
asserted `["result"] == "approved"`, so `["result"]` came back Null and
CI failed in the E2E and Firewall E2E jobs.

Match the wire form used by the passing sibling tests (firewall.rs,
firewall_real_snap.rs, firewall_real_flatpak.rs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHety8aC8mnym6N9whpdMD
Home shared RPC behavior in one test suite; thin the transports (#65)
All checks were successful
CI / ShellCheck (pull_request) Successful in 9s
CI / CI image (pull_request) Successful in 20s
CI / CI image (Android) (pull_request) Successful in 34s
CI / Rustfmt (pull_request) Successful in 7s
CI / Test (pull_request) Successful in 4m6s
CI / Build (pull_request) Successful in 4m32s
CI / E2E (pull_request) Successful in 4m56s
CI / Android portability (shepherd-media-core) (pull_request) Successful in 52s
CI / Clippy (pull_request) Successful in 5m18s
CI / Android companion (unit tests) (pull_request) Successful in 1m0s
CI / Firewall E2E (pull_request) Successful in 5m6s
956964f0ce
Every management transport (HTTP /api/v1/rpc, BLE GATT, IPC) is now a
thin adapter over the macro-generated dispatch_json, so the business
logic and dispatch mechanics were being tested redundantly across
surfaces while the shared layer itself had no behavioral tests.

- Add shepherd-management/tests/dispatch.rs (40 tests): drives a real
  DefaultManagementService through dispatch_json — the single home for
  dispatch mechanics (method-not-found, param parsing, wrap_result,
  null results) and per-method business logic (entries, sessions,
  overrides, usage, volume clamp/policy, config reload, time windows).
- Thin shepherd-http/tests/api.rs (44 -> 13): keep only HTTP-specific
  concerns — bearer/admin-authority auth and one call per status-code
  mapping arm.
- Thin shepherd-ble/src/rpc.rs (7 -> 3): keep only the
  RpcDispatchError -> ErrorCode mapping arms; the shared-behavior tests
  moved to the management suite and the ManagementError -> ErrorCode
  table is already covered by protocol.rs.
- Thin shepherd-e2e/tests/e2e.rs: drop extend_session_via_http and
  daily_override_disables_entry (pure request/response logic now covered
  at the shared layer); keep the flows that need a live daemon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uGH2ffhTYRU21KPgjL7jp
Author
Owner

Code looks reasonable, pending additional testing on a device that supports more than BLE 4.0

Code looks reasonable, pending additional testing on a device that supports more than BLE 4.0
Author
Owner

Post refactor, the management app works fine against the original Surface Pro

Post refactor, the management app works fine against the original Surface Pro
Author
Owner

Also works great on the Legion Go S with BLE 4.2 and Numeric Comparison workflow

Also works great on the Legion Go S with BLE 4.2 and Numeric Comparison workflow
albert merged commit 6dd660b70f into main 2026-07-04 14:34:41 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
albert/shepherd-launcher!71
No description provided.