Compare commits

...

1903 Commits

Author SHA1 Message Date
Peter Steinberger
f6e8a76aab test: update whatsapp reply quote assertions 2025-12-23 02:27:46 +01:00
Peter Steinberger
a3c191006e fix: add whatsapp reply context 2025-12-23 02:26:11 +01:00
Peter Steinberger
dd35ed97b8 🤖 codex: add telegram reply context
# Conflicts:
#	src/telegram/bot.ts
2025-12-23 02:25:26 +01:00
Tu Nombre Real
8431874b15 fix(macOS): remove redundant kickstart -k causing gateway restart loop
The launchd bootstrap already starts the gateway job. The subsequent
kickstart -k was killing it immediately after startup, and combined
with KeepAlive=true, this caused a port-conflict restart loop where
launchd would try to restart while the old instance was still
shutting down.

Symptoms: 'Bootstrap failed: 5: Input/output error' and repeated
'Gateway failed to start: another gateway instance is already
listening' messages in the log.
2025-12-23 01:57:54 +01:00
Peter Steinberger
54d2ccda99 feat(mac): surface update-ready state 2025-12-23 01:42:33 +01:00
Peter Steinberger
926b6d9464 chore: format wake gate + chat theme 2025-12-23 01:41:13 +01:00
Peter Steinberger
abfb6832c3 fix(mac): default session menu checks 2025-12-23 01:36:01 +01:00
Peter Steinberger
ceeea359fc chore: remove shared build artifacts 2025-12-23 01:32:02 +01:00
Peter Steinberger
ef35868bef feat: share wake gate via SwabbleKit 2025-12-23 01:31:59 +01:00
Peter Steinberger
cf48d297dd docs: explain tool exposure in pi-mono 2025-12-23 00:29:38 +00:00
Peter Steinberger
2b20e3d2b0 chore: resolve docs list from cwd 2025-12-23 00:28:55 +00:00
Peter Steinberger
918cbdcf03 refactor: lint cleanups and helpers 2025-12-23 00:28:55 +00:00
Peter Steinberger
f5837dff9c chore: add oxlint type-aware lint 2025-12-23 00:28:55 +00:00
Peter Steinberger
ce04308c17 refactor: remove session syncing metadata 2025-12-23 00:50:51 +01:00
Peter Steinberger
c0c20ebf3e feat: replace clawdis skills with tools 2025-12-22 23:40:57 +00:00
Peter Steinberger
823195a122 style(mac): increase session row padding 2025-12-23 00:10:38 +01:00
Peter Steinberger
581583abb4 fix(mac): drop syncing menu + show state checks 2025-12-23 00:10:38 +01:00
Peter Steinberger
882fd48408 style: add visual effect host for chat 2025-12-23 00:10:38 +01:00
Peter Steinberger
91238df13f chore: alias console subsystem names 2025-12-22 23:06:15 +00:00
Peter Steinberger
ca806897c2 Template: Add smart heartbeat logic for baby agents
- Added heartbeat section with proactive check guidelines
- Includes email, calendar, weather, mentions rotation
- Track checks in heartbeat-state.json
- Know when to reach out vs stay quiet
- Proactive work suggestions (memory, git, docs)

Goal: Baby agents should check in 2-4x daily, not just HEARTBEAT_OK
2025-12-22 22:55:27 +00:00
Peter Steinberger
9118884e92 fix(web): restore creds before auth check 2025-12-22 22:55:27 +00:00
Peter Steinberger
e403f8b620 style(pi): sort imports 2025-12-22 22:55:27 +00:00
Peter Steinberger
6205b955da style(mac): adjust session row padding and menu options 2025-12-22 23:30:25 +01:00
Peter Steinberger
d265a04b19 style(mac): pad session rows + thicken bars 2025-12-22 23:22:36 +01:00
Peter Steinberger
afc09744b4 fix(mac): size highlighted session rows 2025-12-22 22:59:59 +01:00
Peter Steinberger
1e1d76d600 fix(mac): restore sessions bars with injected submenus 2025-12-22 22:49:37 +01:00
Peter Steinberger
0b70aa0c56 fix(mac): hide sessions header when disconnected 2025-12-22 22:09:26 +01:00
Peter Steinberger
4ca6591045 refactor: move OAuth storage and drop legacy sessions 2025-12-22 21:02:48 +00:00
Peter Steinberger
9717f2d374 fix: bump pi deps and fix lint 2025-12-22 20:45:38 +00:00
Peter Steinberger
469c8a1a4b fix(mac): show disconnected sessions + sleeping eyes 2025-12-22 21:13:33 +01:00
Peter Steinberger
9d47b15575 fix(mac): sessions error UI + sleeping icon 2025-12-22 21:02:45 +01:00
Peter Steinberger
a11a204b8e chore(submodules): bump Peekaboo 2025-12-22 19:44:48 +00:00
Peter Steinberger
e3c3d108fe refactor(logging): shorten subsystem prefixes 2025-12-22 19:42:22 +00:00
Peter Steinberger
8cadb5cf18 docs: update group chat commands 2025-12-22 20:36:34 +01:00
Peter Steinberger
f10c8f2b4c feat: add group activation command 2025-12-22 20:36:29 +01:00
Peter Steinberger
5d2d701e1e docs: note mac studio session log location 2025-12-22 20:26:23 +01:00
Peter Steinberger
f24d8473b1 fix(mac): restore session usage bar 2025-12-22 20:14:54 +01:00
Peter Steinberger
3412ff7003 style: add macos chat glass background 2025-12-22 19:55:17 +01:00
Peter Steinberger
15e468f5dd feat: add group chat activation mode 2025-12-22 19:32:12 +01:00
Peter Steinberger
a0dd504991 feat(mac): sessions submenus 2025-12-22 19:29:24 +01:00
Peter Steinberger
19b847b23b style: tighten macos chat composer 2025-12-22 19:08:23 +01:00
Peter Steinberger
3b134c8fef style: tighten chat compose spacing 2025-12-22 19:01:58 +01:00
Peter Steinberger
c872f37aae fix: remove redundant await in CanvasManager 2025-12-22 18:53:14 +01:00
Peter Steinberger
3ce5b9b0d9 test: extend gateway sigterm timeouts 2025-12-22 18:52:35 +01:00
Peter Steinberger
2d7c5f8c53 refactor: migrate embedded pi to sdk 2025-12-22 18:05:44 +01:00
Peter Steinberger
79c0fd27a0 fix: center debug status overlay 2025-12-21 20:43:06 +01:00
Peter Steinberger
b06d1ed072 docs(logging): clarify console color behavior 2025-12-21 17:36:30 +00:00
Peter Steinberger
52e7a4456a refactor(logging): streamline whatsapp console output 2025-12-21 17:36:24 +00:00
Peter Steinberger
f1202ff152 chore: fix lint + build 2025-12-21 15:58:37 +01:00
Peter Steinberger
e4db7cbd2b chore: bump Peekaboo submodule 2025-12-21 15:57:09 +01:00
Peter Steinberger
ff63204d17 fix(web): harden WhatsApp creds persistence 2025-12-21 13:58:31 +00:00
Peter Steinberger
4f3a3e93a9 style: biome formatting 2025-12-21 13:58:27 +00:00
Peter Steinberger
b56d4b90ce fix(logging): repair chalk/tslog typing 2025-12-21 13:58:22 +00:00
Peter Steinberger
6c2f9b3150 chore: update Peekaboo submodule 2025-12-21 14:50:28 +01:00
Peter Steinberger
a808cdce13 fix(android): drop duplicate scaffold asset 2025-12-21 14:50:28 +01:00
Peter Steinberger
a8629e1855 fix(logging): simplify tty color detection 2025-12-21 13:34:13 +00:00
Peter Steinberger
0146784e18 feat(logging): add console color modes 2025-12-21 13:26:50 +00:00
Peter Steinberger
249b85af1e refactor(gateway): switch logs to subsystem logger 2025-12-21 13:24:15 +00:00
Peter Steinberger
efc12ab28d refactor(browser): use subsystem logger 2025-12-21 13:24:15 +00:00
Peter Steinberger
5b2e7d4464 refactor(logging): add subsystem console formatting 2025-12-21 13:24:15 +00:00
Peter Steinberger
bcd3c13e2c feat(macos): surface canvas debug status 2025-12-21 14:21:06 +01:00
Peter Steinberger
7932e966db feat(android): toggle debug canvas status 2025-12-21 14:21:06 +01:00
Peter Steinberger
30d84643db feat(ios): toggle debug canvas status 2025-12-21 14:21:06 +01:00
Peter Steinberger
264c91e620 feat(canvas): gate debug status overlay 2025-12-21 14:21:06 +01:00
Peter Steinberger
db89be4106 chore: update peekaboo submodule 2025-12-21 13:10:20 +00:00
Peter Steinberger
85816a5ee2 fix(cli): hint peekaboo unauthorized 2025-12-21 13:09:48 +00:00
Peter Steinberger
5449e44381 chore: bump Peekaboo submodule 2025-12-21 14:01:28 +01:00
Peter Steinberger
20630b8744 chore: bump Peekaboo + menu cleanup 2025-12-21 13:59:41 +01:00
Peter Steinberger
3b63d1cb77 fix: auto-restart WhatsApp QR login 2025-12-21 13:36:26 +01:00
Peter Steinberger
5703b9e737 docs: clarify restart semantics 2025-12-21 12:47:18 +01:00
Peter Steinberger
02787b5674 build(mac): add notarize flow for release artifacts 2025-12-21 12:33:45 +01:00
Peter Steinberger
4021da524c fix(chat-ui): avoid animated initial scroll 2025-12-21 12:33:41 +01:00
Peter Steinberger
5adec0eae0 fix: align canvas defaults and A2UI auto-nav 2025-12-21 12:32:36 +01:00
Peter Steinberger
3f44f0b753 ui: simplify dashboard health status 2025-12-21 12:31:56 +01:00
Peter Steinberger
2a975f751b refactor(macos): regroup menu sections 2025-12-21 12:29:29 +01:00
Peter Steinberger
03bd049291 docs: refine header ctas for github pages 2025-12-21 12:29:29 +01:00
Peter Steinberger
6ddd36666e feat(ui): make chat the landing view 2025-12-21 11:24:39 +00:00
Peter Steinberger
3791db006e docs: add github/download buttons to pages header 2025-12-21 12:19:08 +01:00
Peter Steinberger
6bf8c0c17a docs: note npm release pitfalls 2025-12-21 04:10:20 +01:00
Peter Steinberger
80e1934f4e style: fix tailscale swiftformat 2025-12-21 03:52:28 +01:00
Peter Steinberger
7415fdb79b chore: whitelist npm files 2025-12-21 03:48:23 +01:00
Peter Steinberger
b850b0dacf ci: install swiftlint and swiftformat for ios 2025-12-21 03:44:18 +01:00
Peter Steinberger
04e3d0c2fe style: swiftformat cleanup 2025-12-21 03:44:12 +01:00
Peter Steinberger
3810519671 chore: update appcast for 2.0.0-beta2 2025-12-21 03:29:03 +01:00
Peter Steinberger
a08c8ef1fa chore: bump version to 2.0.0-beta2 2025-12-21 03:21:49 +01:00
Peter Steinberger
6496a288b8 fix: add A2UI inset vars 2025-12-21 03:21:49 +01:00
Peter Steinberger
9f72eb3374 docs: add canvas gutter guidance 2025-12-21 03:21:48 +01:00
Peter Steinberger
e71c71c6c2 fix: add canvas gutter vars for A2UI 2025-12-21 03:21:48 +01:00
Peter Steinberger
0197fb35fe fix: clear canvas error banner on load 2025-12-21 03:21:48 +01:00
Peter Steinberger
bcc5891e03 fix(mac): allow tailscale localapi http 2025-12-21 02:17:55 +00:00
Peter Steinberger
f90ab3c4c2 fix(mac): trim onboarding checklist 2025-12-21 01:57:18 +00:00
Peter Steinberger
79280f3d93 fix(mac): tighten onboarding layout 2025-12-21 01:57:18 +00:00
Peter Steinberger
ce79d0b9a4 docs: add Peter tailnet/gateway notes 2025-12-21 02:55:32 +01:00
Peter Steinberger
a5b4a01594 fix(mac): shrink onboarding + respect existing workspace 2025-12-21 01:51:48 +00:00
Peter Steinberger
5b25eeb449 refactor(macos): remove manual identity onboarding 2025-12-21 01:39:50 +00:00
Peter Steinberger
fb259e8a50 fix(mac): shrink onboarding height 2025-12-21 01:35:27 +00:00
Peter Steinberger
b82dfe08a2 fix: prefer header mime for media extensions 2025-12-21 02:34:19 +01:00
Peter Steinberger
4671c9e672 fix: align A2UI canvas background 2025-12-21 02:34:19 +01:00
Peter Steinberger
00cdcd4d28 fix(mac): guard onboarding workspace bootstrap 2025-12-21 01:31:31 +00:00
Peter Steinberger
4e1fe88195 Give workspace templates actual personality
- SOUL.md: Philosophy over bullet points, genuine vs performative help
- IDENTITY.md: Invites creativity, frames identity as discovery
- USER.md: Learning about a person, not building a dossier
- BOOTSTRAP.md: Conversational first-run, not robotic steps
- AGENTS.md: 'This folder is home' - clear, direct, practical
- TOOLS.md: Explains why separate from skills, real examples

New agents should boot with spark, not corporate drone energy. 🦞
2025-12-21 01:24:13 +00:00
Peter Steinberger
28ad475ab4 feat(mac): add tailscale settings 2025-12-21 01:16:49 +00:00
Peter Steinberger
104e265633 docs: clarify wacli usage 2025-12-21 02:14:52 +01:00
Peter Steinberger
382d237a60 build: silence mac packaging warnings 2025-12-21 02:06:12 +01:00
Peter Steinberger
de2fd659ab fix(mac): shrink onboarding height 2025-12-21 00:57:11 +00:00
Peter Steinberger
d2fda411f3 docs: add 2.0.0-beta2 changelog 2025-12-21 01:54:27 +01:00
Peter Steinberger
e02944c323 docs: fix npmjs header image 2025-12-21 01:54:27 +01:00
Peter Steinberger
a01f4998c5 ci: split ios workflow 2025-12-21 00:49:20 +00:00
Peter Steinberger
aa198594fd fix(mac): avoid buttonStyle ternary 2025-12-21 00:49:07 +00:00
Peter Steinberger
406a94bf76 fix: use A2UI message context 2025-12-21 01:48:21 +01:00
Peter Steinberger
fef1841fee build: update iOS lint scripts 2025-12-21 01:48:21 +01:00
Peter Steinberger
1cb85fdea8 fix(mac): disambiguate skills install ForEach 2025-12-21 00:47:49 +00:00
Peter Steinberger
78263e81f1 fix(mac): restore skills install ForEach 2025-12-21 00:46:38 +00:00
Peter Steinberger
053c8d5731 feat(gateway): add tailscale auth + pam 2025-12-21 00:44:39 +00:00
Peter Steinberger
d69064f364 fix(gateway): avoid crash in handshake auth 2025-12-21 00:41:06 +00:00
Peter Steinberger
fedb24caf1 fix(ui): stabilize skills action column 2025-12-21 00:37:29 +00:00
Peter Steinberger
6ff8371254 feat(ui): expand control dashboard 2025-12-21 00:34:39 +00:00
Peter Steinberger
7b6eaa819e chore: ignore ClawdisKit .swiftpm 2025-12-21 01:10:06 +01:00
Peter Steinberger
e94aa296e2 feat: refine skills install actions 2025-12-21 01:07:35 +01:00
Peter Steinberger
98891103d0 fix: streamline WhatsApp login flow 2025-12-21 01:07:35 +01:00
Peter Steinberger
383097a03a fix: emit delta-only node system events 2025-12-21 01:07:35 +01:00
Peter Steinberger
2b2f13ca79 fix: restore canvas action bridge 2025-12-21 01:07:35 +01:00
Peter Steinberger
78159a9435 fix(onboarding): nudge bottom padding 2025-12-20 23:52:45 +00:00
Peter Steinberger
b4af7b919e fix(macos): simplify skills view and resize onboarding 2025-12-20 23:45:50 +00:00
Peter Steinberger
65056915d3 fix(onboarding): lift bottom bar 2025-12-20 23:36:24 +00:00
Peter Steinberger
bc3f744e45 chore(canvas): refresh a2ui bundle 2025-12-21 00:25:56 +01:00
Peter Steinberger
fb8da15b01 chore(canvas): rebuild a2ui bundle 2025-12-21 00:25:56 +01:00
Peter Steinberger
62f624b66b fix(mac): re-ensure remote gateway tunnel 2025-12-21 00:25:56 +01:00
Peter Steinberger
ef20053e72 style(tests): format gateway server test 2025-12-21 00:25:56 +01:00
Peter Steinberger
aae68e4f82 style(chatui): fix SwiftFormat warnings 2025-12-21 00:25:56 +01:00
Peter Steinberger
1d715d7b1b chore(ios): link AppIntents framework 2025-12-21 00:24:24 +01:00
Peter Steinberger
1d7110ea8f fix(onboarding): fit chat card 2025-12-20 23:15:35 +00:00
Peter Steinberger
80f70a58e3 fix(chat): refine onboarding bubbles 2025-12-20 23:15:29 +00:00
Peter Steinberger
f7aabeba04 chore(deps): update lockfile 2025-12-20 23:00:31 +00:00
Peter Steinberger
02f6cac9d6 style(chat): use integrated bubble tail 2025-12-20 23:00:21 +00:00
Peter Steinberger
df54fc6098 test(gateway): cover provider status/logout RPCs 2025-12-20 23:51:36 +01:00
Peter Steinberger
fe0fb8d296 chore(canvas): rebuild a2ui bundle 2025-12-20 22:45:15 +00:00
Peter Steinberger
591120a7f7 chore(deps): update dependencies 2025-12-20 22:45:15 +00:00
Peter Steinberger
878f074494 chore(android): update kotlin compiler settings 2025-12-20 23:43:28 +01:00
Peter Steinberger
c1050da852 chore(android): update icons and platform config 2025-12-20 23:43:28 +01:00
Peter Steinberger
873daf079c feat(web): emit provider status updates 2025-12-20 23:43:27 +01:00
Peter Steinberger
df9e4bdd63 chore(macos): tidy discovery and runtime 2025-12-20 23:43:27 +01:00
Peter Steinberger
43ba1671f1 feat(macos): add connections settings
# Conflicts:
#	apps/macos/Sources/Clawdis/SettingsRootView.swift
2025-12-20 23:43:27 +01:00
Peter Steinberger
ce4b68d5fb fix: pre-size menu context card 2025-12-20 23:43:27 +01:00
Peter Steinberger
8c18dd40a3 feat(macos): load models from gateway 2025-12-20 23:43:27 +01:00
Peter Steinberger
e3015bbfb7 test(gateway): cover models.list 2025-12-20 23:43:27 +01:00
Peter Steinberger
817abd8b5f feat(gateway): add models.list 2025-12-20 23:43:27 +01:00
Peter Steinberger
dbc9b00de5 docs: improve oracle skill guidance 2025-12-20 23:41:07 +01:00
Peter Steinberger
b635e83651 chore(pi): bump deps, drop steerable transport 2025-12-20 22:38:12 +00:00
Peter Steinberger
7aeacdcc6c style(settings): widen window 2025-12-20 22:23:15 +00:00
Peter Steinberger
16e4a0c4bd style(onboarding): refine bubble tails 2025-12-20 22:23:06 +00:00
Peter Steinberger
d613800516 fix(onboarding): anchor bottom bar and reduce height 2025-12-20 22:16:13 +00:00
Peter Steinberger
94b89216f7 style(onboarding): add speech bubble tails 2025-12-20 22:08:01 +00:00
Peter Steinberger
153e09120a style(onboarding): lower bottom row 2025-12-20 22:07:51 +00:00
Peter Steinberger
238c0c1b86 fix(onboarding): clearer bubbles and tighter composer 2025-12-20 22:03:24 +00:00
Peter Steinberger
98ff213708 style(onboarding): lower bottom controls 2025-12-20 22:03:13 +00:00
Peter Steinberger
8a2a07eddb fix(macos): always show CLI installer 2025-12-20 22:00:51 +00:00
Peter Steinberger
9076d543f3 fix(onboarding): restore bubbles and spacing 2025-12-20 21:56:03 +00:00
Peter Steinberger
cd77dc9563 fix(onboarding): restore chat bubble styling 2025-12-20 21:47:43 +00:00
Peter Steinberger
9ccf80848d style(onboarding): reduce window height 2025-12-20 21:33:56 +00:00
Peter Steinberger
78cb565dc2 docs: align canvas host port guidance 2025-12-20 22:28:35 +01:00
Peter Steinberger
6a30452b4a fix: use bridge canvas host for nodes 2025-12-20 22:28:35 +01:00
Peter Steinberger
e53442d983 style(voicewake): widen label and clarify language 2025-12-20 21:14:46 +00:00
Peter Steinberger
bc079b29c3 fix(macos): fix skill install target access 2025-12-20 22:01:11 +01:00
Peter Steinberger
cd6addd742 chore(ci): swiftformat macos settings 2025-12-20 21:52:47 +01:00
Peter Steinberger
12d6e1cddd feat(macos): choose skill install target 2025-12-20 21:52:42 +01:00
Peter Steinberger
28e5ebd72b feat(macos): support gateway bind config 2025-12-20 21:52:19 +01:00
Peter Steinberger
e8106109e3 Merge remote-tracking branch 'origin/main' 2025-12-20 21:43:30 +01:00
Peter Steinberger
c71d5a8a77 docs: expand sag pronunciation rules 2025-12-20 21:43:03 +01:00
Peter Steinberger
d1d27a0bd6 style(onboarding): refine icon and bottom bar spacing 2025-12-20 20:24:18 +00:00
Peter Steinberger
ebb7428479 style(onboarding): nudge icon up 2025-12-20 20:19:18 +00:00
Peter Steinberger
3163a42f36 chore(skills): fix eightctl homepage 2025-12-20 21:18:40 +01:00
Peter Steinberger
35a25c3dc2 refactor(macos): collapse control channel status 2025-12-20 21:17:32 +01:00
Peter Steinberger
f34f374179 chore(macos): widen settings window 2025-12-20 21:17:29 +01:00
Peter Steinberger
aa330350fc refactor(macos): simplify sessions header 2025-12-20 21:17:24 +01:00
Peter Steinberger
a2cf1f98d9 refactor(macos): move skills filter into header 2025-12-20 21:17:20 +01:00
Peter Steinberger
f84def1b60 chore(skills): add homepage metadata 2025-12-20 21:12:57 +01:00
Peter Steinberger
91d4c24078 refactor(macos): simplify skills list rows 2025-12-20 21:12:57 +01:00
Peter Steinberger
8fe0b72a04 fix: accept new ssh host keys 2025-12-20 21:06:39 +01:00
Peter Steinberger
2bcdf741f9 feat(cron): require job name 2025-12-20 19:56:49 +00:00
Peter Steinberger
9ae73e87eb fix(onboarding): restore bottom bar padding 2025-12-20 19:50:30 +00:00
Peter Steinberger
77582ff5d4 refactor(macos): refresh skills settings layout 2025-12-20 20:49:32 +01:00
Peter Steinberger
52a2dfe08b feat(onboarding): hide kickoff bubble and tweak typing 2025-12-20 19:46:06 +00:00
Peter Steinberger
09d2165d36 style(onboarding): lower welcome icon 2025-12-20 19:44:35 +00:00
Peter Steinberger
fb9c1f7e65 perf(dmg): shrink rw image before lzma convert 2025-12-20 19:44:26 +00:00
Peter Steinberger
abf05af474 chore(ci): format macos relay 2025-12-20 20:41:21 +01:00
Peter Steinberger
714ba2a58d docs(macos): update bundled bun notes 2025-12-20 19:35:33 +00:00
Peter Steinberger
405ff0377a refactor(macos): bundle single relay binary 2025-12-20 19:35:30 +00:00
Peter Steinberger
8421ef7b4a feat(gateway): add gateway-daemon command 2025-12-20 19:35:30 +00:00
Peter Steinberger
fd151c4fc6 chore(ci): fix biome formatting 2025-12-20 20:33:27 +01:00
Peter Steinberger
b36b20d246 feat(voicewake): add computer wake word 2025-12-20 20:33:03 +01:00
Peter Steinberger
44ffe41775 fix(macos): allow identity refresh off main actor 2025-12-20 20:32:04 +01:00
Peter Steinberger
2ca7c2629c chore(ci): fix swiftformat lint 2025-12-20 20:32:04 +01:00
Josh Palmer
483c0e4cea chore(ci): fix biome + swiftformat lint 2025-12-20 20:32:04 +01:00
Peter Steinberger
7d51bf0eb0 fix(macos): allow identity refresh off MainActor 2025-12-20 19:19:57 +00:00
Peter Steinberger
ab4457e2a3 fix(browser): allow control server without playwright 2025-12-20 19:16:56 +00:00
Peter Steinberger
1eb6d617f5 build(macos): bundle playwright in embedded gateway 2025-12-20 19:16:52 +00:00
Peter Steinberger
21ac34bc6a fix(gateway): start browser control server 2025-12-20 19:16:49 +00:00
Peter Steinberger
c050a82c3a fix(macos): patch bun Long for protobuf 2025-12-20 19:16:44 +00:00
Peter Steinberger
750408d0a2 chore(deps): add chromium-bidi and long 2025-12-20 19:16:41 +00:00
Peter Steinberger
a44a313f77 test: cover ssh autofill helpers 2025-12-20 19:53:15 +01:00
Peter Steinberger
d159602928 refactor: centralize gateway parsing 2025-12-20 19:53:08 +01:00
Peter Steinberger
50e817f193 fix: use local timestamps in agent envelope 2025-12-20 19:40:48 +01:00
Peter Steinberger
929a10e33d fix(web): handle self-chat mode 2025-12-20 19:32:06 +01:00
Peter Steinberger
c38aeb1081 fix: resolve bonjour txt for ssh autofill 2025-12-20 19:28:40 +01:00
Peter Steinberger
35e0894655 fix: merge bonjour txt records for ssh autofill 2025-12-20 19:27:36 +01:00
Peter Steinberger
943f0d475f fix: move host lookup off main thread 2025-12-20 19:26:04 +01:00
Peter Steinberger
96cbab2b22 test: expand mime detection coverage 2025-12-20 19:16:53 +01:00
Peter Steinberger
36c85a617a fix: use file-type for mime sniffing 2025-12-20 19:13:50 +01:00
Peter Steinberger
1356498ee1 docs: add ordercli skill 2025-12-20 18:50:51 +01:00
Peter Steinberger
49ec53f4ae fix: detect main module under PM2 2025-12-20 18:39:17 +01:00
Peter Steinberger
5687a03f0b chore: biome format 2025-12-20 18:39:17 +01:00
Peter Steinberger
cdb2a0736a docs(onboarding): add soul creation step 2025-12-20 17:38:54 +00:00
Peter Steinberger
cfd3efb6e7 docs(templates): update workspace template guidance 2025-12-20 17:35:52 +00:00
Peter Steinberger
8ec0d813c0 test: stabilize gateway sigterm startup 2025-12-20 18:29:46 +01:00
Peter Steinberger
ea5333e5f7 fix: make web inbox non-blocking 2025-12-20 18:24:05 +01:00
Peter Steinberger
b13723d3d7 style: satisfy swiftformat in chat composer 2025-12-20 18:18:30 +01:00
Peter Steinberger
03a4e0c837 docs: update summarize installer spec 2025-12-20 18:01:09 +01:00
Peter Steinberger
f49c20c508 fix: accept duplex upgrade sockets 2025-12-20 18:01:09 +01:00
Peter Steinberger
d3821123ee test: include token for canvas host hello 2025-12-20 18:01:09 +01:00
Peter Steinberger
759ab8acbc test: mock embedded queue in auto-reply tests 2025-12-20 18:01:09 +01:00
Peter Steinberger
7a88071a16 style: format skill installer logic 2025-12-20 18:01:09 +01:00
Peter Steinberger
f3c4d1a181 docs(onboarding): document chat kickoff 2025-12-20 16:52:11 +00:00
Peter Steinberger
4e491757ef feat(web): add whatsapp QR login tool 2025-12-20 16:52:11 +00:00
Peter Steinberger
5936ed7941 feat(chat): restyle onboarding chat UI 2025-12-20 16:52:11 +00:00
Peter Steinberger
6b56f7d643 feat(mac): add onboarding chat kickoff 2025-12-20 16:52:11 +00:00
Peter Steinberger
e618a21f4e style: biome formatting 2025-12-20 17:50:45 +01:00
Peter Steinberger
0f271ab535 refactor: tighten steerable agent loop typing 2025-12-20 17:50:35 +01:00
Peter Steinberger
4c054917ef feat: add uv skill installers 2025-12-20 17:50:29 +01:00
Peter Steinberger
b9eabe532e docs: update mac skills install types 2025-12-20 17:40:09 +01:00
Peter Steinberger
4ee292a952 refactor: drop pnpm skill installer 2025-12-20 17:39:54 +01:00
Peter Steinberger
adc2900aff refactor: trim skill install spec 2025-12-20 17:39:14 +01:00
Peter Steinberger
9c801e9c08 Merge remote-tracking branch 'origin/main' 2025-12-20 17:33:00 +01:00
Peter Steinberger
ba0791b896 feat: add skills search and website 2025-12-20 17:32:40 +01:00
Peter Steinberger
c4a67b7d02 feat: refresh skills metadata and toggles 2025-12-20 17:32:05 +01:00
Peter Steinberger
bd572c775d refactor: remove canvasHost port config 2025-12-20 17:15:43 +01:00
Peter Steinberger
65329496a7 refactor: serve canvas host on gateway port 2025-12-20 17:13:36 +01:00
Peter Steinberger
2288ec7384 fix(mac): align cli button height 2025-12-20 16:02:05 +00:00
Peter Steinberger
80b3b9e00c docs(onboarding): refine bootstrap convo 2025-12-20 15:54:40 +00:00
Peter Steinberger
3876c1679a feat(workspace): add bootstrap ritual 2025-12-20 15:48:57 +00:00
Peter Steinberger
ba85f4a62a test: cover tailnet hello canvas host 2025-12-20 16:45:26 +01:00
Peter Steinberger
a1b34ef0ef refactor: extract canvas a2ui handler 2025-12-20 16:45:26 +01:00
Peter Steinberger
f03d2d1b33 feat: advertise cli path for remote ssh 2025-12-20 16:45:26 +01:00
Peter Steinberger
c7048973bb chore(agent): track upstream steerable loop 2025-12-20 16:45:26 +01:00
Peter Steinberger
e800e84a77 fix(macos): streamline onboarding ui 2025-12-20 15:20:31 +00:00
Peter Steinberger
d306fcb8a2 fix(macos): validate embedded CLI helper 2025-12-20 15:12:57 +00:00
Peter Steinberger
44339a6447 feat(agent): queue steering messages 2025-12-20 16:10:53 +01:00
Peter Steinberger
675aadc6a9 docs: document steering while streaming 2025-12-20 16:10:53 +01:00
Peter Steinberger
d95c09d94a feat(gateway): enrich agent WS logs 2025-12-20 14:54:38 +00:00
Peter Steinberger
f508fd3fa2 feat(macos): auto-enable local gateway 2025-12-20 14:47:37 +00:00
Peter Steinberger
cf96ad8ef9 fix: route voice wake to main 2025-12-20 15:33:28 +01:00
Peter Steinberger
066a2828c4 fix(macos): clarify bridge discovery labels 2025-12-20 14:27:27 +00:00
Peter Steinberger
b6c11154ae Merge branch 'main' of https://github.com/steipete/clawdis 2025-12-20 14:22:08 +00:00
Peter Steinberger
6ca897e055 fix(telegram): normalize chat ids and improve errors 2025-12-20 14:21:49 +00:00
Peter Steinberger
23ffa1905a style: soften hover hud status dot 2025-12-20 15:20:58 +01:00
Peter Steinberger
a88e5968ae fix(macos): hide local bridge discovery 2025-12-20 14:19:22 +00:00
Peter Steinberger
4abaf62783 feat(macos): clarify local gateway choice 2025-12-20 14:11:57 +00:00
Peter Steinberger
9bf5b92d8f fix: clarify remote gateway error 2025-12-20 15:05:57 +01:00
Peter Steinberger
044f525eb8 fix: include tailnetDns in wide-area beacons 2025-12-20 15:02:23 +01:00
Peter Steinberger
554d9bc6ce fix: stabilize a2ui bundle output 2025-12-20 14:54:37 +01:00
Peter Steinberger
49654803aa style: fix lint formatting 2025-12-20 14:54:37 +01:00
Peter Steinberger
44c951e432 test(web): cover tool summary streaming 2025-12-20 13:53:56 +00:00
Peter Steinberger
e1b8c30163 feat(web): toggle tool summaries mid-run 2025-12-20 13:52:04 +00:00
Peter Steinberger
70faa4ff36 feat(web): stream tool summaries 2025-12-20 13:47:07 +00:00
Peter Steinberger
63b63cd66d style(auto-reply): format bare /new 2025-12-20 13:31:46 +00:00
Peter Steinberger
137980b46e fix(agents): support loadSkillsFromDir result 2025-12-20 13:31:46 +00:00
Peter Steinberger
055d839fc3 feat(runtime): bootstrap PATH for clawdis 2025-12-20 13:31:46 +00:00
Peter Steinberger
3e39dd49aa fix: auto-detect tailnet DNS hint 2025-12-20 14:23:53 +01:00
Peter Steinberger
082b4fb193 docs: note imsg chats json 2025-12-20 14:17:34 +01:00
Peter Steinberger
de1f119a7d fix: add ClawdisIPC import 2025-12-20 14:07:07 +01:00
Peter Steinberger
7ce12863b8 fix: clarify SSH test failure 2025-12-20 14:07:07 +01:00
Peter Steinberger
1ab69948a5 chore(canvas): refresh a2ui bundle 2025-12-20 13:06:34 +00:00
Peter Steinberger
13298d84ea test(agents): cover empty managed skills dir 2025-12-20 13:04:59 +00:00
Peter Steinberger
c2c5b28c70 feat(auto-reply): greet on bare /new 2025-12-20 13:04:55 +00:00
Peter Steinberger
6e200ed1c0 fix(agents): handle managed skills list 2025-12-20 12:59:57 +00:00
Peter Steinberger
3fadbb29a1 docs: refresh peekaboo skill details 2025-12-20 13:56:42 +01:00
Peter Steinberger
6e4eef4a49 docs(skill): add clawdis nodes 2025-12-20 12:56:06 +00:00
Peter Steinberger
8feb09aa89 fix(skills): ship runnable brave/openai scripts 2025-12-20 12:54:18 +00:00
Peter Steinberger
e1a3bab7e5 feat(skills): add media/transcription helpers 2025-12-20 12:53:09 +00:00
Peter Steinberger
e0cd5650c5 style: biome formatting 2025-12-20 12:52:14 +00:00
Peter Steinberger
80c09f0845 docs(skill): add clawdis notify 2025-12-20 12:51:20 +00:00
Peter Steinberger
1f831c6037 docs(skill): update canvas A2UI guidance 2025-12-20 12:48:08 +00:00
Peter Steinberger
cc0075e988 feat: add skills settings and gateway skills management 2025-12-20 13:33:42 +01:00
Peter Steinberger
4b44a75bc1 docs: add summarize skill 2025-12-20 13:33:16 +01:00
Peter Steinberger
f46beec20d docs: add clawdis cron skill 2025-12-20 13:33:16 +01:00
Peter Steinberger
973bf67683 feat(skills): add extraDirs load paths 2025-12-20 12:26:58 +00:00
Peter Steinberger
ff6a918e7e feat(skills): load bundled skills 2025-12-20 12:23:53 +00:00
Peter Steinberger
5ef2666127 docs(canvas): update A2UI hosting 2025-12-20 12:17:39 +00:00
Peter Steinberger
ed001a5f55 refactor(canvas): host A2UI via gateway 2025-12-20 12:17:27 +00:00
Peter Steinberger
13ebbd1a2b feat: parse skill install metadata 2025-12-20 13:00:57 +01:00
Peter Steinberger
ca8e556619 docs: align brave-search skill 2025-12-20 13:00:03 +01:00
Peter Steinberger
8900c84155 docs: finalize skill install hints 2025-12-20 13:00:03 +01:00
Peter Steinberger
002d927874 docs: expand skill install hints 2025-12-20 13:00:03 +01:00
Peter Steinberger
cef5bf2768 docs: add skill install hints 2025-12-20 13:00:03 +01:00
Peter Steinberger
529543b36d build: refresh a2ui bundle 2025-12-20 13:00:03 +01:00
Peter Steinberger
636e4d38d5 style: tidy macos swift formatting 2025-12-20 13:00:03 +01:00
Peter Steinberger
2d8e11b78b docs: refine skills 2025-12-20 13:00:03 +01:00
Peter Steinberger
0e2993a6c8 fix(skills): prevent skills loading crash 2025-12-20 11:49:24 +00:00
Peter Steinberger
f0ebad3f21 fix: address skills lint 2025-12-20 12:29:45 +01:00
Peter Steinberger
a02adcc2ef docs: link docs section 2025-12-20 12:27:25 +01:00
Peter Steinberger
d1850aaada feat: add managed skills gating 2025-12-20 12:22:38 +01:00
Peter Steinberger
cf21a15e06 chore: remove dist from repo 2025-12-20 12:22:38 +01:00
Peter Steinberger
13124542cf fix(a2ui): improve modal styling 2025-12-20 11:12:11 +00:00
Peter Steinberger
cd5809d11f fix(a2ui): stabilize canvas host 2025-12-20 10:58:13 +00:00
Peter Steinberger
28938ddb32 chore: update a2ui bundle 2025-12-20 11:32:20 +01:00
Peter Steinberger
3c551fd36f docs(browser): update hook timeouts 2025-12-20 09:47:21 +00:00
Peter Steinberger
94c495c8ed fix(browser): default hook timeout 2m 2025-12-20 09:45:04 +00:00
Peter Steinberger
f54c801bd2 fix(browser): extend hook arm timeouts 2025-12-20 09:43:58 +00:00
Peter Steinberger
429972b5c5 test(browser): cover agent contract 2025-12-20 09:34:22 +00:00
Peter Steinberger
9b8a4d0c76 docs(browser): simplify control contract 2025-12-20 03:27:17 +00:00
Peter Steinberger
235f3ce0ba refactor(browser): simplify control API 2025-12-20 03:27:12 +00:00
Peter Steinberger
06806a1ea1 fix(mac): probe loopback bridge 2025-12-20 03:05:06 +00:00
Peter Steinberger
b1a85d89d2 docs(browser): update browser tool surface 2025-12-20 02:53:26 +00:00
Peter Steinberger
6fc30962d6 refactor(browser): prune browser automation surface 2025-12-20 02:53:22 +00:00
Peter Steinberger
849446ae17 refactor(cli): unify on clawdis CLI + node permissions 2025-12-20 02:08:04 +00:00
Peter Steinberger
479720c169 refactor(browser): trim observe endpoints 2025-12-20 02:07:27 +00:00
Peter Steinberger
0e94c6b025 fix(browser): restore tsc types 2025-12-20 01:27:51 +00:00
Peter Steinberger
1a51257b71 fix(mac): use gateway main session for WebChat 2025-12-20 01:27:51 +00:00
Peter Steinberger
4e74ba996d feat(macos): add unconfigured gateway mode 2025-12-20 02:21:10 +01:00
Peter Steinberger
80a87e5f9e refactor(mac): remove clawdis-mac browser cli 2025-12-20 01:06:27 +00:00
Peter Steinberger
a526d3c1f2 feat(browser): add native action commands 2025-12-20 00:53:56 +00:00
Peter Steinberger
d67bec0740 style: polish logging and lint hints 2025-12-20 01:48:29 +01:00
Peter Steinberger
b2e11c504b fix: tighten iOS main-actor handling 2025-12-20 01:48:29 +01:00
Peter Steinberger
1b38ee8b46 fix: harden device model decoding 2025-12-20 01:48:29 +01:00
Peter Steinberger
afa4a234f9 fix: remove WhatsApp batching delay 2025-12-20 01:48:29 +01:00
Peter Steinberger
46b9006de2 docs(browser): add MCP tool spec 2025-12-19 23:57:35 +00:00
Peter Steinberger
d54ecc3961 test(browser): cover MCP tool routes 2025-12-19 23:57:32 +00:00
Peter Steinberger
fa54950d2e feat(browser): add MCP tool dispatch 2025-12-19 23:57:26 +00:00
Peter Steinberger
0ac7a93c28 fix: decode bonjour escaped utf8 2025-12-19 23:21:07 +01:00
Peter Steinberger
bc2a66da32 refactor: unify gateway discovery on bridge 2025-12-19 23:12:52 +01:00
Peter Steinberger
bcced90f11 style: lighten DMG background for label contrast 2025-12-19 22:51:54 +01:00
Peter Steinberger
eb076165d2 style: refine DMG arrow 2025-12-19 22:44:56 +01:00
Peter Steinberger
9248919b05 docs: note DMG background sizing 2025-12-19 22:39:30 +01:00
Peter Steinberger
5472589ddd fix: align DMG background and icon layout 2025-12-19 22:38:36 +01:00
Peter Steinberger
19f5183176 docs(mac): document dmg packaging 2025-12-19 22:22:14 +01:00
Peter Steinberger
beb6e25ef0 build(macos): add dmg+zip packaging 2025-12-19 22:22:09 +01:00
Peter Steinberger
0ad49c25aa style(macos): add dmg background 2025-12-19 22:22:03 +01:00
Peter Steinberger
d46823333d docs(mac): add bun gateway packaging notes 2025-12-19 22:13:13 +01:00
Peter Steinberger
836f645621 perf(macos): compile embedded gateway with bytecode 2025-12-19 22:11:41 +01:00
Peter Steinberger
96be450cbb fix: handle screen record microphone output 2025-12-19 22:09:38 +01:00
Peter Steinberger
56cb415509 fix: restore mac app build 2025-12-19 22:08:17 +01:00
Peter Steinberger
2ef2136c2c fix(macos): sign bun gateway with jit entitlements 2025-12-19 19:24:49 +01:00
Peter Steinberger
0b16b4481a chore: ignore bun build artifacts 2025-12-19 19:21:27 +01:00
Peter Steinberger
0b18f1b948 docs: update bundled gateway flow 2025-12-19 19:21:27 +01:00
Peter Steinberger
a4d4a30a6b feat(macos): run bundled gateway via launchd 2025-12-19 19:21:27 +01:00
Peter Steinberger
98bbc73925 build(macos): bundle bun gateway 2025-12-19 19:21:26 +01:00
Peter Steinberger
bb7f4abd4b feat(gateway): support bun-compiled embedded gateway 2025-12-19 19:21:26 +01:00
Peter Steinberger
bd63b5a231 fix: show Dock icon during onboarding 2025-12-19 19:21:26 +01:00
Peter Steinberger
590f3d0e8f feat(templates): centralize workspace templates 2025-12-19 18:18:15 +00:00
Peter Steinberger
6cbfa01176 docs: document WhatsApp and Telegram config 2025-12-19 19:03:17 +01:00
Peter Steinberger
f929e1b105 fix: surface gateway failure details 2025-12-19 18:48:30 +01:00
Peter Steinberger
77104395ce docs: overhaul README architecture 2025-12-19 18:41:17 +01:00
Peter Steinberger
c0d5853c63 fix(deps): include playwright-core in dependencies 2025-12-19 18:38:37 +01:00
Peter Steinberger
5bbf5105f1 chore: update appcast for 2.0.0-beta1 2025-12-19 18:24:36 +01:00
Peter Steinberger
5b193d014e ci: lower iOS coverage gate 2025-12-19 18:23:03 +01:00
Peter Steinberger
fc7a63a4de perf: throttle gateway environment checks 2025-12-19 18:21:55 +01:00
Peter Steinberger
aec1869d32 fix(ios): make parseA2UIActionBody nonisolated 2025-12-19 18:10:10 +01:00
Peter Steinberger
377169959d chore: prep 2.0.0-beta1 release 2025-12-19 18:02:30 +01:00
Peter Steinberger
ba497ce57d chore: log gateway env timings 2025-12-19 17:54:23 +01:00
Peter Steinberger
5e7d12fefa perf: move gateway env checks off main 2025-12-19 17:54:18 +01:00
Peter Steinberger
a019d3cd83 chore(protocol): regenerate schema 2025-12-19 17:52:50 +01:00
Peter Steinberger
8c6a592523 style(macos): swiftformat sources 2025-12-19 17:52:26 +01:00
Peter Steinberger
47a1774dc0 Mac: add summarize tool 2025-12-19 17:47:04 +01:00
Peter Steinberger
2bc0c57f18 build(canvas): refresh a2ui bundle 2025-12-19 17:47:04 +01:00
Peter Steinberger
f0705a928a fix(macos): allow fractional timeout 2025-12-19 17:47:04 +01:00
Peter Steinberger
22f9322905 fix(ios): refine canvas and screen handling 2025-12-19 17:47:04 +01:00
Peter Steinberger
6795e78edf fix(macos): reduce node pairing polling 2025-12-19 13:58:33 +00:00
Peter Steinberger
31620fea3a fix(control-ui): wrap long message lines 2025-12-19 09:54:43 +00:00
Peter Steinberger
6b6f2b5414 fix(control-ui): drop /ui alias 2025-12-19 05:13:07 +00:00
Peter Steinberger
c498348a34 fix(control-ui): serve dashboard at root 2025-12-19 05:11:08 +00:00
Peter Steinberger
00fc731d64 feat(macos): add menu link to dashboard 2025-12-19 04:28:32 +00:00
Peter Steinberger
d80d112e09 fix(onboarding): default identity to Clawd 2025-12-19 03:12:10 +00:00
Peter Steinberger
65d723d53c test: add canvas.present IPC coverage 2025-12-19 03:53:55 +01:00
Peter Steinberger
fb3fae43c0 feat(agent): load workspace skills 2025-12-19 03:53:55 +01:00
Peter Steinberger
41108f497b fix(onboarding): load saved identity defaults 2025-12-19 02:40:11 +00:00
Peter Steinberger
beefda7f60 refactor: replace canvas.show with canvas.present 2025-12-19 03:35:33 +01:00
Peter Steinberger
74cdc1cf3e feat: route mac control via nodes 2025-12-19 03:16:25 +01:00
Peter Steinberger
7f3be083c1 feat: add node screen recording across apps 2025-12-19 02:57:00 +01:00
Peter Steinberger
b8012a2281 fix(canvas): load A2UI resources across platforms 2025-12-19 01:53:55 +00:00
Peter Steinberger
95ea67de28 feat: add mac node screen recording and ssh tunnel 2025-12-19 02:33:43 +01:00
Peter Steinberger
1fbd84da39 feat(nodes): add mac node mode + permission UX 2025-12-19 01:48:19 +01:00
Peter Steinberger
beb5b1ad58 docs(agents): require consent for worktrees 2025-12-19 01:18:32 +01:00
Peter Steinberger
77a67484ea feat(pairing): add silent SSH auto-approve 2025-12-19 01:04:47 +01:00
Peter Steinberger
0b4e70e38b CLI: retry --force until gateway port is free 2025-12-18 23:56:08 +00:00
Peter Steinberger
8f0b5d2d97 iOS: fix camera clip clamp regression test 2025-12-19 00:53:06 +01:00
Peter Steinberger
0e3e4f269d iOS: allow Tailnet/MagicDNS canvas actions 2025-12-19 00:52:52 +01:00
Peter Steinberger
d6c5ee86c5 Docs: add nodes overview 2025-12-19 00:29:42 +01:00
Peter Steinberger
3772a29557 macOS: add screen record + safer camera defaults 2025-12-19 00:29:38 +01:00
Peter Steinberger
7831e0040e feat(macos): delay hover HUD 2025-12-19 00:25:46 +01:00
Peter Steinberger
3780f3152c macOS: auto-fill Anthropic OAuth from clipboard 2025-12-18 23:15:08 +00:00
Peter Steinberger
3146f8bdbc CanvasA2UI: refresh bundled renderer 2025-12-18 23:08:07 +00:00
Peter Steinberger
256080e2a2 Canvas host: fix action bridge invocation 2025-12-19 00:04:45 +01:00
Peter Steinberger
47510e2912 feat(macos): hover HUD for activity 2025-12-19 00:04:45 +01:00
Peter Steinberger
0c06276b48 Agent: document 2000px image downscale 2025-12-18 23:02:33 +00:00
Peter Steinberger
d66d5cc17e Agent: avoid silent failures on oversized images 2025-12-18 22:58:31 +00:00
Peter Steinberger
df0c51a63b Gateway: add browser control UI 2025-12-18 22:41:06 +00:00
Peter Steinberger
c34da133f6 CLI: fix nodes canvas snapshot option typing 2025-12-18 23:40:42 +01:00
Peter Steinberger
f237222bc9 Docs: update canvas host defaults and snapshot formats 2025-12-18 23:32:48 +01:00
Peter Steinberger
2a4ccaf993 CLI: add nodes canvas snapshot + duration parsing 2025-12-18 23:32:36 +01:00
Peter Steinberger
ac50a14b6a Gateway: enable canvas host + inject action bridge 2025-12-18 23:32:22 +01:00
Peter Steinberger
06f71d883c Android: JPEG canvas snapshots + camera permission prompts 2025-12-18 23:32:07 +01:00
Peter Steinberger
9ace6af3df iOS: allow A2UI actions from local canvas host 2025-12-18 23:31:49 +01:00
Peter Steinberger
9062f60e3d ClawdisKit: accept jpg for canvas.snapshot 2025-12-18 23:31:34 +01:00
Peter Steinberger
2307756892 iOS: allow HTTP loads in WKWebView 2025-12-18 19:59:43 +01:00
Peter Steinberger
7008493f03 Gateway: raise client maxPayload 2025-12-18 19:48:29 +01:00
Peter Steinberger
b5a89e8907 iOS: support jpeg canvas snapshots 2025-12-18 19:48:29 +01:00
Peter Steinberger
ae58838cc5 Web: fix lint/format for error formatter 2025-12-18 18:22:32 +00:00
Peter Steinberger
9a4fc3e086 Web: improve WhatsApp error formatting 2025-12-18 18:03:25 +00:00
Peter Steinberger
0241f1a29c Web: harden WhatsApp creds handling 2025-12-18 17:19:53 +00:00
Peter Steinberger
801e44f4eb feat(node): show camera capture HUD 2025-12-18 14:49:07 +01:00
Peter Steinberger
856ce06fda style: biome format ws logging 2025-12-18 14:31:10 +01:00
Peter Steinberger
d406d3a058 Gateway: optimize ws logs in normal mode 2025-12-18 13:27:52 +00:00
Peter Steinberger
0b8e8144af ci: relax iOS coverage gate 2025-12-18 14:26:13 +01:00
Peter Steinberger
ad26026802 Gateway: add compact ws verbose logs 2025-12-18 13:07:42 +00:00
Peter Steinberger
c2b8f9a7c3 style: biome format gateway server 2025-12-18 14:00:46 +01:00
Peter Steinberger
ba79977f07 Gateway: shorten ws log tag 2025-12-18 12:58:47 +00:00
Peter Steinberger
16e2193911 fix(ios): restore ScreenController.mode 2025-12-18 13:56:40 +01:00
Peter Steinberger
bb5d26ba9e Gateway: improve verbose ws logs 2025-12-18 12:47:41 +00:00
Peter Steinberger
59f9073e21 ci: retry swiftpm build/test 2025-12-18 13:37:58 +01:00
Peter Steinberger
982f85bf90 chore(naming): remove remaining iris references 2025-12-18 13:30:22 +01:00
Peter Steinberger
acdf70e928 ci: retry submodule checkout 2025-12-18 13:26:09 +01:00
Peter Steinberger
d182f7e4b2 chore(naming): remove Iris codename 2025-12-18 13:18:33 +01:00
Peter Steinberger
790079c3b6 feat(canvas): remove setMode; host A2UI in scaffold 2025-12-18 13:18:24 +01:00
Peter Steinberger
dda6d7f9e1 ci: fix swiftformat 2025-12-18 12:50:59 +01:00
Peter Steinberger
256f0fc765 Docs: add canvas host usage 2025-12-18 11:39:30 +01:00
Peter Steinberger
e1f320276e Android: hide Disconnect without remote 2025-12-18 11:39:23 +01:00
Peter Steinberger
c61bd6c84d A2UI: share web UI and action bridge 2025-12-18 11:38:32 +01:00
Peter Steinberger
8a343aedf2 Docs: document canvasHost 2025-12-18 11:36:46 +01:00
Peter Steinberger
cd729e83b6 Gateway: optional canvas host 2025-12-18 11:35:21 +01:00
Peter Steinberger
cfb36525ab Android: add canvas.a2ui push/reset 2025-12-18 10:44:50 +01:00
Peter Steinberger
6f58a9d643 iOS: support canvas.a2ui push/reset 2025-12-18 10:44:32 +01:00
Peter Steinberger
0913329b03 A2UI: share bundle via ClawdisKit 2025-12-18 10:44:06 +01:00
Peter Steinberger
402b04a68c ci: raise iOS coverage 2025-12-18 10:34:09 +01:00
Peter Steinberger
4a68b4add4 fix(android): show backdrop behind WebView 2025-12-18 09:46:32 +01:00
Peter Steinberger
a74c4db948 Tests: show unpaired nodes in nodes status 2025-12-18 08:38:33 +00:00
Peter Steinberger
0fc5ccb76c Tests: cover node.describe for connected unpaired nodes 2025-12-18 08:38:33 +00:00
Peter Steinberger
98a745b3df macOS: hide node pairing alert host window 2025-12-18 09:37:17 +01:00
Peter Steinberger
24009ed00f macOS: move instance update info to third row 2025-12-18 09:36:07 +01:00
Peter Steinberger
fceab511b3 Android: run canvas WebView loads on main 2025-12-18 08:31:56 +00:00
Peter Steinberger
c6421136f9 Docs: use canvas.* invoke namespace 2025-12-18 08:20:40 +00:00
Peter Steinberger
2f8b75d86e macOS: add leading device icons in Instances 2025-12-18 09:15:50 +01:00
Peter Steinberger
97ec5d52c3 fix(android): allow cleartext for tailnet web 2025-12-18 09:12:06 +01:00
Peter Steinberger
89fcb40557 submodules: bump Peekaboo 2025-12-18 09:06:39 +01:00
Peter Steinberger
5c705ab675 ci: fix swiftformat and bun CI 2025-12-18 08:55:47 +01:00
Peter Steinberger
2f21b94a76 iOS: fix BridgeClient SwiftFormat indent 2025-12-18 08:40:59 +01:00
Peter Steinberger
6f1ae147da ui: improve idle background blend mode fallback 2025-12-18 08:32:06 +01:00
Peter Steinberger
f2d503ad04 Android: drop screen.* invoke aliases 2025-12-18 02:17:35 +00:00
Peter Steinberger
57ee34839d CLI/docs: expose node metadata and commands 2025-12-18 02:06:36 +00:00
Peter Steinberger
82d8526732 macOS: add clawdis-mac node describe and verbose list 2025-12-18 02:06:36 +00:00
Peter Steinberger
742027a447 Gateway: list/describe node capabilities and commands 2025-12-18 02:06:35 +00:00
Peter Steinberger
efed2ae30f Nodes: advertise canvas invoke commands 2025-12-18 02:06:35 +00:00
Peter Steinberger
54830e8401 Bridge: persist advertised invoke commands 2025-12-18 02:05:40 +00:00
Peter Steinberger
ce1a8d70d9 Android: hide connected bridge from discovery list 2025-12-18 02:37:37 +01:00
Peter Steinberger
cd719a8c85 Android: centralize canvas protocol strings 2025-12-18 02:32:34 +01:00
Peter Steinberger
3df53836ca fix(ui): harden idle background animation 2025-12-18 02:27:11 +01:00
Peter Steinberger
7bb058215d Tests: loosen chat.abort mismatch timeout 2025-12-18 01:20:20 +00:00
Peter Steinberger
272015c701 Docs: document canvas.* node.invoke commands 2025-12-18 01:20:20 +00:00
Peter Steinberger
21a27e3b65 Nodes: handle canvas.* commands on iOS/Android 2025-12-18 01:20:20 +00:00
Peter Steinberger
22516437b7 Protocol: switch node.invoke screen.* to canvas.* 2025-12-18 01:20:20 +00:00
Peter Steinberger
ea53f1bec7 Android: test bridge auto-reconnect 2025-12-18 02:18:19 +01:00
Peter Steinberger
33bf5cf42a iOS: centralize canvas commands and capabilities 2025-12-18 02:16:31 +01:00
Peter Steinberger
c976799f8c CLI/docs: mention canvas.* alias 2025-12-18 01:10:40 +00:00
Peter Steinberger
f973b9e0e5 Gateway: alias canvas.* for node.invoke 2025-12-18 01:10:40 +00:00
Peter Steinberger
60321352aa Android: add Voice Wake (foreground/always) 2025-12-18 02:08:57 +01:00
Peter Steinberger
6d60224c93 fix(android): improve webview compatibility 2025-12-18 02:08:53 +01:00
Peter Steinberger
2b2434d239 fix(android): decode UTF-8 TXT records 2025-12-18 01:58:16 +01:00
Peter Steinberger
f8bea661fc iOS: alias canvas.* invoke commands 2025-12-18 01:57:31 +01:00
Peter Steinberger
86225d0eb6 fix(android): improve wide-area bridge discovery 2025-12-18 01:40:08 +01:00
Peter Steinberger
3351c972e7 refactor(android): drop legacy theme fallback 2025-12-18 01:39:57 +01:00
Peter Steinberger
460e170f7a CLI: add nodes status 2025-12-18 00:37:54 +00:00
Peter Steinberger
1a2d39bdf9 Docs: document nodes status 2025-12-18 00:37:54 +00:00
Peter Steinberger
99325040f8 gateway: persist and surface node capabilities 2025-12-18 01:36:38 +01:00
Peter Steinberger
568fcbda54 iOS: allow settings light mode 2025-12-18 01:29:45 +01:00
Peter Steinberger
f4b186a9d3 ui(nodes): unify idle background animation 2025-12-18 01:22:26 +01:00
Peter Steinberger
d862ae17eb clawdis-mac: fetch node list via gateway 2025-12-18 00:16:36 +00:00
Peter Steinberger
9f73131621 Gateway: include node caps + hardware in node.list 2025-12-18 00:16:36 +00:00
Peter Steinberger
99310a5bbb style(android): respect system theme and clamp overlays 2025-12-18 01:15:50 +01:00
Peter Steinberger
1673bf2d44 fix(android): use system DNS for wide-area discovery 2025-12-18 01:04:13 +01:00
Peter Steinberger
4c656ea22f Android: reorder settings sections 2025-12-18 01:00:50 +01:00
Peter Steinberger
7707e3d887 iOS: reorder settings sections 2025-12-18 01:00:36 +01:00
Peter Steinberger
ba204d0330 fix(android): show idle background under WebView 2025-12-18 00:53:31 +01:00
Peter Steinberger
cbb327227a macOS: unify device + OS chip 2025-12-18 00:43:58 +01:00
Peter Steinberger
14fa2f47f5 style(android): improve idle background 2025-12-18 00:41:21 +01:00
Peter Steinberger
579da8cc9b style(android): use tonal surfaces for overlays 2025-12-18 00:34:11 +01:00
Peter Steinberger
5693d7d733 macOS: remove Instances row duplication 2025-12-18 00:28:45 +01:00
Peter Steinberger
07c8fdffd1 macOS: compact Instances row 2025-12-18 00:24:10 +01:00
Peter Steinberger
d3f4db649f style(ios): use Offline bridge status 2025-12-18 00:20:37 +01:00
Peter Steinberger
abbe237cc0 style(android): use Offline bridge status 2025-12-18 00:20:28 +01:00
Peter Steinberger
ac4a65ddfd refactor(android): unify chat status label 2025-12-18 00:20:19 +01:00
Peter Steinberger
693215723a Android: enable immersive fullscreen 2025-12-18 00:07:58 +01:00
Peter Steinberger
5f0e474be1 Android: polish settings UI 2025-12-18 00:07:52 +01:00
Peter Steinberger
0e201c4c18 style(android): make chat more Material 2025-12-17 23:57:14 +01:00
Peter Steinberger
d12ca22b19 feat(android): chat parity + wide-area discovery 2025-12-17 23:49:29 +01:00
Peter Steinberger
c7b80c28a1 macOS: remove stale WebChat exclude 2025-12-17 23:31:46 +01:00
Peter Steinberger
5c2288218f fix(gateway): make chat.abort reliable 2025-12-17 23:28:37 +01:00
Peter Steinberger
0844fa38a8 style(gateway): satisfy biome 2025-12-17 23:27:27 +01:00
Peter Steinberger
3ed33c5856 chore(webchat): remove legacy bundled web assets 2025-12-17 23:27:27 +01:00
Peter Steinberger
b3e466ccb6 nodes: better default display names 2025-12-17 23:15:15 +01:00
Peter Steinberger
875cf9a054 refactor(webchat): SwiftUI-only WebChat UI
# Conflicts:
#	apps/macos/Package.swift
2025-12-17 23:05:28 +01:00
Peter Steinberger
ca85d217ec ChatUI: swiftformat fixes 2025-12-17 23:01:31 +01:00
Peter Steinberger
6652b1f4f3 ui(chat): reduce padding 2025-12-17 23:01:31 +01:00
Peter Steinberger
9fe04f5659 ui(chat): align status pill with send 2025-12-17 23:01:31 +01:00
Peter Steinberger
5b9e51bfaa ui(chat): tighten padding + keep status in composer 2025-12-17 23:01:31 +01:00
Peter Steinberger
cdea744725 ui(chat): move connection pill into composer 2025-12-17 23:01:30 +01:00
Peter Steinberger
44365f2e27 test(chat): harden abort/stream + hide session switching 2025-12-17 23:01:30 +01:00
Peter Steinberger
888dbd7d11 macOS: load device model names from dataset 2025-12-17 22:55:50 +01:00
Peter Steinberger
76ddfc4a9e fix(android): canvas idle background + tailscale DNS 2025-12-17 22:27:16 +01:00
Peter Steinberger
7950a646c3 macOS: show friendly device names in Instances 2025-12-17 22:23:57 +01:00
Peter Steinberger
09819f8b2e fix(agents): fix AgentTool schema typing 2025-12-17 22:12:19 +01:00
Peter Steinberger
69daa24869 fix(test): stabilize chat.abort 2025-12-17 22:12:16 +01:00
Peter Steinberger
35214b6dec test(gateway): stabilize chat abort 2025-12-17 22:04:54 +01:00
Peter Steinberger
fe6bf6966b style(android): format bridge hello 2025-12-17 22:04:51 +01:00
Peter Steinberger
e0276ed4b4 fix(gateway): harden request handling 2025-12-17 22:04:22 +01:00
Peter Steinberger
fce487669b feat(android): iOS canvas background 2025-12-17 22:03:11 +01:00
Peter Steinberger
e6ba373d08 feat(android): add status pill overlay 2025-12-17 22:00:12 +01:00
Peter Steinberger
d4b3d504e4 fix(android): dedupe hello fields 2025-12-17 21:53:38 +01:00
Peter Steinberger
2b2376d4c0 style(swift): fix lint 2025-12-17 21:51:36 +01:00
Peter Steinberger
51bdf01e2e Presence: add device identity fields 2025-12-17 21:51:36 +01:00
Peter Steinberger
9d29fbbf80 Docs/tests: node list hardware fields 2025-12-17 20:11:13 +00:00
Peter Steinberger
a40fc50e5e clawdis-mac: show hardware model in node list 2025-12-17 20:11:05 +00:00
Peter Steinberger
df4e4534f4 Android: advertise device model to bridge 2025-12-17 20:10:58 +00:00
Peter Steinberger
fca6e466b1 macOS: include node hardware identifiers 2025-12-17 20:10:50 +00:00
Peter Steinberger
0321174519 Tests: cover clawdis-mac node list 2025-12-17 20:03:56 +00:00
Peter Steinberger
c452f8c430 clawdis-mac: enrich node list output 2025-12-17 20:03:56 +00:00
Peter Steinberger
079c1d8786 Bridge: advertise node capabilities 2025-12-17 20:03:56 +00:00
Peter Steinberger
0677567cdd macOS: fix InstanceInfo device fields 2025-12-17 20:03:56 +00:00
Peter Steinberger
7fe7c30b17 Mobile: prevent sleep setting 2025-12-17 21:01:47 +01:00
Peter Steinberger
cc1d8060c4 fix(android): bonjour discovery parity 2025-12-17 20:57:04 +01:00
Peter Steinberger
428a82e734 feat(chat): Swift chat parity (abort/sessions/stream) 2025-12-17 20:51:27 +01:00
Peter Steinberger
cc235fc312 Docs: require permission to switch branches 2025-12-17 20:43:04 +01:00
Peter Steinberger
249f97d1ed tools: add blucli 2025-12-17 20:39:34 +01:00
Peter Steinberger
3e9310d6cd Agents: fix pi-tools typing 2025-12-17 20:38:52 +01:00
Peter Steinberger
9051c5891e Canvas: click progress + context-rich actions 2025-12-17 20:34:54 +01:00
Peter Steinberger
56d94e6974 Node pairing: avoid blocking main actor 2025-12-17 20:34:53 +01:00
Peter Steinberger
e6a96bea47 fix(macos): improve canvas A2UI forwarding 2025-12-17 20:31:21 +01:00
Peter Steinberger
cf82e37c36 Menu: reopen canvas without reload 2025-12-17 20:31:21 +01:00
Peter Steinberger
4fb3e0500a Canvas: fix A2UI click actions 2025-12-17 20:31:21 +01:00
Peter Steinberger
9c7d51429e macOS: auto-start gateway for Canvas actions 2025-12-17 20:31:21 +01:00
Peter Steinberger
c1985443fd macOS: fix gateway strict-concurrency issues 2025-12-17 20:31:21 +01:00
Peter Steinberger
17a27fd312 macOS: fold agent control into GatewayConnection 2025-12-17 20:31:21 +01:00
Peter Steinberger
557ffdbe35 Discovery: wide-area bridge DNS-SD
# Conflicts:
#	apps/ios/Sources/Bridge/BridgeDiscoveryModel.swift
#	src/cli/dns-cli.ts
2025-12-17 20:31:02 +01:00
Peter Steinberger
e9bfe34850 chore(canvas): rebuild CanvasA2UI bundle 2025-12-17 19:15:19 +00:00
Peter Steinberger
1a4540d386 feat(macos): show Anthropic auth mode + OAuth connect 2025-12-17 19:15:19 +00:00
Peter Steinberger
a0c4b1e061 test(web): avoid ENOTEMPTY cleanup race 2025-12-17 19:15:19 +00:00
Peter Steinberger
e275ba8d2e chore(a2ui): ignore lit dist build output 2025-12-17 19:15:19 +00:00
Peter Steinberger
db7eeee07b fix(macos): sync node pairing approvals 2025-12-17 19:15:19 +00:00
Peter Steinberger
84d5f24f5f chore(pi): add TODO for mime workaround 2025-12-17 19:15:19 +00:00
Peter Steinberger
42948b70e3 fix(pi): harden image read mime 2025-12-17 19:15:19 +00:00
Peter Steinberger
28d3bd03b2 chore(peekaboo): bump submodule 2025-12-17 19:15:19 +00:00
Peter Steinberger
6148f862b9 CLI: bootstrap invalid wide-area DNS zone 2025-12-17 18:02:25 +01:00
Peter Steinberger
0a32610b37 iOS: satisfy SwiftFormat in bridge discovery 2025-12-17 18:01:01 +01:00
Peter Steinberger
514759bde7 CLI: make dns setup create valid zone 2025-12-17 17:25:34 +01:00
Peter Steinberger
2eb27ffb4a CLI: dns setup supports sudo-owned CoreDNS config 2025-12-17 17:15:51 +01:00
Peter Steinberger
2ce24fdbf8 Nodes: auto-discover clawdis.internal 2025-12-17 17:01:30 +01:00
Peter Steinberger
e9ae10e569 Gateway: wide-area Bonjour via clawdis.internal 2025-12-17 17:01:10 +01:00
Peter Steinberger
a1940418fb GatewayConnection: validate agent message 2025-12-17 16:09:22 +01:00
Peter Steinberger
6fdc62c008 macOS: fold AgentRPC into GatewayConnection 2025-12-17 16:07:37 +01:00
Peter Steinberger
5e5cb7a292 Canvas: forward A2UI actions 2025-12-17 15:41:04 +01:00
Peter Steinberger
f5ab3e41c5 Android: fix unicast discovery address resolution 2025-12-17 15:32:07 +01:00
Peter Steinberger
036bdde764 Android: add unicast discovery domain + app icon 2025-12-17 15:29:45 +01:00
Peter Steinberger
691bf85d7e Canvas: shrink close button 2025-12-17 14:52:32 +01:00
Peter Steinberger
4482965d80 Canvas: add vibrancy close pill 2025-12-17 14:50:29 +01:00
Peter Steinberger
fdca8fb592 Canvas: fix A2UI push rendering 2025-12-17 14:36:42 +01:00
Peter Steinberger
c7c32210e6 Docs: secure wide-area Bonjour over Tailscale 2025-12-17 14:27:49 +01:00
Peter Steinberger
316a04f606 iOS: allow unicast DNS-SD discovery domain 2025-12-17 14:14:17 +01:00
Peter Steinberger
c4da2afb22 Build: add wireit 2025-12-17 13:20:36 +01:00
Peter Steinberger
9eaa45a291 Canvas: fix A2UI v0.8 rendering 2025-12-17 13:20:27 +01:00
Peter Steinberger
81a9439eb2 feat(macos): add menu Canvas open/close 2025-12-17 11:53:57 +01:00
Peter Steinberger
be9b550209 chore: bump Peekaboo submodule 2025-12-17 11:37:30 +01:00
Peter Steinberger
6653813cb9 fix(macos): avoid treating '/' as file target 2025-12-17 11:36:51 +01:00
Peter Steinberger
cf1278295d macOS: update config settings copy 2025-12-17 11:36:21 +01:00
Peter Steinberger
cdb5ddb2da feat(macos): add Canvas A2UI renderer 2025-12-17 11:35:06 +01:00
Peter Steinberger
1cdebb68a0 docs: document embedded agent runtime 2025-12-17 11:29:12 +01:00
Peter Steinberger
fece42ce0a feat: embed pi agent runtime 2025-12-17 11:29:04 +01:00
Peter Steinberger
c5867b2876 Canvas: simplify show + report status 2025-12-17 10:37:35 +01:00
Peter Steinberger
43e257e7de chore: drop agent-scripts AGENTS pointer 2025-12-17 10:08:07 +01:00
Peter Steinberger
9dcdeb15ec fix(macos): anchor canvas panel to active screen 2025-12-17 09:28:53 +01:00
Peter Steinberger
060a209ecb fix(system): inject transitions only 2025-12-17 08:31:23 +01:00
Peter Steinberger
e1e3da946f fix(chat): reduce system spam and cap history 2025-12-16 20:35:03 +01:00
Peter Steinberger
49a9f74753 fix(chat-ui): improve typing dots and composer 2025-12-16 20:13:23 +01:00
Peter Steinberger
74b19843ae fix(gateway): clamp chat.history to 1000 max 2025-12-16 19:55:17 +01:00
Peter Steinberger
d691e28675 fix(gateway): cap chat.history to 1000 messages 2025-12-16 19:44:49 +01:00
Peter Steinberger
2a5f0d6063 fix(gateway): cap chat.history payload size 2025-12-16 19:34:36 +01:00
Peter Steinberger
66a0813e44 test(macos): guard FileHandle read APIs 2025-12-16 10:41:47 +01:00
Peter Steinberger
64d6d25d65 fix(macos): use safe FileHandle reads 2025-12-16 10:41:47 +01:00
Peter Steinberger
b443c20cef docs(changelog): note macOS voice audio fix 2025-12-16 09:35:02 +00:00
Tu Nombre Real
5e8c8367f3 fix(macos): lazy-init AVAudioEngine to prevent Bluetooth audio ducking
Creating AVAudioEngine at singleton init time causes macOS to switch
Bluetooth headphones from A2DP (high quality) to HFP (headset) profile,
resulting in degraded audio quality even when Voice Wake is disabled.

This change makes audioEngine optional and only creates it when voice
recognition actually starts, preventing the profile switch for users
who don't use Voice Wake.

Fixes #30

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 09:35:02 +00:00
Peter Steinberger
2b0f846f1b chore(auto-reply): satisfy biome 2025-12-16 10:30:57 +01:00
Peter Steinberger
e7713a28ae fix(auto-reply): parse agent_end and avoid rpc JSON leaks 2025-12-16 10:28:57 +01:00
Peter Steinberger
7948d071e0 ui(macos): remove Claude auth skip button 2025-12-14 19:23:49 +00:00
Peter Steinberger
fb23717102 ui(macos): polish onboarding wording 2025-12-14 19:22:31 +00:00
Peter Steinberger
3d959c46d0 fix(macos): hide skipped onboarding panes 2025-12-14 19:14:05 +00:00
Peter Steinberger
4cdd61eb78 ui(macos): recommend Opus on Claude step 2025-12-14 19:13:55 +00:00
Peter Steinberger
6d08d84011 ui(macos): tweak Claude sign-in copy 2025-12-14 19:12:52 +00:00
Peter Steinberger
f6cafd1a15 fix(macos): clarify OAuth detection 2025-12-14 19:10:48 +00:00
Peter Steinberger
5792887883 docs(macos): critter-first onboarding copy 2025-12-14 06:26:51 +00:00
Peter Steinberger
e82ee731bf test(ios): bump app coverage 2025-12-14 06:09:28 +00:00
Peter Steinberger
5e09aae4ca test(ios): cover RootCanvas bridge states 2025-12-14 05:51:48 +00:00
Peter Steinberger
740f7b0fb6 test(ios): exercise ScreenController eval 2025-12-14 05:51:12 +00:00
Peter Steinberger
7510a6f66a test(ios): cover ScreenController webview setup 2025-12-14 05:42:39 +00:00
Peter Steinberger
1ff7d458a5 fix(android): avoid non-exhaustive sheet switch 2025-12-14 05:42:39 +00:00
Peter Steinberger
c3528fb201 test(web): stabilize group heartbeat test 2025-12-14 05:36:01 +00:00
Peter Steinberger
3f5dff35f8 Merge remote-tracking branch 'origin/main' 2025-12-14 05:32:24 +00:00
Peter Steinberger
08bfe2b263 Merge remote-tracking branch 'origin/main' 2025-12-14 05:31:06 +00:00
Peter Steinberger
42645a7e0a test(macos): cover control/camera disabled paths 2025-12-14 05:30:39 +00:00
Peter Steinberger
7d4c8ef6b2 fix(camera): harden capture pipeline 2025-12-14 05:30:34 +00:00
Peter Steinberger
a1d7b8db6f refactor(macos): tidy gateway discovery naming 2025-12-14 05:30:07 +00:00
Peter Steinberger
4a3a4558e2 fix(android): respect insets and enable settings scroll 2025-12-14 05:30:07 +00:00
Peter Steinberger
1b83fc85cd fix(ios): update observation env in smoke tests 2025-12-14 05:27:19 +00:00
Peter Steinberger
c1a10b6056 chore: gitignore .worktrees 2025-12-14 05:21:21 +00:00
Peter Steinberger
841a9b4c8a fix(macos): fix oauth base64 helper visibility 2025-12-14 05:19:49 +00:00
Peter Steinberger
f3db02018f fix(chat-ui): reflect gateway connection 2025-12-14 05:19:01 +00:00
Peter Steinberger
4cbaee59cd style(ios): swiftformat 2025-12-14 05:17:59 +00:00
Peter Steinberger
0d10aa4098 ui(ios): animate idle background 2025-12-14 05:17:59 +00:00
Peter Steinberger
f3f8aa5397 fix(ios): use Observation environment in settings 2025-12-14 05:17:59 +00:00
Peter Steinberger
4970af6bb9 fix(macos): satisfy swiftformat 2025-12-14 05:16:03 +00:00
Peter Steinberger
a48aebc78c iOS: Fix canvas touch events and auto-hide status bubble
- Disable scroll on WKWebView to allow touch events to reach canvas
- Add WKNavigationDelegate to intercept clawdis:// deep links from canvas
- Wire up onDeepLink callback to handle taps on canvas buttons
- Auto-hide status bubble after 3 seconds
2025-12-14 05:14:26 +00:00
Peter Steinberger
26bbddde8f style(macos): swiftformat 2025-12-14 05:09:48 +00:00
Peter Steinberger
b48a556de5 refactor(observation): migrate SwiftUI state 2025-12-14 05:06:34 +00:00
Peter Steinberger
aab5c490dc refactor(chat-ui): compact layout 2025-12-14 05:06:34 +00:00
Peter Steinberger
d54cc49d66 feat(android): sync wake words via gateway 2025-12-14 05:06:27 +00:00
Peter Steinberger
0cef22ef83 feat(ios): sync wake words via gateway 2025-12-14 05:06:27 +00:00
Peter Steinberger
7b2f712e20 feat(macos): sync wake words via gateway 2025-12-14 05:06:27 +00:00
Peter Steinberger
1a92127dfa feat(voicewake): add gateway-owned wake words sync 2025-12-14 05:06:27 +00:00
Peter Steinberger
26a05292b9 fix(macos): live-check Pi oauth.json 2025-12-14 04:48:03 +00:00
Peter Steinberger
caaa79bb76 style(ios): swiftformat 2025-12-14 04:47:15 +00:00
Peter Steinberger
b80c0d85e0 style(macos): swiftformat 2025-12-14 04:42:04 +00:00
Peter Steinberger
0641281cfe chore(protocol): sync generated artifacts 2025-12-14 04:42:04 +00:00
Peter Steinberger
f414853d70 fix(config): tolerate session store races 2025-12-14 04:42:04 +00:00
Peter Steinberger
7c677c5057 test: cover identity defaults and pi flags 2025-12-14 04:40:01 +00:00
Peter Steinberger
969c7d1c8e docs(agents): prefer Observation framework 2025-12-14 04:36:07 +00:00
Peter Steinberger
b202480a66 docs(bonjour): document gateway and iOS discovery logging 2025-12-14 04:36:00 +00:00
Peter Steinberger
9e80764c2b feat(ios): add discovery debug logs 2025-12-14 04:36:00 +00:00
Peter Steinberger
f5a5320f8f test(bonjour): cover watchdog and failure modes 2025-12-14 04:36:00 +00:00
Peter Steinberger
7389fc0e25 fix(bonjour): log advertise failures and watchdog 2025-12-14 04:36:00 +00:00
Peter Steinberger
ce915d3438 fix(android): safe area + settings scroll 2025-12-14 04:35:06 +00:00
Peter Steinberger
3ef910d23e test(macos): boost Clawdis coverage to 40% 2025-12-14 04:31:04 +00:00
Peter Steinberger
845b26a73b fix(camera): retain capture delegates 2025-12-14 04:31:04 +00:00
Peter Steinberger
e0545e2f94 fix(chat): improve history + polish SwiftUI panel 2025-12-14 04:31:04 +00:00
Peter Steinberger
01341d983c fix(macos): sane chat window placement 2025-12-14 04:31:04 +00:00
Peter Steinberger
0d68e10dd7 chore(tools): match repo emojis 2025-12-14 04:31:04 +00:00
Peter Steinberger
e6a60c0dc5 chore(tools): add emoji tool names 2025-12-14 04:31:04 +00:00
Peter Steinberger
7dbd5acbb1 fix(webchat): reconnect gateway ws 2025-12-14 04:31:04 +00:00
Peter Steinberger
7a87f3cfb8 fix(macos): suggest critter emojis only 2025-12-14 04:29:07 +00:00
Peter Steinberger
b817225fb8 feat(agent): enforce provider/model and identity defaults 2025-12-14 04:22:38 +00:00
Peter Steinberger
a097c848bb feat(macos): onboard Claude OAuth + identity 2025-12-14 04:22:38 +00:00
Peter Steinberger
a47d3e3e35 ui(macos): skip whatsapp onboarding in remote mode 2025-12-14 04:20:16 +00:00
Peter Steinberger
4d4bcaab1e ci: fix iOS simulator selection indentation 2025-12-14 04:13:07 +00:00
Peter Steinberger
265a3dff27 ci: create iOS simulator when missing 2025-12-14 04:10:06 +00:00
Peter Steinberger
97fe3972c8 chore(macos): silence onboarding type length lint 2025-12-14 04:09:20 +00:00
Peter Steinberger
7c91ce2fa7 refactor(macos): simplify bridge frame handling 2025-12-14 04:09:20 +00:00
Peter Steinberger
951993db17 ui(macos): always enable deep links 2025-12-14 04:06:34 +00:00
Peter Steinberger
357a1a982b style: satisfy formatters 2025-12-14 04:03:32 +00:00
Peter Steinberger
f6f69b408f ui(macos): remove duplicate canvas toggle 2025-12-14 04:00:57 +00:00
Peter Steinberger
98399b85e3 docs: add onboarding spec 2025-12-14 03:59:56 +00:00
Peter Steinberger
38a773f245 test(web): make heartbeat call selection deterministic 2025-12-14 03:59:40 +00:00
Peter Steinberger
e9e2e5026c ui(macos): fix security notice wrapping 2025-12-14 03:57:32 +00:00
Peter Steinberger
8649de6199 ui(macos): make master discovery selectable 2025-12-14 03:53:45 +00:00
Peter Steinberger
3885a2a20f ci: fix yaml indentation for python blocks 2025-12-14 03:51:13 +00:00
Peter Steinberger
dde9fddae4 style(swift): fix lint and formatting warnings 2025-12-14 03:49:34 +00:00
Peter Steinberger
3a08e6df9d ui(macos): skip local onboarding steps in remote mode 2025-12-14 03:49:17 +00:00
Peter Steinberger
f427bec31c ci: fix python heredoc indentation 2025-12-14 03:46:03 +00:00
Peter Steinberger
67e0739bec ui(macos): lower onboarding welcome content 2025-12-14 03:45:27 +00:00
Peter Steinberger
c7022cc139 ci: pick iOS simulator via simctl json 2025-12-14 03:39:33 +00:00
Peter Steinberger
65a0de8979 ci: raise iOS coverage gate to 50% 2025-12-14 03:39:33 +00:00
Peter Steinberger
d0134722af test(ios): cover bridge client + more views 2025-12-14 03:39:33 +00:00
Peter Steinberger
efc7181aa0 fix(macos): hide session store path in remote mode 2025-12-14 03:38:47 +00:00
Peter Steinberger
3729d269d0 feat(macos): move camera setting to General 2025-12-14 03:33:24 +00:00
Peter Steinberger
7dd8a7f2e3 ci: add Android build job 2025-12-14 03:31:00 +00:00
Peter Steinberger
56bbcfc3ee ci: enforce 40% iOS coverage 2025-12-14 03:29:08 +00:00
Peter Steinberger
eec6212cdf test(ios): add smoke coverage tests 2025-12-14 03:29:08 +00:00
Peter Steinberger
a5b3b8743a docs: recommend git repo for workspace backups 2025-12-14 03:19:02 +00:00
Peter Steinberger
4dd9072a2b chore(ios): gitignore provisioning profiles 2025-12-14 03:16:50 +00:00
Peter Steinberger
073285409b feat: bootstrap agent workspace and AGENTS.md 2025-12-14 03:14:58 +00:00
Peter Steinberger
41da61dd6a fix(android): make settings sheet scrollable 2025-12-14 03:13:36 +00:00
Peter Steinberger
35e8dae939 fix(android): inset top buttons for status bar 2025-12-14 03:10:46 +00:00
Peter Steinberger
05e77b69c4 ci: emit swift + iOS coverage 2025-12-14 03:07:43 +00:00
Peter Steinberger
745eefe0be test(macos): cover settings + activity models 2025-12-14 03:06:12 +00:00
Peter Steinberger
d7165b4720 feat(ios): add always-on status overlay 2025-12-14 03:00:55 +00:00
Peter Steinberger
7b1163f75c fix(ios): satisfy Sendable in bridge timeout 2025-12-14 03:00:55 +00:00
Peter Steinberger
507f5623f4 fix: expand reply cwd (~) and document AGENTS 2025-12-14 03:00:18 +00:00
Peter Steinberger
5ace7c9c66 test(macos): add settings view smoke coverage 2025-12-14 02:55:31 +00:00
Peter Steinberger
3b35b762cb fix(macos): avoid health polling in tests 2025-12-14 02:55:31 +00:00
Peter Steinberger
dbd3865e3b test(ios): cover settings host/port parsing 2025-12-14 02:47:07 +00:00
Peter Steinberger
6bf1e6fa06 test(ios): cover voice trigger + camera clamps 2025-12-14 02:47:06 +00:00
Peter Steinberger
1c0170554e fix(ios): timeout bridge connect 2025-12-14 02:41:51 +00:00
Peter Steinberger
1d79254053 ci: run iOS xcodebuild tests 2025-12-14 02:37:47 +00:00
Peter Steinberger
974ab5a8dd test(ios): add bridge session + keychain suites 2025-12-14 02:37:47 +00:00
Peter Steinberger
eaebf4b896 chore(android): update toolchain and deps 2025-12-14 02:37:47 +00:00
Peter Steinberger
cf747e1b82 chore(deps): bump pnpm dependencies 2025-12-14 02:37:47 +00:00
Peter Steinberger
455fe15bd1 Merge remote-tracking branch 'origin/main' 2025-12-14 02:37:13 +00:00
Peter Steinberger
c4d0eb9350 fix(ios): make fastlane beta lane work 2025-12-14 02:35:59 +00:00
Peter Steinberger
10d95348b1 fix(ios): make fastlane beta lane work 2025-12-14 02:35:35 +00:00
Peter Steinberger
f86b1cf6a1 fix(camera): modernize mp4 export 2025-12-14 02:34:22 +00:00
Peter Steinberger
259e9cfccf chore(mac): bump Peekaboo submodule 2025-12-14 02:31:31 +00:00
Peter Steinberger
7318b20f55 chore(fastlane): support p8 key path 2025-12-14 02:20:25 +00:00
Peter Steinberger
322a36f365 chore(fastlane): support p8 key path 2025-12-14 02:19:51 +00:00
Peter Steinberger
b8b20eac6d fix(ios): make connection badge visible 2025-12-14 02:19:20 +00:00
Peter Steinberger
1fb123d701 Merge remote-tracking branch 'origin/main' into tmp/ios-statusicon 2025-12-14 02:18:09 +00:00
Peter Steinberger
138f4bd850 fix(ios): show connection status badge 2025-12-14 02:17:54 +00:00
Peter Steinberger
20abf31093 test(ios): share scheme and add deep link tests 2025-12-14 02:17:44 +00:00
Peter Steinberger
4abc551f9e chore(android): bump AGP to 8.6.1 2025-12-14 02:16:46 +00:00
Peter Steinberger
67707763f7 docs(android): expand node README 2025-12-14 02:14:52 +00:00
Peter Steinberger
df8915cf5c test(android): add bridge unit tests 2025-12-14 02:14:05 +00:00
Peter Steinberger
a1d16c61ec feat(ios): add fastlane setup 2025-12-14 02:10:31 +00:00
Peter Steinberger
64b5eb8279 test(ios): add unit test target 2025-12-14 02:05:50 +00:00
Peter Steinberger
c66122c255 fix(ios): set CFBundleIconName 2025-12-14 02:05:44 +00:00
Peter Steinberger
b792175ec5 feat(android): keep node connected via foreground service 2025-12-14 02:01:56 +00:00
Peter Steinberger
b944bee121 chore(mac): sync vendored Peekaboo 2025-12-14 02:00:55 +00:00
Peter Steinberger
88ff2f79d5 test(macos): cover camera snap defaults 2025-12-14 02:00:48 +00:00
Peter Steinberger
c3fa1fb736 feat(camera): share jpeg transcoder + default maxWidth 2025-12-14 02:00:48 +00:00
Peter Steinberger
e9eb9edc23 fix(ios): remove white border from app icon 2025-12-14 01:58:35 +00:00
Peter Steinberger
e8018d8008 feat(macos): add OpenAI Whisper tool 2025-12-14 01:57:12 +00:00
Peter Steinberger
694a10f604 fix(web): use heartbeat inbound msg for delivery 2025-12-14 01:55:40 +00:00
Peter Steinberger
b2378c01ea feat(android): add Compose node app (bridge+canvas+chat+camera) 2025-12-14 01:55:40 +00:00
Peter Steinberger
e2451484d9 feat(ios): unify manual bridge config and auto-reconnect 2025-12-14 01:55:40 +00:00
Peter Steinberger
dccdc950bf feat(gateway): add bridge RPC chat history and push 2025-12-14 01:55:40 +00:00
Peter Steinberger
dd7be2bfd8 feat(macos): refresh tools roster 2025-12-14 01:54:10 +00:00
Peter Steinberger
66b05163e3 fix(ios): ensure app icon asset catalog 2025-12-14 01:50:51 +00:00
Peter Steinberger
7789bf6907 chore(mac): sync vendored Peekaboo 2025-12-14 01:47:05 +00:00
Peter Steinberger
8b6abe0151 fix(web): heartbeat fallback after group inbound 2025-12-14 01:26:40 +00:00
Peter Steinberger
25eb40ab31 chore(macos): swiftformat 2025-12-14 01:11:22 +00:00
Peter Steinberger
0336c1fa37 fix(ios): use mac icon + avoid voice wake crash 2025-12-14 01:09:40 +00:00
Peter Steinberger
09541de076 fix(mac): move menu separator below context card 2025-12-14 00:57:34 +00:00
Peter Steinberger
2583fb66cc fix(webchat): stream assistant events and correlate runId 2025-12-14 00:56:06 +00:00
Peter Steinberger
037ea92679 docs(site): update docs nav 2025-12-14 00:55:38 +00:00
Peter Steinberger
38f65e7053 Merge remote-tracking branch 'origin/main' 2025-12-14 00:55:14 +00:00
Peter Steinberger
ebbc416d4b test(cli): cover camera flags 2025-12-14 00:54:49 +00:00
Peter Steinberger
13c4f8da2b Merge remote-tracking branch 'origin/main' 2025-12-14 00:52:57 +00:00
Peter Steinberger
099b8c9fa5 Merge origin/main 2025-12-14 00:52:40 +00:00
Peter Steinberger
1638d32e1c docs: sync telegram + remote summaries 2025-12-14 00:52:37 +00:00
Peter Steinberger
13e1c93c74 docs(site): fix Clawd setup link 2025-12-14 00:52:14 +00:00
Peter Steinberger
affbd48a3f docs(site): refresh footer + agent blurb 2025-12-14 00:50:57 +00:00
Peter Steinberger
00f83ca7af docs(index): update architecture + quickstart 2025-12-14 00:50:41 +00:00
Peter Steinberger
441bd25f90 docs(clawd): update install + session store path 2025-12-14 00:50:26 +00:00
Peter Steinberger
128df57005 docs: refer to session store 2025-12-14 00:50:12 +00:00
Peter Steinberger
a80cd26341 docs: clarify legacy control + sessions path 2025-12-14 00:49:54 +00:00
Peter Steinberger
700212608a docs(remote): clarify ssh tunneling 2025-12-14 00:49:34 +00:00
Peter Steinberger
8fb064ed70 docs(telegram): clarify polling + webhook config 2025-12-14 00:49:18 +00:00
Peter Steinberger
a92eb1f33d feat(camera): add snap/clip capture 2025-12-14 00:48:58 +00:00
Peter Steinberger
2454e67e09 feat(ios): reconnect to last discovered gateway 2025-12-14 00:48:16 +00:00
Peter Steinberger
862a490038 feat(ios): pulse settings indicator 2025-12-14 00:48:09 +00:00
Peter Steinberger
ffc57d5f20 Merge remote-tracking branch 'origin/main' 2025-12-14 00:43:22 +00:00
Peter Steinberger
e96654ced1 docs(site): note fn+F2 on mac 2025-12-14 00:42:53 +00:00
Peter Steinberger
dd763b45e1 chore(ci): sync protocol + swiftformat 2025-12-14 00:36:30 +00:00
Peter Steinberger
2710841801 docs(readme): reflect gateway + companion apps 2025-12-14 00:34:26 +00:00
Peter Steinberger
a9e1eabcbd docs(changelog): add 2.0.0-beta1 entry 2025-12-14 00:34:22 +00:00
Peter Steinberger
30a2e47390 chore(mac): bump Peekaboo submodule 2025-12-14 00:32:52 +00:00
Peter Steinberger
f7076c38ea feat(ios): reconnect to last bridge 2025-12-14 00:27:26 +00:00
Peter Steinberger
e6d522493b feat(chat): share SwiftUI chat across macOS+iOS 2025-12-14 00:17:07 +00:00
Peter Steinberger
c286573f5c docs(ios): update Iris connect runbook 2025-12-14 00:08:00 +00:00
Peter Steinberger
aef18b7359 fix(gateway): resolve iOS node invokes 2025-12-14 00:00:05 +00:00
Peter Steinberger
17e183f5cf chore(protocol): regen swift models 2025-12-13 23:51:18 +00:00
Peter Steinberger
a53d8ed4e4 feat(instances): show OS version 2025-12-13 23:51:18 +00:00
Peter Steinberger
755e329b01 docs(readme): describe macOS + iOS companion apps 2025-12-13 23:50:23 +00:00
Peter Steinberger
765c466d6d docs(ios): add Iris connection runbook 2025-12-13 23:49:38 +00:00
Peter Steinberger
cf3becfb2e refactor(macos)!: remove clawdis-mac ui; host PeekabooBridge 2025-12-13 23:49:29 +00:00
Peter Steinberger
b508f642b2 iOS: configurable voice wake words 2025-12-13 23:49:22 +00:00
Peter Steinberger
3fcee21ff7 feat(gateway): add node.invoke for iOS canvas 2025-12-13 23:45:16 +00:00
Peter Steinberger
b01cb41950 iOS: copy bridge URL/host/port 2025-12-13 23:40:12 +00:00
Peter Steinberger
7642cbb5b7 iOS: show local IP in settings 2025-12-13 23:37:02 +00:00
Peter Steinberger
7a6334d920 iOS: copy + clean bridge address 2025-12-13 23:32:57 +00:00
Peter Steinberger
d96bc38bea style(macos): mark Reject destructive 2025-12-13 23:32:57 +00:00
Peter Steinberger
a31a569d52 chore(peekaboo): update submodule 2025-12-13 23:22:24 +00:00
Peter Steinberger
0d3aacd316 chore: bump Peekaboo submodule 2025-12-13 23:02:04 +00:00
Peter Steinberger
ece8a3e701 fix(macos): clamp web chat to visible frame 2025-12-13 22:38:10 +00:00
Peter Steinberger
ceb3980b93 iOS: disable VoiceWake on Simulator 2025-12-13 20:52:31 +00:00
Peter Steinberger
01eba1b8d9 chore: bump Peekaboo submodule 2025-12-13 20:46:20 +00:00
Peter Steinberger
cf28ea0d1c test: raise vitest coverage 2025-12-13 20:37:56 +00:00
Peter Steinberger
41dd3b11b7 fix: harden pi package resolution 2025-12-13 20:37:46 +00:00
Peter Steinberger
5a1687484c fix(ci): stabilize runners 2025-12-13 20:04:33 +00:00
Peter Steinberger
6143338116 chore(swift): run swiftformat and clear swiftlint 2025-12-13 19:53:17 +00:00
Peter Steinberger
39c232548c fix(macos): restore control + webchat build 2025-12-13 19:38:35 +00:00
Peter Steinberger
e2a93e17f9 refactor: apply stashed bridge + CLI changes 2025-12-13 19:30:46 +00:00
Peter Steinberger
0b990443de style(macos): tidy settings and CLI 2025-12-13 19:23:41 +00:00
Peter Steinberger
02fe19effa chore(macos): expose remote test helper 2025-12-13 19:22:57 +00:00
Peter Steinberger
920cc9ac38 fix(ios): avoid actor-isolated access from audio tap 2025-12-13 19:14:36 +00:00
Peter Steinberger
ba22890205 feat(browser): add ai snapshot refs + click 2025-12-13 18:48:55 +00:00
Peter Steinberger
a59cfa7670 chore(deps): add playwright-core 2025-12-13 18:48:49 +00:00
Peter Steinberger
7cdd7c5333 fix(browser): apply clawd theme color 2025-12-13 18:41:31 +00:00
Peter Steinberger
e3379b960e chore(peekaboo): update bridge host TeamID check 2025-12-13 18:35:26 +00:00
Peter Steinberger
7b675864a8 feat(browser): add DOM inspection commands 2025-12-13 18:33:04 +00:00
Peter Steinberger
3b853b329f fix(bridge): prefer bonjour TXT displayName 2025-12-13 18:31:06 +00:00
Peter Steinberger
537c515dde fix(macos): show full browser tab ids 2025-12-13 18:17:01 +00:00
Peter Steinberger
238afbc2f8 fix(browser): accept targetId prefixes 2025-12-13 18:17:01 +00:00
Peter Steinberger
2a71c20ee4 fix(mac): place debug menu under Settings 2025-12-13 18:11:00 +00:00
Peter Steinberger
40c66b1741 chore(webchat): refresh bundled assets 2025-12-13 18:10:29 +00:00
Peter Steinberger
94ad808028 fix(mac): clarify attach-only gateway errors 2025-12-13 18:10:29 +00:00
Peter Steinberger
0c8b5ed59a test(mac): cover codesign + node manager paths 2025-12-13 18:10:29 +00:00
Peter Steinberger
56fe23549c feat(browser): clamp screenshots under 5MB 2025-12-13 18:10:29 +00:00
Peter Steinberger
867d7e5d25 chore(peekaboo): bump submodule 2025-12-13 18:09:45 +00:00
Peter Steinberger
a0cd761c96 fix(mac): flatten config sections + use checkboxes 2025-12-13 18:06:32 +00:00
Peter Steinberger
7c3502f031 fix(ios): improve bridge discovery and pairing UX 2025-12-13 17:58:03 +00:00
Peter Steinberger
61ab07ced3 fix(mac): flatten debug sections + use checkboxes 2025-12-13 17:57:45 +00:00
Peter Steinberger
281c6d6069 chore(deps): update JS deps 2025-12-13 17:52:23 +00:00
Peter Steinberger
82634dfe3b fix(mac): add divider below context 2025-12-13 17:51:25 +00:00
Peter Steinberger
9be3394bac fix(cli): improve browser control errors 2025-12-13 17:37:37 +00:00
Peter Steinberger
4228ee326c fix(browser): open tabs via CDP websocket 2025-12-13 17:37:37 +00:00
Peter Steinberger
fa1110e4d3 refactor(mac): reorganize debug settings 2025-12-13 17:36:35 +00:00
Peter Steinberger
050c47d3a7 fix(macos): encode gateway params without AnyHashable 2025-12-13 17:31:11 +00:00
Peter Steinberger
161895ed1a fix(mac): show clawd browser path in config 2025-12-13 17:23:41 +00:00
Peter Steinberger
aeffdc3632 fix(mac): show link cursor in About 2025-12-13 17:18:22 +00:00
Peter Steinberger
ecf0da1796 docs(mac): document clawdis ui passthrough 2025-12-13 17:17:42 +00:00
Peter Steinberger
990fafa988 fix(mac): use pointing hand cursor on tool links 2025-12-13 17:15:31 +00:00
Peter Steinberger
ceb0a8b3e3 fix(macos): surface gateway sessions load errors 2025-12-13 17:15:00 +00:00
Peter Steinberger
3b283f3167 fix(cli): improve ui arg passthrough 2025-12-13 17:12:51 +00:00
Peter Steinberger
2b29d08064 chore(peekaboo): bump submodule 2025-12-13 17:11:14 +00:00
Peter Steinberger
86ed3de1c1 feat(browser): add clawdis-mac browser controls 2025-12-13 17:05:58 +00:00
Peter Steinberger
acf035d848 fix(mac): align config tab padding 2025-12-13 17:00:44 +00:00
Peter Steinberger
cab71c9711 fix(mac): polish config + cron layouts 2025-12-13 16:59:25 +00:00
Peter Steinberger
c17440f5b4 feat(mac): host PeekabooBridge for ui 2025-12-13 16:56:22 +00:00
Peter Steinberger
fd566bda14 chore(submodule): add Peekaboo 2025-12-13 16:56:22 +00:00
Peter Steinberger
e47dccbe87 chore(webchat): refresh webchat bundle 2025-12-13 16:48:53 +00:00
Peter Steinberger
2a172f9779 fix(mac): expand config settings width 2025-12-13 16:48:36 +00:00
Peter Steinberger
ce630a6381 feat(webchat): polish SwiftUI chat 2025-12-13 16:45:35 +00:00
Peter Steinberger
a882798143 fix(mac): hide empty MCP servers section 2025-12-13 16:44:43 +00:00
Peter Steinberger
44f9327087 test(gateway): extend sessions RPC coverage 2025-12-13 16:36:09 +00:00
Peter Steinberger
e654676148 docs(session): note gateway session source of truth 2025-12-13 16:33:22 +00:00
Peter Steinberger
840e266b5d feat(macos): load sessions via gateway 2025-12-13 16:33:14 +00:00
Peter Steinberger
7d89fa2591 feat(gateway): add sessions list/patch RPC 2025-12-13 16:32:42 +00:00
Peter Steinberger
5f67c023a2 docs(clawdis-mac): improve help for browser control 2025-12-13 16:26:48 +00:00
Peter Steinberger
af3e5b299c feat(clawdis-mac): add browser subcommand 2025-12-13 16:26:48 +00:00
Peter Steinberger
b3b4013637 feat(mac): restructure config settings grid 2025-12-13 16:26:48 +00:00
Peter Steinberger
9ad341d668 feat(mac): add browser control menu toggle 2025-12-13 16:26:48 +00:00
Peter Steinberger
d7a8d9a1c7 fix(browser): default control url uses 18791 2025-12-13 16:26:48 +00:00
Peter Steinberger
2d36ae6326 fix(browser): derive cdp port from control url 2025-12-13 16:26:48 +00:00
Peter Steinberger
208ba02a4a feat(browser): add clawd browser control 2025-12-13 16:26:48 +00:00
Peter Steinberger
4cdb21c5cd docs: pixel lobster terminal theme 2025-12-13 16:23:15 +00:00
Peter Steinberger
5d6cc8125b test(telegram): cover inbound media download 2025-12-13 16:18:48 +00:00
Peter Steinberger
237933069e fix(telegram): download inbound media via file_path 2025-12-13 16:18:44 +00:00
Peter Steinberger
99660db73f fix(macos): prevent menubar menu width jump 2025-12-13 15:50:57 +00:00
Peter Steinberger
68fa676cbf chore(webchat): refresh bundled webchat 2025-12-13 14:19:42 +00:00
Peter Steinberger
8794f002d5 Merge remote-tracking branch 'origin/main' 2025-12-13 14:15:48 +00:00
Peter Steinberger
d52ef185b1 fix(macos): make status lines non-selectable 2025-12-13 13:59:53 +00:00
Peter Steinberger
3ca77c46c7 fix(ui): improve light-mode green for context bar 2025-12-13 13:55:16 +00:00
Peter Steinberger
89d5d807ee docs: pi-only terminology 2025-12-13 13:26:44 +00:00
Peter Steinberger
9e3427a37e docs(readme): pi-only wording 2025-12-13 13:26:44 +00:00
Peter Steinberger
7ce25ecfca docs(site): refresh clawdis.ai for Pi 2025-12-13 13:26:44 +00:00
Peter Steinberger
1ca77bee26 chore(ios): rename app to Clawdis 2025-12-13 13:11:31 +00:00
Peter Steinberger
5dbc7cc68d feat(onboarding): highlight voice wake, panel, and tools 2025-12-13 13:04:41 +00:00
Peter Steinberger
0d45c78917 fix(onboarding): drop finish footer line 2025-12-13 13:02:03 +00:00
Peter Steinberger
31fb4f7c8b fix(macos): install gateway via npm 2025-12-13 13:00:59 +00:00
Peter Steinberger
e9acb6fad5 fix(ui): align SSH target discovery row 2025-12-13 12:58:00 +00:00
Peter Steinberger
ab402e1178 docs(onboarding): explain primary gateway and remotes 2025-12-13 12:55:09 +00:00
Peter Steinberger
293701f520 fix(onboarding): tighten welcome copy and raise nav 2025-12-13 12:50:30 +00:00
Peter Steinberger
7b38ba0e65 refactor(cron): drop auto-migration 2025-12-13 12:45:02 +00:00
Peter Steinberger
5d8ee8fc28 docs(cron): update store + run log paths 2025-12-13 12:38:12 +00:00
Peter Steinberger
3e2e4be680 refactor(cron): move store into ~/.clawdis/cron 2025-12-13 12:38:08 +00:00
Peter Steinberger
3863fe6412 fix(ios): stabilize voice wake + bridge UI 2025-12-13 12:29:39 +00:00
Peter Steinberger
2b71ea21ad fix(gateway): advertise bonjour hostname 2025-12-13 12:29:39 +00:00
Peter Steinberger
36f21c5a4f feat!(mac): move screenshot to ui 2025-12-13 12:29:39 +00:00
Peter Steinberger
cf90bd9c86 feat(macos): manage cron jobs 2025-12-13 12:09:27 +00:00
Peter Steinberger
5f159c43c5 feat(cli): expand cron commands 2025-12-13 12:09:20 +00:00
Peter Steinberger
c02613e15f feat(cron): post isolated summaries 2025-12-13 12:09:15 +00:00
Peter Steinberger
32cd1175fb refactor(cron): simplify main-summary prefix config 2025-12-13 11:43:18 +00:00
Peter Steinberger
0152e053e1 feat!(mac): add ui screens + text clawdis-mac 2025-12-13 11:42:42 +00:00
Peter Steinberger
8d1e73edc7 feat(cron): always post isolated summaries to main 2025-12-13 11:33:46 +00:00
Peter Steinberger
a5f51eadf1 macOS: add onboarding security notice 2025-12-13 11:23:46 +00:00
Peter Steinberger
4ac21a4f63 docs(onboarding): explain WhatsApp + Telegram setup 2025-12-13 11:19:54 +00:00
Peter Steinberger
91fdf2aa25 macOS: align context padding 2025-12-13 11:16:33 +00:00
Peter Steinberger
44614d4a7d Merge remote-tracking branch 'origin/main' 2025-12-13 11:14:56 +00:00
Peter Steinberger
0e9f617667 macOS: align sessions list with header 2025-12-13 11:14:50 +00:00
Peter Steinberger
7e7e348a14 fix(bonjour): normalize hostnames for beacons 2025-12-13 11:14:05 +00:00
Peter Steinberger
cc3d0d1ef7 Merge remote-tracking branch 'origin/main' 2025-12-13 11:11:32 +00:00
Peter Steinberger
5b608718bb test(clawdiskit): cover BonjourEscapes decoding 2025-12-13 11:10:30 +00:00
Peter Steinberger
3a6ab81549 fix(ui): increase onboarding horizontal padding 2025-12-13 11:10:22 +00:00
Peter Steinberger
c48681b2f0 Merge remote-tracking branch 'origin/main' 2025-12-13 11:04:31 +00:00
Peter Steinberger
86d786cbc0 macOS: increase context card row spacing 2025-12-13 11:04:11 +00:00
Peter Steinberger
ec653b7b80 chore: share bonjour escapes + refresh webchat bundle 2025-12-13 10:59:48 +00:00
Peter Steinberger
cbc34e1c8a fix(ui): show bonjour masters inline 2025-12-13 10:48:25 +00:00
Peter Steinberger
1f37d94f9e feat(discovery): bonjour beacons + bridge presence 2025-12-13 04:28:43 +00:00
Peter Steinberger
3ee0e041fa Merge remote-tracking branch 'origin/main' 2025-12-13 04:01:20 +00:00
Peter Steinberger
4074f4fffa macOS: adjust context card padding 2025-12-13 04:00:48 +00:00
Peter Steinberger
7286fd6e3f feat(macos): add master discovery to onboarding 2025-12-13 04:00:25 +00:00
Peter Steinberger
4b608117a2 fix(discovery): lazy-load bonjour; add tests 2025-12-13 03:55:36 +00:00
Peter Steinberger
47b4d245aa test(cron): cover default-enabled scheduling 2025-12-13 03:54:21 +00:00
Peter Steinberger
36ff508fec macOS: stabilize context menu card layout 2025-12-13 03:52:09 +00:00
Peter Steinberger
772b5fdf0f feat(cron): default scheduler enabled 2025-12-13 03:49:42 +00:00
Peter Steinberger
eace21dcae feat(discovery): gateway bonjour + node pairing bridge 2025-12-13 03:47:53 +00:00
Peter Steinberger
163080b609 test(cron): cover disabled scheduler 2025-12-13 03:43:55 +00:00
Peter Steinberger
4938fbffa8 feat(macos): show cron scheduler status 2025-12-13 03:43:51 +00:00
Peter Steinberger
d5db20c296 feat(cli): add cron status + warn when disabled 2025-12-13 03:43:47 +00:00
Peter Steinberger
415cb857d9 feat(cron): add scheduler status endpoint 2025-12-13 03:43:40 +00:00
Peter Steinberger
a641250da6 macOS: prewarm context menu card 2025-12-13 03:42:36 +00:00
Peter Steinberger
4d674a3f17 macOS: compact context menu context rows 2025-12-13 03:30:50 +00:00
Peter Steinberger
12d9a13af0 fix(mac): preserve SwiftUI menu delegate 2025-12-13 03:11:06 +00:00
Peter Steinberger
164841f299 refactor(mac): inject context card as NSMenuItem view 2025-12-13 03:03:08 +00:00
Peter Steinberger
778361686c macOS: widen settings window 2025-12-13 03:00:35 +00:00
Peter Steinberger
29907a4c3f docs(mac): drop screenshot alias plan 2025-12-13 02:51:48 +00:00
Peter Steinberger
81f38342bf Merge remote-tracking branch 'origin/main' 2025-12-13 02:50:57 +00:00
Peter Steinberger
36b93c8dc7 security(macos): require TeamID for control socket 2025-12-13 02:50:20 +00:00
Peter Steinberger
e95fdbbc37 fix(ios): prettify bonjour endpoint labels 2025-12-13 02:48:06 +00:00
Peter Steinberger
3001f115b6 fix(mac): keep context row labels together 2025-12-13 02:47:39 +00:00
Peter Steinberger
21649d81d2 fix(presence): report bridged iOS nodes 2025-12-13 02:35:35 +00:00
Peter Steinberger
5118ba3dd2 macOS: add Cron settings tab 2025-12-13 02:34:38 +00:00
Peter Steinberger
f9409cbe43 Cron: add scheduler, wakeups, and run history 2025-12-13 02:34:38 +00:00
Peter Steinberger
572d17f46b feat(mac): tighten context session row 2025-12-13 02:34:37 +00:00
Peter Steinberger
f466f1bf46 feat(mac): compact context session rows 2025-12-13 02:34:37 +00:00
Peter Steinberger
594315d90b ui(ios): glassy settings button 2025-12-13 02:19:34 +00:00
Peter Steinberger
f84895f1f1 fix(ios): make canvas full-bleed 2025-12-13 02:15:03 +00:00
Peter Steinberger
952810f76c docs(agents): forbid git stash without consent 2025-12-13 02:08:18 +00:00
Peter Steinberger
73ccbedcdb ui(ios): clean up connected bridge list 2025-12-13 02:02:38 +00:00
Peter Steinberger
7ef83311bb feat(bridge): show node ip in pairing 2025-12-13 01:57:40 +00:00
Peter Steinberger
416c376077 feat(ios): add close button and ready canvas 2025-12-13 01:49:04 +00:00
Peter Steinberger
ef83a07066 fix(macos): harden remote ssh tunnel 2025-12-13 01:43:23 +00:00
Peter Steinberger
ae0c1573fd refactor(swift): rename ClawdisNodeKit to ClawdisKit 2025-12-13 01:33:30 +00:00
Peter Steinberger
378e5acd23 feat(deeplink): forward agent links via bridge 2025-12-13 01:19:36 +00:00
Peter Steinberger
a56daa6c06 feat(macos): add Allow Canvas toggle to settings 2025-12-13 01:19:36 +00:00
Peter Steinberger
84399e62ae fix(mac): render context sessions card with labels 2025-12-13 01:18:42 +00:00
Peter Steinberger
387615e99f feat(mac): show session labels under context bars 2025-12-13 01:10:17 +00:00
Peter Steinberger
f98ab2d037 fix(macos): prevent control socket hangs 2025-12-13 01:02:47 +00:00
Peter Steinberger
19ce08b4d0 fix(mac): avoid collapsed context pills in menu 2025-12-13 00:51:05 +00:00
Peter Steinberger
8cc2dc715c refactor(ios): minimal full-screen canvas 2025-12-13 00:50:20 +00:00
Peter Steinberger
ca20a2dc06 Merge remote-tracking branch 'origin/main' 2025-12-13 00:48:01 +00:00
Peter Steinberger
f9b1a96c89 chore(macos): move Permissions tab after Tools 2025-12-13 00:47:08 +00:00
Peter Steinberger
854f07d735 feat(mac): compact context sessions in menu 2025-12-13 00:39:25 +00:00
Peter Steinberger
7f4f01009b refactor(ios): remove manual URL controls 2025-12-13 00:31:52 +00:00
Peter Steinberger
117b01acbd fix(ios): avoid MainActor isolation in audio tap 2025-12-13 00:27:15 +00:00
Peter Steinberger
2b38ddf78d fix(ios): avoid actor isolation in audio tap 2025-12-13 00:27:15 +00:00
Peter Steinberger
5e51107711 fix(mac): size context bar to menu 2025-12-13 00:23:00 +00:00
Peter Steinberger
3bb33bdeed fix(mac): render context bar as image 2025-12-13 00:19:29 +00:00
Peter Steinberger
9b9fa009d1 fix(mac): render context bar reliably 2025-12-13 00:13:33 +00:00
Peter Steinberger
072ad8d371 fix(mac): show cached context usage 2025-12-12 23:44:55 +00:00
Peter Steinberger
8846ffec64 fix: expose heartbeat controls and harden mac CLI 2025-12-12 23:34:26 +00:00
Peter Steinberger
3b72ed6e1a feat(macos): add clawdis://agent deep link 2025-12-12 23:33:38 +00:00
Peter Steinberger
35b7c0f558 feat(mac): show context usage bars 2025-12-12 23:33:15 +00:00
Peter Steinberger
d5d80f4247 feat(gateway)!: switch handshake to req:connect (protocol v2) 2025-12-12 23:29:57 +00:00
Peter Steinberger
e915ed182d fix(macos): clarify presence update source label 2025-12-12 23:27:08 +00:00
Peter Steinberger
c3aed2543e fix(status): account cached prompt tokens 2025-12-12 23:22:24 +00:00
Peter Steinberger
e502ad13f9 fix(node): prevent iOS VoiceWake crash 2025-12-12 23:07:30 +00:00
Peter Steinberger
952d924581 fix(mac): recover control tunnel after restart
# Conflicts:
#	apps/macos/Sources/Clawdis/GatewayConnection.swift
2025-12-12 23:07:30 +00:00
Peter Steinberger
0484aba892 test(web): retry session tmp cleanup 2025-12-12 22:55:39 +00:00
Peter Steinberger
03c84d0f11 fix(mac): make Canvas file watcher reliable 2025-12-12 22:50:25 +00:00
Peter Steinberger
086f98471e docs: finalize gateway refactor notes 2025-12-12 22:27:18 +00:00
Peter Steinberger
cc4f0d8acc test(macos): cover gateway endpoint store 2025-12-12 22:27:18 +00:00
Peter Steinberger
c7bd4b5c1d refactor(macos): extract gateway payload decoding 2025-12-12 22:27:18 +00:00
Peter Steinberger
14e3b34a8e refactor(macos): centralize gateway endpoint resolution 2025-12-12 22:27:18 +00:00
Peter Steinberger
6354dddff2 fix(macos): avoid ptt audio teardown race 2025-12-12 22:24:24 +00:00
Peter Steinberger
c50c3699d9 fix(macos): keep voice wake overlay on top 2025-12-12 22:09:14 +00:00
Peter Steinberger
6a7f955818 refactor(macos): replace gateway NotificationCenter with event bus 2025-12-12 22:06:40 +00:00
Peter Steinberger
9cf457be0a fix(bridge): use default Bonjour domain 2025-12-12 21:59:04 +00:00
Peter Steinberger
e31383a8f1 fix(ios): harden voice wake callbacks 2025-12-12 21:59:04 +00:00
Peter Steinberger
13b8dc61ba fix(mac): timeout ClawdisCLI socket calls 2025-12-12 21:57:33 +00:00
Peter Steinberger
61085f6141 fix(macos): avoid external open for about:blank 2025-12-12 21:56:54 +00:00
Peter Steinberger
d8cb1daa78 test(macos): cover gateway connection reuse 2025-12-12 21:42:16 +00:00
Peter Steinberger
de2e341947 fix(mac): avoid double-trigger voice wake 2025-12-12 21:37:59 +00:00
Peter Steinberger
e944a0239d fix(macos): share gateway websocket connection 2025-12-12 21:35:00 +00:00
Peter Steinberger
ce8db12b22 fix(mac): keep voice overlay above canvas 2025-12-12 21:26:04 +00:00
Peter Steinberger
1d41129b6c feat(ios): add settings UI 2025-12-12 21:19:39 +00:00
Peter Steinberger
6d6c3ad2c4 feat(ios): add ClawdisNode app scaffold 2025-12-12 21:19:39 +00:00
Peter Steinberger
0b532579d8 feat(bridge): add Bonjour node bridge 2025-12-12 21:19:39 +00:00
Peter Steinberger
b9007dc721 feat(mac): add rolling diagnostics log 2025-12-12 21:19:39 +00:00
Peter Steinberger
211efffa10 fix(gateway): treat webchat last as whatsapp 2025-12-12 21:05:39 +00:00
Peter Steinberger
e3b50b7d12 fix(macos): show tool-use badge glyph 2025-12-12 21:02:38 +00:00
Peter Steinberger
aae49f1d68 fix(gateway): don"t let webchat clobber last route 2025-12-12 21:00:33 +00:00
Peter Steinberger
6b4141247e feat(macos): enlarge tool-use badge 2025-12-12 20:45:51 +00:00
Peter Steinberger
327f6e7e25 fix(mac): persist Canvas frame across reopen 2025-12-12 20:33:40 +00:00
Peter Steinberger
296c0a6b70 feat(mac): allow Canvas placement and resizing 2025-12-12 20:28:19 +00:00
Peter Steinberger
356b6e0483 fix(mac): keep voice wake listening 2025-12-12 20:13:41 +00:00
Peter Steinberger
08a473fb35 fix(mac): keep Canvas below Voice Wake overlay 2025-12-12 20:10:29 +00:00
Peter Steinberger
893eef846d fix(mac): add draggable/closable Canvas hover chrome 2025-12-12 20:08:15 +00:00
Peter Steinberger
4ecd35c275 fix(mac): render Canvas HTML correctly 2025-12-12 20:01:12 +00:00
Peter Steinberger
27a7d9f9d1 feat(mac): add agent-controlled Canvas panel 2025-12-12 19:54:01 +00:00
Peter Steinberger
c0abab226d Merge remote-tracking branch 'origin/main' 2025-12-12 19:28:10 +00:00
Peter Steinberger
f1320b79ce feat(mac): add overlay notification delivery 2025-12-12 19:27:38 +00:00
Peter Steinberger
bf41197b97 fix(mac): open settings for microphone permission 2025-12-12 19:25:21 +00:00
Peter Steinberger
3f7fcad9ac fix(mac): ignore cancelled webchat navigations 2025-12-12 19:20:47 +00:00
Peter Steinberger
e2ad0ed9f7 fix(mac): disable restricted time-sensitive entitlement 2025-12-12 19:20:47 +00:00
Peter Steinberger
d2158966db fix(mac): treat timeSensitive as best-effort 2025-12-12 18:58:07 +00:00
Peter Steinberger
8086c66ab8 fix(mac): keep remote control tunnel alive 2025-12-12 18:44:44 +00:00
Peter Steinberger
7d37195c1a fix(mac): serve webchat locally in remote mode 2025-12-12 18:41:38 +00:00
Peter Steinberger
241cf10bdb refactor(mac): embed work badge in status icon 2025-12-12 18:40:33 +00:00
Peter Steinberger
337ae05ed8 build(mac): enable time-sensitive notifications 2025-12-12 18:40:09 +00:00
Peter Steinberger
378e39d7ad test(cli): verify gateway exits 0 on SIGTERM 2025-12-12 18:30:19 +00:00
Peter Steinberger
8fb3aef917 fix(gateway): handle SIGTERM shutdown cleanly 2025-12-12 18:28:08 +00:00
Peter Steinberger
c86cb4e9a5 macOS: add --priority flag for time-sensitive notifications
Add NotificationPriority enum with passive/active/timeSensitive levels
that map to UNNotificationInterruptionLevel. timeSensitive breaks
through Focus modes for urgent notifications.

Usage: clawdis-mac notify --title X --body Y --priority timeSensitive
2025-12-12 18:27:12 +00:00
Peter Steinberger
8ca240fb2c fix(gateway): ignore stale lastTo for voice 2025-12-12 18:11:26 +00:00
Peter Steinberger
9ea697ac09 style(test): biome format 2025-12-12 18:07:33 +00:00
Peter Steinberger
37eaa49e4c fix(mac): allow typing in web chat panel 2025-12-12 18:07:27 +00:00
Peter Steinberger
e64ca7c583 fix(agent): send tau rpc prompt as string 2025-12-12 18:04:13 +00:00
Peter Steinberger
62a7a07127 fix(gateway): ack agent requests immediately 2025-12-12 18:00:49 +00:00
Peter Steinberger
bc618ec290 refactor(auto-reply): remove pi json fallback 2025-12-12 17:43:11 +00:00
Peter Steinberger
0780859a4d fix(auto-reply): prefer Pi RPC by default 2025-12-12 17:30:34 +00:00
Peter Steinberger
79818f73c0 fix(mac): harden gateway frame decoding 2025-12-12 17:30:21 +00:00
Peter Steinberger
957d7fbe2a test(voice): cover gateway last-channel whatsapp 2025-12-12 17:29:04 +00:00
Peter Steinberger
6e9d3092a7 fix(voice): persist WhatsApp last route 2025-12-12 17:28:07 +00:00
Peter Steinberger
7dab927260 fix(presence): hide cli sessions; use numeric mac build 2025-12-12 17:27:11 +00:00
Peter Steinberger
c417517f43 fix(mac): reflect agent activity in menu icon 2025-12-12 17:20:06 +00:00
Peter Steinberger
fd0314a6bd fix(mac): avoid static UserDefaults in InstanceIdentity 2025-12-12 16:59:51 +00:00
Peter Steinberger
6a05d60f41 fix(presence): dedupe instances via stable instanceId 2025-12-12 16:57:25 +00:00
Peter Steinberger
cd84c5ad08 fix(macos): prevent gateway request double-resume 2025-12-12 16:52:36 +00:00
Peter Steinberger
b0187d7f28 Merge remote-tracking branch 'origin/main' 2025-12-12 16:47:24 +00:00
Peter Steinberger
7a1d64fff9 style(tests): format imports 2025-12-12 16:47:10 +00:00
Peter Steinberger
debcf19199 fix(presence): stabilize instance identity 2025-12-12 16:47:07 +00:00
Peter Steinberger
ea76425f86 docs(readme): describe voice wake reply routing 2025-12-12 16:42:09 +00:00
Peter Steinberger
88936b6216 fix(macos): fix clawdis-mac --version 2025-12-12 16:40:50 +00:00
Peter Steinberger
e6edcd9a7f Merge remote-tracking branch 'origin/main' 2025-12-12 16:39:27 +00:00
Peter Steinberger
af78762421 style(mac): hud glass voice overlay 2025-12-12 16:39:11 +00:00
Peter Steinberger
bf159bd316 fix(mac): prevent crash decoding GatewayFrame 2025-12-12 16:37:59 +00:00
Peter Steinberger
9eda40234f test: cover main last-channel routing 2025-12-12 16:35:47 +00:00
Peter Steinberger
00336f554f docs: clarify voice wake last-channel routing 2025-12-12 16:26:19 +00:00
Peter Steinberger
a524b9ae9b feat(voicewake): route replies to last channel 2025-12-12 16:22:30 +00:00
Peter Steinberger
3f1bcac077 Merge remote-tracking branch 'origin/main' 2025-12-12 16:10:02 +00:00
Peter Steinberger
679ced7840 mac: remove voice wake forward pref 2025-12-12 16:09:31 +00:00
Peter Steinberger
7422f54212 mac: add gog CLI, remove Gmail/Calendar MCPs
- Add gog (unified Google CLI for Gmail, Calendar, Drive, Contacts)
- Remove Gmail MCP and Google Calendar MCP entries (replaced by gog)
- gog installs via brew: steipete/tap/gog
2025-12-12 15:48:36 +00:00
Peter Steinberger
b0384d0335 fix(mac): cache webchat panel 2025-12-12 15:33:41 +00:00
Peter Steinberger
6b64039fcb fix(mac): keep webchat boot dots 2025-12-12 15:01:20 +00:00
Peter Steinberger
19e7c708ce test(mac): cover concurrent gateway connect 2025-12-12 14:29:09 +00:00
Peter Steinberger
c8ca5803fc fix(mac): webchat ws connect 2025-12-12 14:18:53 +00:00
Peter Steinberger
5f48abb451 fix(mac): serialize gateway connect 2025-12-12 14:14:33 +00:00
Peter Steinberger
491fd6b74d mac: lock control socket to team-signed peers 2025-12-12 01:22:24 +00:00
Peter Steinberger
f1ff24d634 web: default to self-only without config 2025-12-12 01:22:03 +00:00
Peter Steinberger
0242383ec3 test(gateway): cover port lock guard 2025-12-11 18:53:40 +00:00
Peter Steinberger
768887fc0f style(pi): wrap mode arg lookup 2025-12-11 18:53:34 +00:00
Peter Steinberger
958c13e02d mac: replace xpc with unix socket control channel 2025-12-11 16:31:15 +01:00
Peter Steinberger
f417b51fb6 chore(gateway): use ws bind as lock 2025-12-11 15:17:40 +00:00
Peter Steinberger
47a1f757a9 lint: format and stabilize gateway health 2025-12-10 18:00:33 +00:00
Peter Steinberger
27ad3b917f chore(gateway): log pre-hello ws closures 2025-12-10 16:58:56 +00:00
Peter Steinberger
93a5784c58 feat(gateway): allow webchat port override 2025-12-10 16:55:17 +00:00
Peter Steinberger
e9fd73141d health: gateway-only status and stable reconnect 2025-12-10 16:47:38 +00:00
Peter Steinberger
6c005b3d35 fix(session): ignore agent meta session id 2025-12-10 16:38:22 +00:00
Peter Steinberger
2967bc5988 health: stop direct baileys probes 2025-12-10 16:35:42 +00:00
Peter Steinberger
55772eec5a gateway: force ws-only clients 2025-12-10 16:27:54 +00:00
Peter Steinberger
c2adda1cfe chore: drop rpc->json fallback 2025-12-10 15:58:45 +00:00
Peter Steinberger
51d77aea2e fix(auto-reply): acknowledge reset triggers 2025-12-10 15:55:20 +00:00
Peter Steinberger
8f456ea73b fix(agent): send structured prompt to tau rpc 2025-12-10 15:52:39 +00:00
Peter Steinberger
6459582952 gateway: add webchat handshake logging 2025-12-10 15:32:34 +00:00
Peter Steinberger
3796882d22 webchat: improve logging and static serving 2025-12-10 15:32:29 +00:00
Peter Steinberger
4db69c8eac fix(auto-reply): fall back to json when rpc prompt empty 2025-12-10 14:58:03 +00:00
Peter Steinberger
f6a86e5527 telegram: fix verbose log ordering 2025-12-10 14:33:09 +00:00
Peter Steinberger
063b35f1dc mac: surface gateway auth failures 2025-12-10 14:32:54 +00:00
Peter Steinberger
b61147aed0 fix(auto-reply): guard empty rpc prompt 2025-12-10 14:26:03 +00:00
Peter Steinberger
fd3516bc82 fix(pi): skip -p when running rpc 2025-12-10 14:21:38 +00:00
Peter Steinberger
5e4a91f996 Auto-reply: reject empty inbound messages 2025-12-10 13:51:06 +00:00
Peter Steinberger
df4331da04 gateway: dedupe system-event presence 2025-12-10 11:48:17 +00:00
Peter Steinberger
fe3a983d35 mac: include instance id in presence beacons 2025-12-10 11:48:13 +00:00
Peter Steinberger
53c349cb86 RPC: auto-cancel hook UI prompts 2025-12-10 11:46:28 +00:00
Peter Steinberger
8f37f15a33 RPC: handle tau auto-compaction retries 2025-12-10 11:40:32 +00:00
Peter Steinberger
81385cf820 pi: parse turn_end streams 2025-12-10 11:31:28 +00:00
Peter Steinberger
cce65e19e1 mac: add attach-only gateway toggle 2025-12-10 11:31:28 +00:00
Peter Steinberger
c4b02645f5 fix: persist usage from rpc 2025-12-10 11:31:28 +00:00
Peter Steinberger
49e70746f0 webchat: show real ws errors 2025-12-10 11:31:28 +00:00
Peter Steinberger
00ace3bb63 test: add semver and gateway helpers coverage 2025-12-10 11:31:28 +00:00
Peter Steinberger
efde37eb36 test: add gateway/runtime utility coverage 2025-12-10 11:31:28 +00:00
Peter Steinberger
84499ab969 mac: drop yarn fallback 2025-12-10 03:49:25 +01:00
Peter Steinberger
5d26bb2566 gateway: include last input in presence events 2025-12-10 03:48:53 +01:00
Peter Steinberger
657450c40c fix(voice): unify overlay send flow 2025-12-10 02:52:42 +01:00
Peter Steinberger
cf2b659491 mac: simplify package manager picker 2025-12-10 02:49:39 +01:00
Peter Steinberger
e9679ce993 chore(mac): align remote ssh controls 2025-12-10 02:48:46 +01:00
Peter Steinberger
68c5d61d60 mac: move debug toggles to footer 2025-12-10 02:48:19 +01:00
Peter Steinberger
c4f0236ec0 mac: inline gateway status row 2025-12-10 02:46:59 +01:00
Peter Steinberger
1839c144fa mac: remove divider above active toggle 2025-12-10 02:44:56 +01:00
Peter Steinberger
d077936a21 mac: align web chat UI with web 2025-12-10 02:18:50 +01:00
Peter Steinberger
6c1638890c chore(test): document force run and relax coverage scope 2025-12-10 01:06:44 +00:00
Peter Steinberger
7f0f789953 webchat: add centered boot loader 2025-12-10 01:04:34 +00:00
Peter Steinberger
83a2a7a1c2 mac: add swiftui web chat option 2025-12-10 02:03:59 +01:00
Peter Steinberger
70fb4d452e mac: tidy menu and gateway support 2025-12-10 01:00:53 +00:00
Peter Steinberger
5ed1d4e178 test: drop obsolete reply session placeholder 2025-12-10 01:00:44 +00:00
Peter Steinberger
35834d3dba webchat: handle bind errors gracefully 2025-12-10 01:00:34 +00:00
Peter Steinberger
260d9b9770 test: add test:force helper 2025-12-10 01:00:29 +00:00
Peter Steinberger
3907e9eedd test: isolate gateway lock per run 2025-12-10 00:58:59 +00:00
Peter Steinberger
cf8b00890f fix: stabilize health probe and gateway handshake 2025-12-10 00:52:43 +00:00
Peter Steinberger
f1fd25e95e chore: update dependencies 2025-12-10 00:48:50 +00:00
Peter Steinberger
426503e062 infra: use flock gateway lock 2025-12-10 00:46:50 +00:00
Peter Steinberger
b1834b7cf8 mac: avoid spawning local gateway in remote mode 2025-12-10 01:44:03 +01:00
Peter Steinberger
27f9cd591d mac: route remote mode through SSH 2025-12-10 01:43:59 +01:00
Peter Steinberger
5bbc7c8ba2 mac: silence proc_pidpath warning 2025-12-10 01:43:34 +01:00
Peter Steinberger
08f8f58971 mac: add browser webchat debug entry 2025-12-10 01:33:15 +01:00
Peter Steinberger
7871e705bf mac: show full command and kill controls for ports 2025-12-10 01:24:05 +01:00
Peter Steinberger
1820308ba2 fix: expand gateway attach log 2025-12-10 00:19:18 +00:00
Peter Steinberger
a07229846f mac: treat pnpm/bun processes as expected gateways 2025-12-10 01:10:50 +01:00
Peter Steinberger
a7e4656834 mac: drop legacy log path 2025-12-10 00:05:05 +00:00
Peter Steinberger
872d54a2dd mac: guard ports and sweep stale tunnels 2025-12-10 01:04:37 +01:00
Peter Steinberger
496136b52c style(webchat): add body padding class on error 2025-12-10 00:04:22 +00:00
Peter Steinberger
c4eff00ed7 mac: centralize log path lookup 2025-12-10 00:03:37 +00:00
Peter Steinberger
27d8aa0f04 style(webchat): pad error view 2025-12-10 00:02:51 +00:00
Peter Steinberger
bb057b1dad fix: keep tools list stable 2025-12-10 00:02:18 +00:00
Peter Steinberger
3b9d84e2b1 mac: global outside-click monitor and highlight helper 2025-12-10 00:51:02 +01:00
Peter Steinberger
1a17de9d39 fix(webchat): serve root assets correctly 2025-12-09 23:50:28 +00:00
Peter Steinberger
f6ade5dc84 mac: add port diagnostics for gateway 2025-12-10 00:49:33 +01:00
Peter Steinberger
2116f19106 fix(mac): keep overlay on token mismatch 2025-12-10 00:48:15 +01:00
Peter Steinberger
b73a7e07d2 mac: open latest log file 2025-12-09 23:45:50 +00:00
Peter Steinberger
14d3a624d8 fix(webchat): load root path 2025-12-09 23:40:26 +00:00
Peter Steinberger
dd88345483 gateway: cache health snapshot 2025-12-09 23:39:02 +00:00
Peter Steinberger
e58d5a54b1 mac: toggle panel purely from visibility 2025-12-09 23:36:51 +01:00
Peter Steinberger
2a95a5bf8a Add package manager selector and hide uninstalled tools 2025-12-09 22:32:20 +00:00
Peter Steinberger
0c4e67a951 mac: ensure panel toggle doesn't reopen 2025-12-09 23:32:01 +01:00
Peter Steinberger
78d41b8e41 test: cover chat attachments 2025-12-09 23:31:14 +01:00
Peter Steinberger
d5347176e1 mac: close panel on second click 2025-12-09 23:25:49 +01:00
Peter Steinberger
6d91dad8e4 mac: tie highlight to panel visibility 2025-12-09 23:20:16 +01:00
Peter Steinberger
1dd5c97ae0 feat: add ws chat attachments 2025-12-09 23:16:57 +01:00
Peter Steinberger
e80e5b0801 mac: revert webchat menu fallback 2025-12-09 23:15:35 +01:00
Peter Steinberger
052d8ba879 fix(macos): harden presence decode 2025-12-09 22:08:55 +00:00
Peter Steinberger
d08ca9585a mac: clear status highlight via menu delegate 2025-12-09 23:02:02 +01:00
Peter Steinberger
50c33dfcdf chore: bump pi deps for tau rpc 2025-12-09 21:53:00 +00:00
Peter Steinberger
42c3c2b804 fix: prevent stuck mac health checks 2025-12-09 21:53:00 +00:00
Peter Steinberger
f83eeac5e2 fix(mac): keep webchat panel alive 2025-12-09 21:53:00 +00:00
Peter Steinberger
d5517ede45 mac: clear highlight on panel close 2025-12-09 22:40:11 +01:00
Peter Steinberger
2339f1a01d chore(mac): add separator before general toggles 2025-12-09 21:28:46 +00:00
Peter Steinberger
6129924eb2 chore: remove legacy rpc command 2025-12-09 21:28:39 +00:00
Peter Steinberger
fce9ded30a feat(webchat): sync theme with system 2025-12-09 21:22:21 +00:00
Peter Steinberger
8489907cf5 feat(telegram): add typing cue 2025-12-09 21:14:10 +00:00
Peter Steinberger
84ccde268e mac/webchat: remove panel padding 2025-12-09 21:14:10 +00:00
Peter Steinberger
c191df5434 fix: relaunch app after debug restart 2025-12-09 22:13:43 +01:00
Peter Steinberger
f49934a75b mac: respect webchat disabled for left click 2025-12-09 22:11:10 +01:00
Peter Steinberger
be3326d0d9 chore(webchat): log url on gateway start 2025-12-09 21:10:49 +00:00
Peter Steinberger
7919019b67 fix(mac): disable smoothing and await watchdog 2025-12-09 22:09:25 +01:00
Peter Steinberger
89d856a487 fix(mac): snap critter drawing to pixels 2025-12-09 22:08:21 +01:00
Peter Steinberger
978a24ffab fix(mac): keep ptt overlay until release 2025-12-09 22:08:17 +01:00
Peter Steinberger
bd41cf377a feat(webchat): auto-start at root 2025-12-09 21:07:53 +00:00
Peter Steinberger
3ee3f7e30b mac: add gateway reconnect watchdog 2025-12-09 21:07:39 +00:00
Peter Steinberger
a032614dc7 mac: make status rows disabled menu items 2025-12-09 22:02:15 +01:00
Peter Steinberger
0377d13d3d mac: disable status rows in menu 2025-12-09 21:59:17 +01:00
Peter Steinberger
06fdfc2e14 mac icon: render 36px retina backing 2025-12-09 21:56:37 +01:00
Peter Steinberger
510552c5e6 mac: harden webchat panel 2025-12-09 21:43:54 +01:00
Peter Steinberger
6675c273fd mac: panel highlight when webchat open 2025-12-09 21:41:24 +01:00
Peter Steinberger
9131a69983 Debug menu: add sessions icon and separator 2025-12-09 21:40:04 +01:00
Peter Steinberger
a8baf0ef45 chore(gateway): color ws direction logs 2025-12-09 20:37:01 +00:00
Peter Steinberger
5e4f32d808 chore(mac): include os version and locale in handshake 2025-12-09 20:37:01 +00:00
Peter Steinberger
5a8d18edf3 web: reuse active listener for sends 2025-12-09 20:37:01 +00:00
Peter Steinberger
f34b238713 Debug menu: session controls and thinking/verbose 2025-12-09 21:32:21 +01:00
Peter Steinberger
ad5c7d97ca mac: left-click webchat panel 2025-12-09 21:29:21 +01:00
Peter Steinberger
c35f9c1315 docs: refresh gateway cli params 2025-12-09 20:28:10 +00:00
Peter Steinberger
e84ed61339 cli: gateway subcommands, drop ipc probes 2025-12-09 20:27:35 +00:00
Peter Steinberger
8265829105 Menu: add icons to debug submenu 2025-12-09 21:24:36 +01:00
Peter Steinberger
a76d00a08e chore: drop gateway ipc remnants 2025-12-09 20:21:41 +00:00
Peter Steinberger
131864b940 gateway: drop ipc and simplify cli 2025-12-09 20:18:50 +00:00
Peter Steinberger
d33a3f619a fix(mac): harden gateway lock and ip decoding 2025-12-09 20:12:54 +00:00
Peter Steinberger
1a0e57d926 Menu: add more debug utilities 2025-12-09 21:11:28 +01:00
Peter Steinberger
5e5845547e gateway: improve conflict handling and logging 2025-12-09 20:07:24 +00:00
Peter Steinberger
0de944be28 telegram: show name and id in envelope 2025-12-09 19:56:18 +00:00
Peter Steinberger
5df438fd2a fix: enforce gateway single instance 2025-12-09 19:40:01 +00:00
Peter Steinberger
6329f60dff chore(mac): add divider before session toggles 2025-12-09 19:14:01 +00:00
Peter Steinberger
0bf9a87293 chore(mac): dedupe local gateway label 2025-12-09 19:13:46 +00:00
Peter Steinberger
6ae4c49c1a fix(mac): encode gateway params with protocol AnyCodable 2025-12-09 19:10:19 +00:00
Peter Steinberger
c683ae69af gateway: log provider errors verbosely 2025-12-09 19:10:10 +00:00
Peter Steinberger
ab9b12e883 gateway: enforce hello order and modern json 2025-12-09 19:09:06 +00:00
Peter Steinberger
c41b506741 mac: fix gateway hello types 2025-12-09 19:02:53 +00:00
Peter Steinberger
848180dc08 mac: fix local path string 2025-12-09 19:02:53 +00:00
Peter Steinberger
a7d39913fd mac: fix actor call and label warnings 2025-12-09 19:02:53 +00:00
Peter Steinberger
85ca2152e4 feat(mac): reuse running gateway 2025-12-09 19:02:53 +00:00
Peter Steinberger
b11b33b63c test(overlay): cover token guard outcomes 2025-12-09 19:51:51 +01:00
Peter Steinberger
239f58b584 fix(overlay): dismiss on token mismatch; keep gateway log clear helper 2025-12-09 19:50:05 +01:00
Peter Steinberger
474cb48a14 fix(ptt): dismiss empty overlay immediately on key up 2025-12-09 19:48:35 +01:00
Peter Steinberger
577b0dfe1d mac: show local gateway path when overridden 2025-12-09 18:46:31 +00:00
Peter Steinberger
2918e00d33 fix(mac): restore gateway clear log 2025-12-09 18:44:22 +00:00
Peter Steinberger
ffc930b871 surface: envelope inbound messages for agent 2025-12-09 18:43:21 +00:00
Peter Steinberger
55bffeba4a chore: add gateway env/process manager after rename 2025-12-09 19:38:19 +01:00
Peter Steinberger
2adb14c320 fix: improve app restart and gateway logs 2025-12-09 18:37:04 +00:00
Peter Steinberger
0d4bf1c15a fix(ptt): ignore stale recognition callbacks 2025-12-09 19:17:16 +01:00
Peter Steinberger
a3bf2bdd8c chore: rename relay to gateway 2025-12-09 18:00:01 +00:00
Peter Steinberger
bc3a14cde2 docs: add docs:list helper and front matter 2025-12-09 17:51:05 +00:00
Peter Steinberger
b3d4e5cfdf mac: simplify degraded labels 2025-12-09 17:45:27 +00:00
Peter Steinberger
885355ce53 settings: clarify pause toggles gateway messaging 2025-12-09 17:40:59 +00:00
Peter Steinberger
a4d5b68134 mac: honor local relay path 2025-12-09 17:40:44 +00:00
Peter Steinberger
67f2bc1385 web: log disconnect error detail in reconnect loop 2025-12-09 17:38:49 +00:00
Peter Steinberger
d8fb2f9175 chore(mac): make package/restart skip ts relay 2025-12-09 17:36:24 +00:00
Peter Steinberger
fcc8d59588 fix(mac): avoid crash decoding gateway frames 2025-12-09 17:36:16 +00:00
Peter Steinberger
1f19ca1665 chore: drop runner shim and add committer helper 2025-12-09 17:24:25 +00:00
Peter Steinberger
d04f7fc6e9 msg: retry web/telegram sends and add regression tests 2025-12-09 17:23:04 +00:00
Peter Steinberger
f9370718bc web: show surface + host/ip chips in chat UI 2025-12-09 17:23:00 +00:00
Peter Steinberger
8d888b426f chore: format swift/ts and fix gateway lint 2025-12-09 17:11:25 +00:00
Peter Steinberger
b6bd39660f IPC: rename relay socket to gateway.sock 2025-12-09 17:04:58 +00:00
Peter Steinberger
959ba94eca macOS: add settings previews 2025-12-09 18:04:11 +01:00
Peter Steinberger
d5cd1058ab Mac: surface gateway errors in remote test 2025-12-09 18:01:15 +01:00
Peter Steinberger
80c7b04831 Menu: add debug submenu actions 2025-12-09 17:57:21 +01:00
Peter Steinberger
7017756140 UI: unify refresh buttons 2025-12-09 17:54:12 +01:00
Peter Steinberger
d9a132b649 chore: update dependencies 2025-12-09 17:43:22 +01:00
Peter Steinberger
60a68aa136 Gateway: start providers and route sends to their surface 2025-12-09 16:38:43 +00:00
Peter Steinberger
464e4c1938 Gateway: honor verbose for Baileys and show log path 2025-12-09 16:33:04 +00:00
Peter Steinberger
796f630a7c Status: color provider lines 2025-12-09 16:31:38 +00:00
Peter Steinberger
dc8f9e043d Tests: cover gateway --force helpers 2025-12-09 16:31:28 +00:00
Peter Steinberger
6afcf43ff2 CLI: add gateway --force option 2025-12-09 16:28:26 +00:00
Peter Steinberger
e0ea7be499 Docs: rename relay command to gateway 2025-12-09 17:24:57 +01:00
Peter Steinberger
4bf968a45a CLI: add gateway verbose flag 2025-12-09 17:17:58 +01:00
Peter Steinberger
a86963d62d Debug: rename restart button to Gateway 2025-12-09 16:16:14 +00:00
Peter Steinberger
e40f9c9730 Mac: launch gateway and add relay installer 2025-12-09 16:15:53 +00:00
Peter Steinberger
96be7c8990 tests: cover agent sequencing, tick watchdog, presence fingerprint 2025-12-09 17:05:47 +01:00
Peter Steinberger
3ced3f4c82 ci/docs: enforce protocol check and deprecate control api 2025-12-09 17:03:05 +01:00
Peter Steinberger
72eb240c3b gateway: harden ws protocol and liveness 2025-12-09 17:02:58 +01:00
Peter Steinberger
20d247b3f7 Mac: type agent events end-to-end 2025-12-09 15:38:22 +01:00
Peter Steinberger
318457cb2c chore(swabble): apply swiftformat 2025-12-09 15:36:41 +01:00
Peter Steinberger
336c9d6caa Mac: build GatewayProtocol target and typed presence handling 2025-12-09 15:35:06 +01:00
Peter Steinberger
a7737912b0 Mac: use typed GatewayFrame + forward-compatible Swift generator 2025-12-09 15:26:31 +01:00
Peter Steinberger
f244aba03d Protocol: legacy shim file for Xcode references 2025-12-09 15:23:51 +01:00
Peter Steinberger
b0c196cf82 Protocol: add TypeBox-driven Swift generator 2025-12-09 15:21:16 +01:00
Peter Steinberger
cf5769753a Protocol: lint fixes for client/program 2025-12-09 15:18:34 +01:00
Peter Steinberger
d1217e84c7 CLI: remove relay/heartbeat legacy commands 2025-12-09 15:06:44 +01:00
Peter Steinberger
172ce6c79f Gateway: discriminated protocol schema + CLI updates 2025-12-09 15:01:13 +01:00
Peter Steinberger
2746efeb25 WebChat: loopback snapshot hydration 2025-12-09 14:41:55 +01:00
Peter Steinberger
b2e7fb01a9 Gateway: finalize WS control plane 2025-12-09 14:41:41 +01:00
Peter Steinberger
9ef1545d06 Coordinator: centralize voice sessions for wake and push-to-talk 2025-12-09 05:41:41 +01:00
Peter Steinberger
fc1d58b631 WebChat: fix packaged root resolution 2025-12-09 04:36:15 +00:00
Peter Steinberger
2ebad55a59 Relay: force app to run relay via system node 2025-12-09 04:36:05 +00:00
Peter Steinberger
d66a05dc41 RPC: route logs to stderr to keep stdout JSON clean 2025-12-09 04:30:22 +00:00
Peter Steinberger
998a5b080d Update auto-reply and voice wake runtime 2025-12-09 04:15:01 +00:00
Peter Steinberger
39a0f54b0d Runtime: drop bun support 2025-12-09 04:13:56 +00:00
Peter Steinberger
024a823c78 Runtime: delay restart inside actor; log RPC unexpected payload 2025-12-09 05:02:56 +01:00
Peter Steinberger
1bbb424322 Overlay: block new sessions while sending; delay runtime restart 2025-12-09 05:02:03 +01:00
Peter Steinberger
b04f04776b fix(mac): make rpc parsing tolerate stray stdout 2025-12-09 05:01:50 +01:00
Peter Steinberger
f0860ec145 chore(instances): harden presence refresh and fix lint 2025-12-09 04:51:54 +01:00
Peter Steinberger
658e0c6b03 Presence: resilient local fallback 2025-12-09 04:48:21 +01:00
Peter Steinberger
49fa093767 Overlay: log token drops and immediate auto-send 2025-12-09 04:47:05 +01:00
Peter Steinberger
51aed3ca0a chore(mac): apply swiftformat and lint fixes 2025-12-09 04:42:44 +01:00
Peter Steinberger
b9cc914729 Docs: clarify relay launch mechanism 2025-12-09 03:36:16 +00:00
Peter Steinberger
d084a37e11 feat(mac): tokenized voice overlay adoption 2025-12-09 04:35:13 +01:00
Peter Steinberger
cfd2c41c21 fix(rpc): keep stdout json-only 2025-12-09 04:34:11 +01:00
Peter Steinberger
9dee4c158d chore(instances): log empty payloads and add local fallback 2025-12-09 04:29:34 +01:00
Peter Steinberger
6b8011228e fix(presence): always seed self entry and log counts 2025-12-09 03:21:59 +00:00
Peter Steinberger
2cd27d0d4a Relay: enforce single instance lock 2025-12-09 03:17:23 +00:00
Peter Steinberger
3dff09424d VoiceWake: drop unused forward health check state 2025-12-09 03:12:37 +00:00
Peter Steinberger
8e15a6e798 Overlay: safety dismiss and logging; keep PTT final send 2025-12-09 04:04:45 +01:00
Peter Steinberger
2756e12762 VoiceWake: drop remote ssh config and harden template parsing 2025-12-09 03:04:08 +00:00
Peter Steinberger
4eb71bcd14 rpc: ensure worker is killed if it hangs on shutdown 2025-12-09 03:04:00 +00:00
Peter Steinberger
2177df51a8 feat(status): enrich session details 2025-12-09 03:00:10 +00:00
Peter Steinberger
40c8e4832a WebChat: make tunnel restart handler hop to MainActor 2025-12-09 03:58:28 +01:00
Peter Steinberger
3377bd4ae5 PTT: wait for final transcript before send/dismiss 2025-12-09 03:57:08 +01:00
Peter Steinberger
38c4f4f76c feat(instances): beacon on connect and relay self-entry 2025-12-09 03:57:08 +01:00
Peter Steinberger
280c7c851f tests: cover voicewake template defaults 2025-12-09 02:52:04 +00:00
Peter Steinberger
af9ccf0c09 VoiceWake: route forwarding via agent rpc 2025-12-09 02:50:58 +00:00
Peter Steinberger
e7cdac90f5 mac: stop leaking ssh processes on quit 2025-12-09 02:50:58 +00:00
Peter Steinberger
7aefcab8b0 Health: clean degraded message; PTT hotkey monitors 2025-12-09 03:46:52 +01:00
Peter Steinberger
514b90ac69 VoiceWake: autoplay chime on selection 2025-12-09 03:42:03 +01:00
Peter Steinberger
dbcb97949f macOS: centralize sound effect catalog/player 2025-12-09 03:42:03 +01:00
Peter Steinberger
76d559efc1 macOS: log control responses 2025-12-09 02:41:18 +00:00
Peter Steinberger
8d8584849c RPC: fix presence imports 2025-12-09 02:39:41 +00:00
Peter Steinberger
59a2cbefcb RPC: extract stdio loop and tests 2025-12-09 02:37:04 +00:00
Peter Steinberger
c568284f1b Build: fix RPC sendable params and CLI imports 2025-12-09 03:33:16 +01:00
Peter Steinberger
a8b26570e0 macOS: include mail sounds in chime picker 2025-12-09 03:28:29 +01:00
Peter Steinberger
5a74b40ae4 macOS: broaden chime sound catalog 2025-12-09 03:27:17 +01:00
Peter Steinberger
04f595cd97 Control: route health/heartbeat over RPC stdio 2025-12-09 02:26:08 +00:00
Peter Steinberger
99a3102134 Docs: voice overlay plan and fix web mocks 2025-12-09 03:25:55 +01:00
Peter Steinberger
3a42979e53 Voice wake: log overlay lifecycle and enforce PTT cooldown 2025-12-09 03:20:52 +01:00
Peter Steinberger
912a53318e fix(voicewake): snap overlay to top-right 2025-12-09 03:18:05 +01:00
Peter Steinberger
421401ae3f Voice wake: drop stale recognition callbacks 2025-12-09 03:08:22 +01:00
Peter Steinberger
e15475449c fix merge; add control logging 2025-12-09 01:46:09 +00:00
Peter Steinberger
31750b5ee5 style(macos): remove quit separator and resize settings 2025-12-09 02:28:05 +01:00
Peter Steinberger
bc92f6d4a4 feat(macos): add instances tab and presence beacons 2025-12-09 02:25:45 +01:00
Peter Steinberger
1969e78d54 feat: surface system presence for the agent 2025-12-09 02:25:37 +01:00
Peter Steinberger
317f666d4c Voice wake: send or dismiss on release 2025-12-09 02:25:06 +01:00
Peter Steinberger
3fe68a051a fix: block partial replies on external chat surfaces 2025-12-09 01:48:12 +01:00
Peter Steinberger
5bfecc6152 fix: stop partial replies for whatsapp/telegram surfaces 2025-12-09 01:41:05 +01:00
Peter Steinberger
e44ed2681f refactor: type tau rpc stream events 2025-12-09 01:41:05 +01:00
Peter Steinberger
27a545f79d chore: harden rpc assistant streaming types 2025-12-09 01:41:05 +01:00
Peter Steinberger
6b10f4241d feat(macos): surface session activity in menu bar 2025-12-09 01:41:05 +01:00
Peter Steinberger
73cc34467a control: log incoming health requests 2025-12-09 00:38:42 +00:00
Peter Steinberger
ec1ff52dfb control: reconnect on EOF and relax rpc text parse 2025-12-09 00:29:31 +00:00
Peter Steinberger
2761c40781 test: ensure tool events emit without verbose 2025-12-09 01:24:16 +01:00
Peter Steinberger
e981d90209 fix: always emit tool events 2025-12-09 01:22:50 +01:00
Peter Steinberger
f965e1c3ff chore: single-source working state from agent events 2025-12-09 01:17:01 +01:00
Peter Steinberger
5b5a79b90b chore(mac): drop duplicate job-state tracking 2025-12-09 01:06:46 +01:00
Peter Steinberger
15729e9ea0 macos: log health timeout and control requests 2025-12-09 00:00:50 +00:00
Peter Steinberger
d9eb320bba ci: test node and bun runtimes 2025-12-09 01:00:35 +01:00
Peter Steinberger
cba016df74 chore(mac): prefer host runtime for remote relay 2025-12-09 00:59:56 +01:00
Peter Steinberger
cf36f5a23b chore: guard host runtime and simplify packaging 2025-12-09 00:59:56 +01:00
Peter Steinberger
34d2527606 chore: tidy agent event streaming types 2025-12-09 00:59:56 +01:00
Peter Steinberger
8e8e695db9 feat(mac): add agent events debug window 2025-12-09 00:59:56 +01:00
Peter Steinberger
9928f1b3c1 macOS: extract attributed string helper 2025-12-09 00:59:56 +01:00
Peter Steinberger
36c91c3984 relay: don't crash when webchat port is busy 2025-12-08 23:49:57 +00:00
Peter Steinberger
b7b1714f32 feat: forward tool/assistant events to agent bus 2025-12-09 00:44:30 +01:00
Peter Steinberger
2d1f1640f3 chore: ignore macOS swiftpm cache 2025-12-09 00:43:45 +01:00
Peter Steinberger
371a30f08b feat: stream tool/job events over control channel 2025-12-09 00:31:39 +01:00
Peter Steinberger
40dd23337c feat: broadcast agent events over control channel 2025-12-09 00:28:03 +01:00
Peter Steinberger
3114dfd39b refactor(mac): split menubar UI into smaller files 2025-12-09 00:27:53 +01:00
Peter Steinberger
04b34adec6 macos: show detailed health failure 2025-12-08 23:20:14 +00:00
Peter Steinberger
594e837440 feat: emit job-state events from rpc 2025-12-09 00:18:14 +01:00
Peter Steinberger
c77fa12bda fix(mac): stabilize voice wake visuals 2025-12-09 00:12:43 +01:00
Peter Steinberger
5674c9f4c2 Mac: clarify runtime comments 2025-12-09 00:08:19 +01:00
Peter Steinberger
bc01488a75 fix(mac): switch push-to-talk to right option 2025-12-08 23:50:31 +01:00
Peter Steinberger
c3c6880382 macos: timeout control health probes 2025-12-08 22:45:58 +00:00
Peter Steinberger
1f2f5858c0 docs: note Mac app for relay debugging 2025-12-08 23:37:46 +01:00
Peter Steinberger
22259a322d macos: keep remote control tunnel alive 2025-12-08 23:28:03 +01:00
Peter Steinberger
06f59f4e8a Build: update webchat bundle 2025-12-08 23:20:10 +01:00
Peter Steinberger
2b7adeb220 VoiceWake: track listening state for PTT 2025-12-08 23:17:11 +01:00
Peter Steinberger
05bd452f76 control: drop runtime export of type-only HeartbeatEventPayload 2025-12-08 23:15:33 +01:00
Peter Steinberger
a6426d0ac5 macos: swap bubble shadow for 1px border 2025-12-08 23:14:00 +01:00
Peter Steinberger
5dd5c9c605 macos: add inset margin so overlay shadow isn't clipped 2025-12-08 22:56:49 +01:00
Peter Steinberger
0e4b28ac25 macos: fail fast when SSH tunnel exits 2025-12-08 22:53:40 +01:00
Peter Steinberger
62fecdcaa8 VoiceWake: guard trigger chime 2025-12-08 22:52:51 +01:00
Peter Steinberger
440558c44f macos: add soft shadow behind overlay bubble 2025-12-08 22:51:04 +01:00
Peter Steinberger
fa9a92f214 macos: deepen shadow on close pill 2025-12-08 22:45:40 +01:00
Peter Steinberger
c5af11f6bd Remove overlay bar meter 2025-12-08 22:45:40 +01:00
Peter Steinberger
ad3254deb6 macos: restore overlay close button 2025-12-08 21:40:18 +00:00
Peter Steinberger
fce04b9424 macos: stabilize close hover and unclipped button 2025-12-08 22:38:51 +01:00
Peter Steinberger
2d512c714b VoiceWake: button meter + fix label color 2025-12-08 22:38:30 +01:00
Peter Steinberger
6298c586fd macos: stabilize control connection wait 2025-12-08 21:37:07 +00:00
Peter Steinberger
abca8535cf macos: blink critter when overlay dismisses empty 2025-12-08 22:34:11 +01:00
Peter Steinberger
677374de86 macos: sync ears with overlay visibility 2025-12-08 22:31:03 +01:00
Peter Steinberger
92d015333a VoiceWake: add level meter 2025-12-08 22:28:49 +01:00
Peter Steinberger
6c91304400 macos: refine speech noise floor tracking 2025-12-08 22:24:12 +01:00
Peter Steinberger
04b5002d8f macos: polish voice overlay and remote command handling 2025-12-08 22:23:24 +01:00
Peter Steinberger
9bde7a6daa macos: harden control channel connect continuation 2025-12-08 22:16:05 +01:00
Peter Steinberger
33b54f3d0c ux: float close button outside bubble, stronger shadow 2025-12-08 22:11:38 +01:00
Peter Steinberger
c5b073702c macos: control channel diagnostics and tunnel-based testing 2025-12-08 22:04:02 +01:00
Peter Steinberger
e38bdd0d2d control: seed events, add tests, update remote doc 2025-12-08 22:03:46 +01:00
Peter Steinberger
9c54e48194 fix: avoid auto-send task init error 2025-12-08 22:02:03 +01:00
Peter Steinberger
12e048a7fb ux: float close button outside bubble and reduce hover flicker 2025-12-08 21:59:05 +01:00
Peter Steinberger
11400e43dc chore: sync webchat bundle and voice wake settings 2025-12-08 21:51:08 +01:00
Peter Steinberger
293b4960f3 macos: use control channel for health and heartbeat 2025-12-08 21:50:51 +01:00
Peter Steinberger
22996854f7 relay: add control channel and heartbeat stream 2025-12-08 21:50:24 +01:00
Peter Steinberger
71e58c768c docs: add control channel reference 2025-12-08 21:50:16 +01:00
Peter Steinberger
bb3606b64f VoiceWake: centralize send chime and guard play 2025-12-08 21:25:30 +01:00
Peter Steinberger
7a82777fc5 ux: add hover/ edit close button and keep overlay until escape or send 2025-12-08 21:22:04 +01:00
Peter Steinberger
ec046411f1 VoiceWake: skip send chime when nothing to send 2025-12-08 20:57:41 +01:00
Peter Steinberger
ffaf968940 VoiceWake: streamline chimes, default to Glass 2025-12-08 20:50:34 +01:00
Peter Steinberger
feb70aeb6b VoiceWake: add chimes for trigger and send 2025-12-08 20:45:05 +01:00
Peter Steinberger
ded106b9e3 ux: keep window in edit, add escape to cancel; fix lint drift 2025-12-08 20:22:56 +01:00
Peter Steinberger
cfdcabc8b4 VoiceWake: sanitize triggers only when applying 2025-12-08 20:20:56 +01:00
Peter Steinberger
ab448988ff RPC: stream heartbeat events to menu 2025-12-08 20:18:54 +01:00
Peter Steinberger
e3089d60ea HeartbeatStore: fix main-actor cleanup 2025-12-08 20:17:38 +01:00
Peter Steinberger
34f892ae82 VoiceWake: keep empty trigger rows 2025-12-08 20:13:49 +01:00
Peter Steinberger
fbbf0ed41c ux: top-align overlay content 2025-12-08 20:10:39 +01:00
Peter Steinberger
66a8780fa2 ui: strip label color attributes so text uses primary color 2025-12-08 20:00:36 +01:00
Peter Steinberger
2c610258d1 ux: use primary text color in display label 2025-12-08 19:57:29 +01:00
Peter Steinberger
f7430d74a7 ux: wrap label to overlay width, remove label background 2025-12-08 19:43:07 +01:00
Peter Steinberger
421d6db592 ux: keep vibrancy, brighten label, ensure wrapping 2025-12-08 19:36:48 +01:00
Peter Steinberger
1d385fd35a ui: drop translucency for overlay background 2025-12-08 19:20:46 +01:00
Peter Steinberger
7cb31581d5 ux: brighten display label and wrap properly 2025-12-08 19:15:58 +01:00
Peter Steinberger
768d550ee2 ux: show vibrant label until edit, then switch to text view 2025-12-08 19:11:59 +01:00
Peter Steinberger
4fd7480557 chore: launch app in restart script instead of launch agent 2025-12-08 19:01:29 +01:00
Peter Steinberger
7c0f0a59eb tweak: strengthen partial transcript tint 2025-12-08 18:54:02 +01:00
Peter Steinberger
93aeee1611 tweak: centralize overlay max/min heights 2025-12-08 18:52:19 +01:00
Peter Steinberger
86d9e1e816 fix: hide overlay scrollbar unless content overflows 2025-12-08 18:50:14 +01:00
Peter Steinberger
73211c900b perf(mac): move blocking launchctl/webchat work off main 2025-12-08 18:42:13 +01:00
Peter Steinberger
a19d4c19d3 tweak: allow overlay to grow to 400px then scroll 2025-12-08 18:33:14 +01:00
Peter Steinberger
cf3b7f2c16 fix: keep overlay attributed colors and auto-resize 2025-12-08 18:28:17 +01:00
Peter Steinberger
2f21dd81b0 docs/macos: simplify sag install (auto-tap) 2025-12-08 18:19:54 +01:00
Peter Steinberger
db3b3ed9eb fix: polish voice overlay and webchat lint 2025-12-08 17:32:34 +01:00
Peter Steinberger
9625d94aa0 fix(mac): surface webchat load failures and preflight reachability 2025-12-08 17:24:08 +01:00
Peter Steinberger
5dec7d534f docs: document push-to-talk hotkey 2025-12-08 17:24:08 +01:00
Peter Steinberger
0317eec10d feat(mac): add push-to-talk hotkey 2025-12-08 17:24:08 +01:00
Peter Steinberger
a34ab1d36e Webchat: clean server build and add ws types 2025-12-08 16:21:56 +00:00
Peter Steinberger
7144a0fb9b Webchat: push updates over WebSocket 2025-12-08 16:19:33 +00:00
Peter Steinberger
421924b73f fix: restart webchat tunnel on main actor 2025-12-08 17:14:43 +01:00
Peter Steinberger
466236e32f fix(mac): harden remote webchat tunnel and keep it alive 2025-12-08 17:14:43 +01:00
Peter Steinberger
636f2d659f chore: tighten webchat types and formatting 2025-12-08 17:14:43 +01:00
Peter Steinberger
838a9c000c fix: resize overlay on text updates and keep final tint 2025-12-08 17:14:43 +01:00
Peter Steinberger
7a7c59e91a Webchat: poll session for messages/thinking 2025-12-08 16:14:12 +00:00
Peter Steinberger
1ac6ab4428 Agent: add thinkingOnce flag 2025-12-08 16:12:24 +00:00
Peter Steinberger
dc3c82ad40 Webchat: sync thinking level with session 2025-12-08 16:10:14 +00:00
Peter Steinberger
0f0a2dddfe chore: use 5s silence before speech, 2s after 2025-12-08 17:06:12 +01:00
Peter Steinberger
c3f955d3f1 chore: fix lint warnings and formatting 2025-12-08 17:05:27 +01:00
Peter Steinberger
7b1832bd24 chore: extend voice capture hard stop to 120s 2025-12-08 16:58:38 +01:00
Peter Steinberger
148c9533ae chore: use 2s silence or 5s max capture 2025-12-08 16:55:08 +01:00
Peter Steinberger
df96318662 fix(mac): run remote health with pnpm under zsh 2025-12-08 16:52:42 +01:00
Peter Steinberger
d9d0be0256 fix: finalize only after full 1s silence 2025-12-08 16:52:13 +01:00
Peter Steinberger
de70d82cea fix(mac): surface health errors instead of pending 2025-12-08 16:50:20 +01:00
Peter Steinberger
81db44f584 feat: add outcome-based dismiss animations 2025-12-08 16:49:58 +01:00
Peter Steinberger
d733d246f0 chore: remove overlay shadow/border 2025-12-08 16:45:25 +01:00
Peter Steinberger
1c5170b759 fix: animate overlay resizing on updates 2025-12-08 16:44:44 +01:00
Peter Steinberger
367526f750 feat: show partial transcripts with subdued tint 2025-12-08 16:44:00 +01:00
Peter Steinberger
7a0830de15 feat: tint partial transcripts and stabilize delays 2025-12-08 16:41:33 +01:00
Peter Steinberger
a5fbfa3748 fix: delay logic waits for post-trigger content 2025-12-08 16:38:33 +01:00
Peter Steinberger
912a7a1781 test: cover trigger trimming for voice wake 2025-12-08 16:36:53 +01:00
Peter Steinberger
563701fed8 fix: trim overlay transcript to post-trigger 2025-12-08 16:35:03 +01:00
Peter Steinberger
414889e03b feat: add adaptive voice wake delays 2025-12-08 16:34:06 +01:00
Peter Steinberger
8d2de036d5 feat: refine voice wake overlay animations 2025-12-08 16:34:06 +01:00
Peter Steinberger
764761cfa5 feat: add voice wake overlay 2025-12-08 16:34:06 +01:00
Peter Steinberger
90a0bb5acb feat(cli): unify relay providers and heartbeat flag 2025-12-08 16:34:06 +01:00
Peter Steinberger
0e4379f075 Webchat: cap/ persist attachments and strip data URLs 2025-12-08 14:59:26 +00:00
Peter Steinberger
968c5dc4aa Webchat: update bundled assets after attachment support 2025-12-08 14:48:03 +00:00
Peter Steinberger
fedb15d5d0 Webchat: inline attachments to agent RPC and fix status compile 2025-12-08 14:46:33 +00:00
Peter Steinberger
ccc6bf05e8 status: read token usage from pi session logs 2025-12-08 14:46:15 +00:00
Peter Steinberger
a40e56bcb7 Docs: webchat now served in-process, no CLI spawn 2025-12-08 14:15:03 +00:00
Peter Steinberger
52453eaeff Webchat: run agent in-process for RPC 2025-12-08 14:14:00 +00:00
Peter Steinberger
ff3337feed Webchat: resolve static root in packaged app 2025-12-08 14:07:20 +00:00
Peter Steinberger
cd30a99fae feat(macos): add voice wake mic picker 2025-12-08 15:05:57 +01:00
Peter Steinberger
081460e59d macOS webchat: use relay HTTP transport directly 2025-12-08 13:12:34 +00:00
Peter Steinberger
17a6d716ad Webchat: auto-start server and simplify config 2025-12-08 13:12:34 +00:00
Peter Steinberger
d833de793d Split clawdis node vs mac helper commands 2025-12-08 13:26:12 +01:00
Peter Steinberger
a6ff62c79c SSH remote uses clawdis only 2025-12-08 13:20:55 +01:00
Peter Steinberger
92457f7fab Remote web chat tunnel and onboarding polish 2025-12-08 12:50:37 +01:00
Peter Steinberger
17fa2f4053 refactor(cli): drop tmux helpers and update help copy 2025-12-08 12:43:13 +01:00
Peter Steinberger
bce84376d3 webchat: send via http rpc endpoint and show errors 2025-12-08 12:23:45 +01:00
Peter Steinberger
be87cdddeb webchat: surface bootstrap errors in UI 2025-12-08 12:17:39 +01:00
Peter Steinberger
dc22661744 webchat: move serving to relay loopback and tunnel from mac app 2025-12-08 11:54:30 +01:00
Peter Steinberger
dc69d20ec9 docs: outline web chat move to relay server 2025-12-08 11:25:00 +01:00
Peter Steinberger
22ed7ea3f2 build: silence grammy type errors for mac packaging 2025-12-08 11:04:17 +01:00
Peter Steinberger
2112fa919a webchat: fetch remote sessions via CLI and log missing history 2025-12-08 01:55:09 +01:00
Peter Steinberger
f65702a8a8 chore(ci): fix lint and swiftformat failures 2025-12-08 01:48:53 +01:00
Peter Steinberger
68d19d4717 webchat: load remote history from tau fallback and send to session 2025-12-08 01:36:00 +01:00
Peter Steinberger
a6e0ec38e7 VoiceWake: capture utterance and add prefix 2025-12-08 01:35:42 +01:00
Peter Steinberger
6415ae79be webchat: make remote mode load history and send via rpc 2025-12-08 01:27:18 +01:00
Peter Steinberger
79b76fb5f4 ui: drop default sound picker; use cli per-notification sound 2025-12-08 00:56:36 +01:00
Peter Steinberger
42012389c4 health: surface ssh output when probe fails 2025-12-08 00:52:31 +01:00
Peter Steinberger
4b5c43f080 copy: rename menu toggle to Remote Clawdis Active when remote 2025-12-08 00:41:31 +01:00
Peter Steinberger
d16e5090a6 copy: capitalize send heartbeats menu label 2025-12-08 00:40:30 +01:00
Peter Steinberger
ddbe680a58 feat(macos): add Sparkle updates and release docs 2025-12-08 00:18:16 +01:00
Peter Steinberger
2f50b57e76 ui: remove duplicate health row in General 2025-12-08 00:17:29 +01:00
Peter Steinberger
dc291fa811 ui: move Clawdis active toggle to top 2025-12-08 00:16:25 +01:00
Peter Steinberger
a1d499ed64 copy: shorten tailscale tip 2025-12-08 00:14:58 +01:00
Peter Steinberger
629f2e0043 fix: stop voice wake tester after short post-trigger silence 2025-12-07 23:43:50 +01:00
Peter Steinberger
5d321c4dac copy: rename recognition language label 2025-12-07 23:35:58 +01:00
Peter Steinberger
9d751e0c72 ui: place health row under remote picker and improve timeout message 2025-12-07 23:34:49 +01:00
Peter Steinberger
6f8fb561c6 ui: tidy tables, links, and hide redundant voice wake forwarder 2025-12-07 23:26:28 +01:00
Peter Steinberger
1019872832 ui: move health/cli info to Debug; add single health row in General 2025-12-07 23:22:54 +01:00
Peter Steinberger
091471293d ui: fold remote mode label into picker 2025-12-07 23:21:00 +01:00
Peter Steinberger
d7281286ba ui: reuse compact remote card in General and hide voice wake forwarder 2025-12-07 23:20:14 +01:00
Peter Steinberger
5cfda2803d fix: remote test uses CLI path discovery again 2025-12-07 23:12:33 +01:00
Peter Steinberger
9ee7a14685 ui: make General tab scrollable 2025-12-07 23:06:10 +01:00
Peter Steinberger
40a6574b95 ui: align voice wake forwarding with remote mode 2025-12-07 23:04:51 +01:00
Peter Steinberger
891e1388ba style: bump onboarding height to 840px 2025-12-07 22:58:05 +01:00
Peter Steinberger
0fba7d41a6 chore: refresh webchat bundle 2025-12-07 22:57:12 +01:00
Peter Steinberger
1595fb8739 docs: move grammY research note to docs/grammy.md 2025-12-07 22:53:58 +01:00
Peter Steinberger
ebc852b358 chore: update dependencies 2025-12-07 22:53:36 +01:00
Peter Steinberger
5f5846a08b Telegram: enable grammY throttler and webhook tests 2025-12-07 22:52:57 +01:00
Peter Steinberger
4d3d9cca2a Add Bun bundle docs and Telegram grammY support 2025-12-07 22:47:05 +01:00
Peter Steinberger
7b77e9f9ae macOS: surface stderr in health failure text 2025-12-07 21:37:06 +00:00
Peter Steinberger
0f74e372ba MenuBar: fix health label age string 2025-12-07 19:03:49 +01:00
Peter Steinberger
a3b99dc309 Utilities: add age helper for menu health label 2025-12-07 19:02:50 +01:00
Peter Steinberger
d73d571f19 Launch agent: disable autostart without killing running app 2025-12-07 19:01:14 +01:00
Peter Steinberger
8a8ac1ffe6 style: increase onboarding window height 2025-12-07 19:01:14 +01:00
Peter Steinberger
d463c82c95 build: add local node bin to restart script PATH 2025-12-07 19:01:14 +01:00
Peter Steinberger
558af7a454 chore: surface helper install status in onboarding 2025-12-07 19:01:14 +01:00
Peter Steinberger
d57ebb3c94 style: enlarge onboarding window to fit full permission list 2025-12-07 19:01:14 +01:00
Peter Steinberger
855976df84 style: compact remote setup card and move advanced ssh fields 2025-12-07 19:01:14 +01:00
Peter Steinberger
6c2a8d6047 style: increase onboarding content height 2025-12-07 19:01:14 +01:00
Peter Steinberger
38a856f7ff style: tighten onboarding hero spacing 2025-12-07 19:01:14 +01:00
Peter Steinberger
fb2a7d8cd1 VoiceWake: add escaping regression tests 2025-12-07 19:01:14 +01:00
Peter Steinberger
b3f79e5b02 macOS: fix web chat agent PATH and surface stderr 2025-12-07 17:31:14 +00:00
Peter Steinberger
1722148333 macOS: show last health result with age in menu 2025-12-07 17:23:51 +00:00
Peter Steinberger
27e96999cf VoiceWake: document escape path and reset stale forward command 2025-12-07 18:23:34 +01:00
Peter Steinberger
7efa152418 VoiceWake: document escape path and reset stale forward command 2025-12-07 18:23:34 +01:00
Peter Steinberger
2a45455c80 feat: add remote clawd toggle 2025-12-07 18:23:34 +01:00
Peter Steinberger
c06f49cb3e macOS: merge status row and fix webchat bundle deps 2025-12-07 17:20:42 +00:00
Peter Steinberger
b837c68df8 VoiceWake: remove python hop; use escaped literal under /bin/sh 2025-12-07 18:03:25 +01:00
Peter Steinberger
f3ebb2e9ce test(mac): cover voice wake helpers 2025-12-07 17:56:40 +01:00
Peter Steinberger
df9f72134b refactor(mac): split voice wake settings 2025-12-07 17:55:07 +01:00
Peter Steinberger
4ff5004d7c webchat: bypass api key prompts in embedded mode 2025-12-07 17:55:07 +01:00
Peter Steinberger
bdf3d60148 webchat: hide model selector in embedded UI 2025-12-07 17:55:07 +01:00
Peter Steinberger
e2c6546b61 auto-reply: enrich chat status 2025-12-07 16:53:33 +00:00
Peter Steinberger
1f0ee9837b macOS: fix health shell timeout race 2025-12-07 16:53:32 +00:00
Peter Steinberger
71072f084e VoiceWake: send transcript via python/base64 instead of stdin 2025-12-07 17:45:43 +01:00
Peter Steinberger
98651c2a14 webchat: bundle assets with rolldown 2025-12-07 17:44:37 +01:00
Peter Steinberger
74e5e5e182 docs(mac): document privacy-off logging 2025-12-07 17:35:13 +01:00
Peter Steinberger
16f9dbfe37 VoiceWake: include ssh cmd on failure 2025-12-07 17:30:45 +01:00
Peter Steinberger
12f74de9b3 VoiceWake: pipe transcript to ssh forwarder 2025-12-07 16:59:22 +01:00
Peter Steinberger
fec49e1e28 chore(webchat): increase server logging for module load debugging 2025-12-07 16:55:49 +01:00
Peter Steinberger
9dd9bb7092 chore(webchat): add server logging and ensure buildable 2025-12-07 16:49:08 +01:00
Peter Steinberger
9c07aab2d6 voice wake: log ssh command at info level 2025-12-07 16:43:18 +01:00
Peter Steinberger
41a84cef23 chore(webchat): wait for local server and add debug logging 2025-12-07 16:39:21 +01:00
Peter Steinberger
8942e3e78d voice wake: log full ssh command for debug 2025-12-07 16:38:49 +01:00
Peter Steinberger
040fe58693 chore: format macOS sources 2025-12-07 16:35:58 +01:00
Peter Steinberger
45398b7660 voice wake: use clean PATH (no inherited junk) 2025-12-07 16:33:56 +01:00
Peter Steinberger
f3950a5a65 feat(macos): serve web chat over localhost to avoid cors 2025-12-07 16:30:10 +01:00
Peter Steinberger
6f6c5129d1 chore: bump version to 2.0.0 2025-12-07 16:28:57 +01:00
Peter Steinberger
139697b9cd voice wake: keep default key when identity is blank 2025-12-07 16:23:35 +01:00
Peter Steinberger
ddd459426d voice wake: show identity not found when configured 2025-12-07 16:18:42 +01:00
Peter Steinberger
3387c135ad Icon: add ear holes on voice wake 2025-12-07 16:15:40 +01:00
Peter Steinberger
73133b61fb chore(macos): allow file access for web chat modules 2025-12-07 16:14:13 +01:00
Peter Steinberger
ba0f594548 voice wake: surface ssh failures (missing key/no output) 2025-12-07 16:13:40 +01:00
Peter Steinberger
f4fa9bf51a fix(macos): load web chat from bundled html 2025-12-07 16:13:40 +01:00
Peter Steinberger
9aea85a953 General: add bottom inset to quit button 2025-12-07 15:11:47 +00:00
Peter Steinberger
f878e5e635 fix(mac): keep pnpm health output json-safe 2025-12-07 15:09:56 +00:00
Peter Steinberger
4e2fb38d62 debug: hide helper subtext while sending 2025-12-07 15:47:30 +01:00
Peter Steinberger
ee845376b5 rpc: surface raw error lines and auto-start worker 2025-12-07 15:46:26 +01:00
Peter Steinberger
75234da135 Debug: surface detailed voice send errors 2025-12-07 14:41:45 +00:00
Peter Steinberger
7dc9434aec chore(macos): enlarge about icon 2025-12-07 15:34:44 +01:00
Peter Steinberger
5986cf4254 docs: record current rpc protocol and heartbeat toggle 2025-12-07 15:34:02 +01:00
Peter Steinberger
f6db636473 Debug: make voice wake test follow config 2025-12-07 14:33:46 +00:00
Peter Steinberger
b30db08110 feat: add heartbeat toggle with live RPC control 2025-12-07 15:32:48 +01:00
Peter Steinberger
2dbef6105d agent: allow deliver when json output 2025-12-07 15:16:55 +01:00
Peter Steinberger
eeee9625c1 chore(macos): tighten voice wake control widths 2025-12-07 15:09:16 +01:00
Peter Steinberger
76559b352b debug: surface ssh error details in voice test 2025-12-07 15:07:56 +01:00
Peter Steinberger
a3bf0d6002 fix(macos): honor pnpm/node when locating clawdis for health 2025-12-07 15:07:38 +01:00
Peter Steinberger
96ae0dd23a fix(macos): handle missing clawdis CLI for health check 2025-12-07 15:03:05 +01:00
Peter Steinberger
9c9e04c5a0 debug: add voice forward test button 2025-12-07 15:00:02 +01:00
Peter Steinberger
15381c7832 ci: use macos-latest with Xcode 26.1 2025-12-07 15:00:01 +01:00
Peter Steinberger
175f929023 macOS: widen voice wake label spacing 2025-12-07 13:57:05 +00:00
Peter Steinberger
a23846b3a1 chore(macos): simplify health status menu and messaging 2025-12-07 14:54:58 +01:00
Peter Steinberger
42c74e864a chore(macos): align recognition language row styling 2025-12-07 14:52:43 +01:00
Peter Steinberger
809f5d6d8e chore(macos): align mic level bar width 2025-12-07 14:52:05 +01:00
Peter Steinberger
ff41a61432 chore(macos): clean up CLI helper subtext 2025-12-07 14:49:56 +01:00
Peter Steinberger
28b531593a fix(macos): resolve clawdis path for health check 2025-12-07 14:49:18 +01:00
Peter Steinberger
4d2f4f1be3 chore(macos): make debug settings scrollable 2025-12-07 14:48:12 +01:00
Peter Steinberger
f97415755b chore(macos): remove focus ring on about icon 2025-12-07 14:46:54 +01:00
Peter Steinberger
67fa82cf14 agent: deliver via rpc and voice forward 2025-12-07 06:05:00 +01:00
Peter Steinberger
1d38f5a4d5 Revert "fix: auto-start rpc worker for agent calls"
This reverts commit e70f8471a8.
2025-12-07 05:54:47 +01:00
Peter Steinberger
e70f8471a8 fix: auto-start rpc worker for agent calls 2025-12-07 05:54:15 +01:00
Peter Steinberger
093e737af9 fix: keep launch agent alive and inject PATH 2025-12-07 05:49:59 +01:00
Peter Steinberger
1ae0b44bc5 fix(health): reveal logs alerts when missing; align actions 2025-12-07 05:46:47 +01:00
Peter Steinberger
17aeec59a3 fix: raise voice wake forward timeout to 30s 2025-12-07 05:46:05 +01:00
Peter Steinberger
b20507ef0a chore(health): kick off health refresh at app launch 2025-12-07 05:44:09 +01:00
Peter Steinberger
753995a91d Docs: add no-real-data rule to AGENTS 2025-12-07 04:43:25 +00:00
Peter Steinberger
67c67dd86d Docs: swap to obviously fake phone numbers 2025-12-07 04:42:58 +00:00
Peter Steinberger
fdc0b283d7 Docs: scrub personal phone example 2025-12-07 04:40:08 +00:00
Peter Steinberger
2abc51789e UI: streamline relay status label 2025-12-07 04:39:45 +00:00
Peter Steinberger
1190b9c278 Health: strengthen probe tests 2025-12-07 04:39:24 +00:00
Peter Steinberger
3a8e049093 chore: fix test import and lint 2025-12-07 05:38:29 +01:00
Peter Steinberger
f32a647a20 test: cover command resolver fallbacks 2025-12-07 05:38:29 +01:00
Peter Steinberger
4645f512d1 fix: reuse resolver for agent rpc launch 2025-12-07 05:38:29 +01:00
Peter Steinberger
3d89999a06 docs: add voice wake forwarding tips to agents 2025-12-07 05:38:29 +01:00
Peter Steinberger
cb5c932447 Health: CLI probe and mac UI surfacing 2025-12-07 04:38:20 +00:00
Peter Steinberger
ddf8aef4f7 Settings: move session store path to Debug 2025-12-07 04:38:08 +00:00
Peter Steinberger
2714ed503b CLI: add health probe command 2025-12-07 04:33:22 +00:00
Peter Steinberger
78d96355dd Settings: inline heartbeat inputs 2025-12-07 04:32:28 +00:00
Peter Steinberger
bf429b7e87 Settings: add heartbeat controls 2025-12-07 04:30:24 +00:00
Peter Steinberger
2f44046622 chore(agent): start rpc worker at launch, fail if not running 2025-12-07 05:24:54 +01:00
Peter Steinberger
fb106967bc fix(macos): guard unavailable speech recognizer 2025-12-07 05:22:20 +01:00
Peter Steinberger
32720bd372 feat(agent): add rpc status command and tests; rpc only path 2025-12-07 05:20:50 +01:00
Peter Steinberger
fb1de5c1c6 chore(agent): drop cli fallback, rpc only for sends 2025-12-07 05:16:16 +01:00
Peter Steinberger
69cb71ad7e feat(agent): use persistent rpc worker for agent sends 2025-12-07 05:14:45 +01:00
Peter Steinberger
0a9b98ed67 feat(cli): add stdin/stdout rpc loop for agent sends 2025-12-07 05:10:58 +01:00
Peter Steinberger
e1c4a5989b docs: outline RPC plan for agent CLI 2025-12-07 05:08:14 +01:00
Peter Steinberger
cac988f8e2 fix(webchat): wire agent CLI send into web chat view 2025-12-07 05:04:34 +01:00
Peter Steinberger
bbe92a3a40 Mac: fix agent XPC by invoking CLI agent 2025-12-07 04:03:06 +00:00
Peter Steinberger
a489550752 feat(cli): add agent send command and wire through XPC 2025-12-07 05:00:52 +01:00
Peter Steinberger
f1dbff1dd4 fix(voicewake): log ssh/cli failure instead of staying silent 2025-12-07 04:58:57 +01:00
Peter Steinberger
55ea0f398b test(voicewake): cover trigger matching for runtime listener 2025-12-07 04:53:59 +01:00
Peter Steinberger
38abb044d0 feat(macos): run live voice wake listener and animate ears 2025-12-07 04:52:27 +01:00
Peter Steinberger
ca4e76b34f test: add voice wake forwarder cache coverage 2025-12-07 04:52:26 +01:00
Peter Steinberger
55e0086958 fix: harden remote voice wake CLI lookup 2025-12-07 04:43:08 +01:00
Peter Steinberger
050ebb3b19 Mac: add relay restart button in Debug 2025-12-07 03:42:50 +00:00
Peter Steinberger
31f788eb5e CLI: allow --provider flag for login/logout (default whatsapp) 2025-12-07 03:41:27 +00:00
Peter Steinberger
f23b16db2b build: require signing identity for mac packaging 2025-12-07 04:38:45 +01:00
Peter Steinberger
060f80c239 feat: add icon animation setting 2025-12-07 04:38:45 +01:00
Peter Steinberger
6c3d3b98b8 chore: purge warelay references 2025-12-07 03:36:57 +00:00
Peter Steinberger
21dfbd0103 feat(macos): detect installed CLI helper 2025-12-07 04:35:34 +01:00
Peter Steinberger
1a10569f6d Logging: use /tmp/clawdis for default pino logs 2025-12-07 03:32:37 +00:00
Peter Steinberger
33396ca9c1 Mac: debug log button shows path and opens in Finder 2025-12-07 03:29:58 +00:00
Peter Steinberger
36ba1ff790 Mac: debug log button falls back to legacy path 2025-12-07 03:20:04 +00:00
Peter Steinberger
fdfcff2bb5 Mac: link Debug log button to pino log 2025-12-07 03:15:30 +00:00
Peter Steinberger
c74c1a0c5f fix: stabilize tools action width 2025-12-07 04:13:19 +01:00
Peter Steinberger
faca83e1e8 fix: ensure remote clawdis-mac path 2025-12-07 04:12:54 +01:00
Peter Steinberger
759ab54e59 VoiceWake: ssh check also verifies remote clawdis-mac 2025-12-07 04:01:00 +01:00
Peter Steinberger
3c61524f26 Mac: allow signed CLI + same-uid XPC clients 2025-12-07 02:48:24 +00:00
Peter Steinberger
40013c2b61 fix(mac): bundle WebChat resources when packaging 2025-12-07 03:36:47 +01:00
Peter Steinberger
5d5e7393f8 docs(mac): document webchat auto-open and debug flow 2025-12-07 03:34:49 +01:00
Peter Steinberger
71c5511e6c chore(mac): add webchat auto-open flag and verbose logging 2025-12-07 03:31:03 +01:00
Peter Steinberger
ea83982062 Docs: add clawlog helper note 2025-12-07 03:30:24 +01:00
Peter Steinberger
cdbbdcba5f Docs: describe mac XPC setup 2025-12-07 02:27:59 +00:00
Peter Steinberger
aeb708fe07 Mac: secure XPC and register mach service via launchd 2025-12-07 02:27:17 +00:00
Peter Steinberger
78c67ed53d Mac: stabilize XPC and voice wake handling 2025-12-07 02:09:54 +00:00
Peter Steinberger
ea37ee6cb3 feat(mac): add automation permission 2025-12-07 02:34:21 +01:00
Peter Steinberger
2e67c5a045 VoiceWake: stabilize test card height 2025-12-07 02:33:32 +01:00
Peter Steinberger
752bc5a454 VoiceWake: align mic + level rows 2025-12-07 02:32:57 +01:00
Peter Steinberger
3a4bf8f213 VoiceWake: compact SSH test row 2025-12-07 02:32:05 +01:00
Peter Steinberger
bc20664c18 tools: add clawlog helper for unified logs 2025-12-07 02:25:55 +01:00
Peter Steinberger
e27690e894 VoiceWake: log detection, hold to 1s silence, ssh log clarity 2025-12-07 02:24:18 +01:00
Peter Steinberger
bac5ac18f7 fix: gate voice wake permissions 2025-12-07 02:19:50 +01:00
Peter Steinberger
e906b87450 VoiceWake: keep listening until silence, gate enable on permissions 2025-12-07 02:18:37 +01:00
Peter Steinberger
9d0415f9e9 VoiceWake: make tab content scrollable 2025-12-07 02:17:17 +01:00
Peter Steinberger
1d807911e4 VoiceWake: better ssh target parsing and error detail 2025-12-07 02:17:17 +01:00
Peter Steinberger
f51f8ffe45 scripts: make restart clean step resilient 2025-12-07 02:17:17 +01:00
Peter Steinberger
ea9930816f Mac: disable KeepAlive; launch toggle controls agent 2025-12-07 01:13:48 +00:00
Peter Steinberger
699cb92e86 Mac: let launch checkbox toggle launchd agent 2025-12-07 01:09:49 +00:00
Peter Steinberger
f4f4f2d314 Mac: run via launchd agent with mach service 2025-12-07 01:05:05 +00:00
Peter Steinberger
374472deda VoiceWake: add SSH connectivity check UI 2025-12-07 02:03:54 +01:00
Peter Steinberger
b27f0dd490 Settings: keep tabs fixed, only content scrolls 2025-12-07 02:03:54 +01:00
Peter Steinberger
141d2b5626 VoiceWake: add SSH forwarder tests 2025-12-07 02:03:54 +01:00
Peter Steinberger
cf0f44823a VoiceWake: add SSH forward target 2025-12-07 02:03:54 +01:00
Peter Steinberger
6355113af9 chore(mac): move relay status row directly under Active toggle 2025-12-07 02:03:54 +01:00
Peter Steinberger
00ef7ec522 Mac: align app version with package.json 2025-12-07 01:00:47 +00:00
Peter Steinberger
9497a4cb5a CLI: fix --version by reading app Info.plist 2025-12-07 00:59:37 +00:00
Peter Steinberger
0f71667625 CLI: add --version flag 2025-12-07 00:55:33 +00:00
Peter Steinberger
8b20e0166d CLI: add --help and usage 2025-12-07 00:53:22 +00:00
Peter Steinberger
567644dabd Mac: privileged CLI helper install via osascript 2025-12-07 00:50:56 +00:00
Peter Steinberger
9ef8cdadf6 Mac: lighten tool cards 2025-12-07 00:17:54 +00:00
Peter Steinberger
c911568306 Mac: remove Tools & MCP header 2025-12-07 00:16:39 +00:00
Peter Steinberger
ce02f798e4 Mac: fix voice wake actor crash; add mic entitlement 2025-12-07 00:10:29 +00:00
Peter Steinberger
21bb2fb03f Mac: add mic entitlement to signing helper 2025-12-06 23:52:54 +00:00
Peter Steinberger
11311d07e5 mac: tidy About metadata layout 2025-12-07 00:48:05 +01:00
Peter Steinberger
4426bf2615 Docs: note SIGN_IDENTITY for mac signing 2025-12-06 23:45:17 +00:00
Peter Steinberger
515e973964 Mac: fix permission prompt crash 2025-12-06 23:31:56 +00:00
Peter Steinberger
0a6b934ac1 mac: show build metadata in About 2025-12-07 00:30:58 +01:00
Peter Steinberger
b2e3013898 mac: add signing helper and document debug bundle 2025-12-07 00:30:58 +01:00
Peter Steinberger
757cedc233 fix: remove legacy relay references 2025-12-06 23:21:25 +00:00
Peter Steinberger
ab316b348a docs: update relay run mode 2025-12-07 00:16:32 +01:00
Peter Steinberger
7b7c4bd116 chore: fix swiftlint after split 2025-12-07 00:14:03 +01:00
Peter Steinberger
82e751a153 macOS: split AppMain into focused modules 2025-12-07 00:10:35 +01:00
Peter Steinberger
c5c50a2141 fix(mac): bundle web chat UI deps 2025-12-07 00:05:38 +01:00
Peter Steinberger
9c32e630a0 docs: add 500 LOC cap to guardrails 2025-12-06 23:59:08 +01:00
Peter Steinberger
02e26996c1 fix(mac): run relay with cwd set to configured project root 2025-12-06 23:57:40 +01:00
Peter Steinberger
b25b72ae19 chore(mac): rename relay root label to Clawdis project root 2025-12-06 23:56:23 +01:00
Peter Steinberger
ff36375581 feat(mac): show relay attention badge without dimming paused state 2025-12-06 23:54:56 +01:00
Peter Steinberger
c9f5edbc1d feat(mac): make relay project root configurable from Debug tab 2025-12-06 23:51:34 +01:00
Peter Steinberger
ec00e0a952 fix(mac): run pnpm from project root and set PNPM_HOME for relay 2025-12-06 23:49:59 +01:00
Peter Steinberger
51a4b86495 fix(mac): resolve relay executable via common paths and pnpm fallback 2025-12-06 23:48:44 +01:00
Peter Steinberger
c3866b7d6b docs: document debug signing and bundle id 2025-12-06 23:46:25 +01:00
Peter Steinberger
6dafca79be build: sign debug app and use stable bundle id 2025-12-06 23:46:19 +01:00
Peter Steinberger
7aca8d2d1c fix(mac): harden relay spawn path and show status 2025-12-06 23:45:16 +01:00
Peter Steinberger
7daef74fc6 chore: move relay status below toggles 2025-12-06 23:44:20 +01:00
Peter Steinberger
58d0f3053d feat(mac): show relay run indicator in menu 2025-12-06 23:43:36 +01:00
Peter Steinberger
649f644c75 chore: reorder settings tabs 2025-12-06 23:41:21 +01:00
Peter Steinberger
ad2a26611a chore: move model reload to debug tab 2025-12-06 23:40:50 +01:00
Peter Steinberger
89bb7d0211 fix(macos): avoid voice tester crash 2025-12-06 23:39:13 +01:00
Peter Steinberger
b3564bf2b4 chore(mac): guard Darwin import for relay manager 2025-12-06 23:26:29 +01:00
Peter Steinberger
16f452cf2e feat(macos): add tools tab installers 2025-12-06 23:25:17 +01:00
Peter Steinberger
56cedad707 chore: remove bin/warelay.js 2025-12-06 23:17:01 +01:00
Peter Steinberger
4b6325908b feat: unify main session and icon cues 2025-12-06 23:16:23 +01:00
Peter Steinberger
460d8fc094 feat(mac): add child relay process manager 2025-12-06 22:05:14 +01:00
Peter Steinberger
c435236ceb mac: streamline model config UI 2025-12-06 21:39:25 +01:00
Peter Steinberger
39254229a0 mac: fix notification prompt and center onboarding toggle 2025-12-06 21:38:21 +01:00
Peter Steinberger
6182b205c8 mac: fix web chat boot in WKWebView 2025-12-06 21:33:35 +01:00
Peter Steinberger
e528b439bc build: add mac icon pipeline 2025-12-06 21:00:32 +01:00
Peter Steinberger
629140d66c docs: document macOS Voice Wake and on-device processing 2025-12-06 05:24:27 +01:00
Peter Steinberger
46ed4f2de1 docs: clarify Voice Wake runs on-device 2025-12-06 05:23:28 +01:00
Peter Steinberger
46d55a8ada fix: harden model catalog parsing 2025-12-06 05:21:07 +01:00
Peter Steinberger
5e6af3d732 fix: add Config tab title case 2025-12-06 05:17:57 +01:00
Peter Steinberger
0d07c58989 fix: expose Config tab in settings 2025-12-06 05:15:15 +01:00
Peter Steinberger
6f80be0653 mac: add webview debug logging 2025-12-06 05:13:33 +01:00
Peter Steinberger
1916e688a6 feat: load PI model catalog and add dropdown in Config tab 2025-12-06 05:10:21 +01:00
Peter Steinberger
07e56ddeb5 docs: note bundled web chat assets 2025-12-06 05:03:51 +01:00
Peter Steinberger
88c8009116 feat: move CLI config into its own Settings tab 2025-12-06 05:03:03 +01:00
Peter Steinberger
42d843297d mac: bundle web chat assets 2025-12-06 05:01:28 +01:00
Peter Steinberger
15cdeeddaf feat: add config editor for clawdis model and session store 2025-12-06 04:27:50 +01:00
Peter Steinberger
3c13a265bc mac: add web chat bridge and docs 2025-12-06 04:14:14 +01:00
Peter Steinberger
93eec9ac3c mac: expand settings layout and dock toggle 2025-12-06 04:11:31 +01:00
Peter Steinberger
df7dbff683 ui: align live level row with mic picker 2025-12-06 04:08:26 +01:00
Peter Steinberger
2e6265963b chore: align lint/format configs with peekaboo defaults 2025-12-06 04:07:22 +01:00
Peter Steinberger
b88b18df93 fix: apply dock icon preference at launch 2025-12-06 04:04:23 +01:00
Peter Steinberger
fbf5333b39 chore: run formatters and lint 2025-12-06 04:03:48 +01:00
Peter Steinberger
c6e3b490f5 ci: add swiftlint/swiftformat for mac app 2025-12-06 04:02:43 +01:00
Peter Steinberger
19677f0622 ci: add macOS app build 2025-12-06 03:56:49 +01:00
Peter Steinberger
a8932c2c25 feat: add additional voice wake languages + clean locale labels 2025-12-06 03:55:47 +01:00
Peter Steinberger
f93e33d9de fix: ignore cancellation and keep mic meter during test 2025-12-06 03:55:47 +01:00
Peter Steinberger
649e6efc4a fix: decouple voice tester from main actor 2025-12-06 03:55:47 +01:00
Peter Steinberger
a7d3619ec4 fix: avoid audio tap isolation crash 2025-12-06 03:55:47 +01:00
Peter Steinberger
daca3a5fc9 fix: stabilize voice wake test 2025-12-06 03:55:47 +01:00
Peter Steinberger
135a52020c fix: run speech tap and handlers on safe queues 2025-12-06 03:55:47 +01:00
Peter Steinberger
bf21ed7282 feat: add language picker for Voice Wake 2025-12-06 03:55:47 +01:00
Peter Steinberger
b5f65e3304 chore: gate Voice Wake on macOS 26 2025-12-06 03:55:47 +01:00
Peter Steinberger
45400a1758 fix: show live transcript in voice wake test 2025-12-06 03:55:47 +01:00
Peter Steinberger
acc88bc2b4 tweak: faster mic meter response 2025-12-06 03:55:47 +01:00
Peter Steinberger
98b0595275 fix: pause mic meter while running voice wake test 2025-12-06 03:55:47 +01:00
Peter Steinberger
4efecfdfa0 feat: add live mic meter to Voice Wake 2025-12-06 03:55:47 +01:00
Peter Steinberger
b5afb9d3ab feat: add mic selection to Voice Wake settings 2025-12-06 03:55:47 +01:00
Peter Steinberger
6a1d58d4e7 mac: fix voice wake mic picker build 2025-12-06 03:55:47 +01:00
Peter Steinberger
d2a3db4c78 mac: add app icon and tidy voice picker 2025-12-06 03:55:47 +01:00
Peter Steinberger
f207788c0a refactor: make voice wake tester an actor 2025-12-06 03:55:46 +01:00
Peter Steinberger
84b44069c8 fix: run voice wake permission callbacks off the main actor 2025-12-06 03:55:46 +01:00
Peter Steinberger
09ed3f37db fix: keep voice wake permission callbacks on main actor 2025-12-06 03:55:46 +01:00
Peter Steinberger
f444604e7c feat: surface mic and speech permissions 2025-12-06 03:55:46 +01:00
Peter Steinberger
e1c9885566 chore: vendor swabble and add speech usage strings 2025-12-06 03:55:46 +01:00
Peter Steinberger
4e7d905783 mac: lock onboarding page width to 640 2025-12-06 03:55:46 +01:00
Peter Steinberger
cb35e3a766 mac: add sessions tab to settings 2025-12-06 03:55:46 +01:00
Peter Steinberger
67fe5ed699 chore(mac): widen settings and keep critter static when paused 2025-12-06 03:55:46 +01:00
Peter Steinberger
e863fd78d6 CLI: compact sessions table output 2025-12-06 00:49:21 +00:00
Peter Steinberger
4ea2518e79 mac: align settings window layout 2025-12-06 01:33:28 +01:00
Peter Steinberger
b508ab240f fix(mac): stop critter animation when paused 2025-12-06 01:29:53 +01:00
Peter Steinberger
c545ec727c chore(mac): rely on status item disable for dimming 2025-12-06 01:27:20 +01:00
Peter Steinberger
f290b9a145 fix(mac): align restart/package to use .build 2025-12-06 01:23:54 +01:00
Peter Steinberger
fa1eb9bf25 fix(mac): rebuild into .build-local and clean cache 2025-12-06 01:21:31 +01:00
Peter Steinberger
3a32b83181 chore(mac): label toggle as Clawdis Active 2025-12-06 01:17:25 +01:00
Peter Steinberger
2e393b7d5c docs: note trimmy-style restart and dimming 2025-12-06 01:15:42 +01:00
Peter Steinberger
12e5b8124e chore(mac): rebuild and relaunch like trimmy 2025-12-06 01:15:01 +01:00
Peter Steinberger
26e939c1eb fix(mac): dim menubar icon like trimmy 2025-12-06 01:07:15 +01:00
Peter Steinberger
f09390a412 mac: fix settings window size persistence 2025-12-06 00:56:06 +01:00
Peter Steinberger
3067807802 mac: trimmy-style padding and debug toggle 2025-12-06 00:55:10 +01:00
Peter Steinberger
60f4c9f5b3 mac: tighten onboarding card layout 2025-12-06 00:52:22 +01:00
Peter Steinberger
b0ecafcb8d mac: bring onboarding layout closer to vibetunnel 2025-12-06 00:50:22 +01:00
Peter Steinberger
ddfb76e9e0 fix: bundle pi dependency and directive handling 2025-12-06 00:49:46 +01:00
Peter Steinberger
6f27f742fe feat(mac): add critter ear/leg wiggles 2025-12-06 00:49:30 +01:00
Peter Steinberger
c1a64301ce mac: lock settings window size 2025-12-06 00:46:24 +01:00
Peter Steinberger
1ee690e87c mac: match trimmy about layout 2025-12-06 00:44:22 +01:00
Peter Steinberger
28e0dbc02f fix: harden directive handling 2025-12-05 23:43:30 +00:00
Peter Steinberger
a2604a36bc mac: tighten settings layout 2025-12-06 00:42:41 +01:00
Peter Steinberger
d031c5c7fa mac: auto-show onboarding on first run 2025-12-06 00:40:09 +01:00
Peter Steinberger
5d01b32c10 mac: polish onboarding and lifecycle 2025-12-06 00:38:02 +01:00
Peter Steinberger
4fe651079c fix(mac): align critter legs 2025-12-06 00:38:02 +01:00
Peter Steinberger
a573ea4aeb chore: open settings from menu and restart packaged app 2025-12-06 00:38:02 +01:00
Peter Steinberger
13704d9da5 chore: add settings shortcut and restart packaging 2025-12-06 00:38:02 +01:00
Peter Steinberger
73a1e137e6 feat: trimmy-style settings tabs and CLI helper bundling 2025-12-06 00:38:02 +01:00
Peter Steinberger
0ec9c6c3cf fix(mac): show critter menubar icon 2025-12-06 00:38:02 +01:00
Peter Steinberger
4aa275e13c feat(mac): animate menubar icon 2025-12-06 00:38:02 +01:00
Peter Steinberger
b557a73c3f feat: richer mac settings panes and template icon 2025-12-06 00:38:02 +01:00
Peter Steinberger
b66098ea20 chore: bundle mac app and custom menu icon 2025-12-06 00:38:02 +01:00
Peter Steinberger
d0cefecd0d chore: add mac build+run helper 2025-12-06 00:38:02 +01:00
Peter Steinberger
38a4e9806f chore: ignore mac build artifacts 2025-12-06 00:38:02 +01:00
Peter Steinberger
3c64a57c84 revert prompt-too-long fallback and keep inline directives 2025-12-05 23:18:03 +00:00
Peter Steinberger
36b0796976 fix: handle prompt-too-long by resetting session and continuing inline directives 2025-12-05 23:01:37 +00:00
Peter Steinberger
3241d81ce5 fix: allow inline directives to continue and add mixed-message test 2025-12-05 22:57:52 +00:00
Peter Steinberger
d7a188fb34 fix: broaden prompt-echo guard and add heartbeat directive test 2025-12-05 22:56:07 +00:00
Peter Steinberger
5b217b2042 fix: suppress heartbeat directive acks and add coverage 2025-12-05 22:54:17 +00:00
Peter Steinberger
4cb2a92037 fix: avoid echoing prompts when rpc returns empty 2025-12-05 22:52:21 +00:00
Peter Steinberger
24d90c17c2 fix: ignore directives inside history blocks 2025-12-05 22:49:41 +00:00
Peter Steinberger
c95c6d72e9 test: cover directive parsing and abort/restart prefixes 2025-12-05 22:29:49 +00:00
Peter Steinberger
99b174f495 fix: avoid directive hits inside URLs and add tests 2025-12-05 22:28:36 +00:00
Peter Steinberger
5949ef0e2c chore: rename package to clawdis 2025-12-05 23:19:46 +01:00
Peter Steinberger
d75d64df64 chore: ignore macOS .DS_Store globally 2025-12-05 23:19:04 +01:00
Peter Steinberger
a5164df293 feat: add mac companion app 2025-12-05 23:18:47 +01:00
Peter Steinberger
690113dd73 Add bundled pi default and session token reporting 2025-12-05 23:18:43 +01:00
Peter Steinberger
fe87160b19 chore: add system marker to directives and abort 2025-12-05 21:37:11 +00:00
Peter Steinberger
fffe1be521 docs: note directive short-circuit 2025-12-05 21:30:01 +00:00
Peter Steinberger
dc02bcee74 fix: normalize directive triggers and short-circuit 2025-12-05 21:29:41 +00:00
Peter Steinberger
e7a9313135 chore: remove twilio and expand pi cli detection 2025-12-05 21:13:23 +00:00
Peter Steinberger
5492845659 feat: stream turn completions and tighten rpc timeout 2025-12-05 21:13:17 +00:00
Peter Steinberger
29dfe89137 chore: redact long texts in web logs 2025-12-05 19:21:23 +00:00
Peter Steinberger
0da3f84a2e fix: ignore rpc toolcall deltas to avoid duplicate replies 2025-12-05 19:16:03 +00:00
Peter Steinberger
c25b0c1a66 docs: update for web-only pi rpc 2025-12-05 19:04:09 +00:00
Peter Steinberger
7c7314f673 chore: drop twilio and go web-only 2025-12-05 19:03:59 +00:00
Peter Steinberger
869cc3d497 Route pi agent prompts via RPC stdin 2025-12-05 18:34:05 +00:00
Peter Steinberger
f315bf074b fix: harden pi rpc prompt handling 2025-12-05 18:24:45 +00:00
Peter Steinberger
d33f9ddf44 docs: add repo link to homepage 2025-12-05 17:51:11 +00:00
Peter Steinberger
fcf0c28132 chore: make pi-only rpc with fixed sessions 2025-12-05 17:50:02 +00:00
Peter Steinberger
b3e50cbb33 Switch to clawdis RPC mode and complete rebrand 2025-12-05 17:22:53 +00:00
Peter Steinberger
20cb709ae3 chore: organize imports after rebrand 2025-12-04 18:02:51 +00:00
Peter Steinberger
916a41ed60 branding: default to clawdis paths and launchd label 2025-12-04 18:01:30 +00:00
Peter Steinberger
9797a9993a docs: document agent CLI and changelog 2025-12-04 17:55:38 +00:00
Peter Steinberger
04ce98148d web: fix mentioned JID extraction typing 2025-12-04 17:54:51 +00:00
Peter Steinberger
34eb75f634 auto-reply: honor /new after timestamp prefixes 2025-12-04 17:54:20 +00:00
Peter Steinberger
05b76281f7 CLI: add agent command for direct agent runs 2025-12-04 17:54:20 +00:00
Eng. Juan Combetto
4a35bcec21 fix: resolve lint errors (unused vars, imports, formatting)
- Prefix unused test variables with underscore
- Remove unused piSpec import and idleMs class member
- Fix import ordering and code formatting
2025-12-04 16:15:17 +00:00
Eng. Juan Combetto
518af0ef24 config: support clawdis.json path for rebranding
- Add CONFIG_PATH_CLAWDIS (~/.clawdis/clawdis.json) as preferred path
- Keep CONFIG_PATH_LEGACY (~/.warelay/warelay.json) for backward compatibility
- Update loadConfig() to check clawdis.json first, fallback to warelay.json
- Fix TypeScript type error in extractMentionedJids (null handling)

Part of the warelay → clawdis rebranding effort.
2025-12-04 16:15:17 +00:00
Peter Steinberger
a155ec0599 auto-reply: handle group think/verbose directives 2025-12-04 02:29:32 +00:00
Peter Steinberger
80979cf4d0 🦞 Add backlinks to clawd.me, soul.md, steipete.me 2025-12-03 15:46:29 +00:00
Peter Steinberger
a27ee2366e 🦞 Rebrand to CLAWDIS - add docs, update README
- New README with CLAWDIS branding
- docs/index.md - Main landing page
- docs/configuration.md - Config guide
- docs/agents.md - Agent integration guide
- docs/security.md - Security lessons (including the find ~ incident)
- docs/troubleshooting.md - Debug guide
- docs/lore.md - The origin story

EXFOLIATE!
2025-12-03 15:45:43 +00:00
Peter Steinberger
7bc56d7cfe test: cover verbose directive in group batches 2025-12-03 15:45:43 +00:00
Peter Steinberger
088bdb3313 fix: allow directive-only toggles inside group batches 2025-12-03 15:45:43 +00:00
Peter Steinberger
89d49cd925 chore: bump version to 1.4.0 2025-12-03 15:45:43 +00:00
Peter Steinberger
84f8d8733e docs: note media-only mention fix 2025-12-03 15:45:43 +00:00
Peter Steinberger
07f323222b fix(web): capture mentions from media captions 2025-12-03 15:45:43 +00:00
Peter Steinberger
a321bf1a90 fix(web): surface media fetch failures 2025-12-03 15:45:43 +00:00
Peter Steinberger
92a0763a74 changelog: note verbose tool emoji/previews 2025-12-03 15:45:43 +00:00
Peter Steinberger
e878780808 auto-reply: single emoji per verbose tool line 2025-12-03 15:45:43 +00:00
Peter Steinberger
cb5f1fa99d auto-reply: emoji + result preview for verbose tool calls 2025-12-03 15:45:43 +00:00
Peter Steinberger
b55ac994ea feat(web): prime group sessions with member roster 2025-12-03 15:45:43 +00:00
Peter Steinberger
3a8d6b80e0 auto-reply: surface tool args from rpc start events 2025-12-03 15:45:43 +00:00
Peter Steinberger
3354a68373 Create CNAME 2025-12-03 16:44:03 +01:00
Peter Steinberger
edc894f6c7 fix(web): annotate group replies with sender 2025-12-03 13:25:34 +00:00
Peter Steinberger
f68714ec8e fix(web): unwrap ephemeral/view-once and keep mentions 2025-12-03 13:15:46 +00:00
Peter Steinberger
7be9352a3a test(web): ensure group messages carry sender + bypass allowFrom 2025-12-03 13:12:05 +00:00
Peter Steinberger
3a782b6ace fix(web): let group pings bypass allowFrom 2025-12-03 13:11:01 +00:00
Peter Steinberger
47d0b6fc14 changelog: note logging capture and verbose trace 2025-12-03 13:09:29 +00:00
Peter Steinberger
8204351d67 fix(web): allow group replies past allowFrom 2025-12-03 13:08:54 +00:00
Peter Steinberger
4c3635a7c0 logging: route console output into pino 2025-12-03 13:07:47 +00:00
Peter Steinberger
7ea43b0145 fix(web): detect self number mentions in group chats 2025-12-03 12:43:20 +00:00
Peter Steinberger
6afe6f4ecb feat(web): add group chat mention support 2025-12-03 12:35:18 +00:00
Peter Steinberger
273f2b61d0 Docs: document /restart WhatsApp command 2025-12-03 12:16:51 +00:00
Peter Steinberger
0824873ffb Add /restart WhatsApp command to restart warelay 2025-12-03 12:14:32 +00:00
Peter Steinberger
8f99b13305 Pi: stream tool results faster (0.5s, flush after 5) 2025-12-03 12:08:58 +00:00
Peter Steinberger
9253702966 Pi: stream assistant text during RPC runs 2025-12-03 11:50:49 +00:00
Peter Steinberger
3958450223 Tau RPC: resolve on agent_end or exit 2025-12-03 11:34:00 +00:00
Peter Steinberger
cc596ef011 Pi: resume Tau sessions with --continue 2025-12-03 11:33:51 +00:00
Peter Steinberger
8220b11770 Tau RPC: wait for agent_end when tools run 2025-12-03 11:29:12 +00:00
Peter Steinberger
62c54cd47c Web: simplify logout message 2025-12-03 11:04:12 +00:00
Peter Steinberger
e34d0d69aa Chore: satisfy lint after tool-meta refactor 2025-12-03 10:42:10 +00:00
Peter Steinberger
597e7e6f13 Refactor: extract tool meta formatter + debouncer 2025-12-03 10:30:01 +00:00
Peter Steinberger
b460fd61bd Verbose: shorten meta paths when aggregating 2025-12-03 10:26:41 +00:00
Peter Steinberger
c9b5df8184 Verbose: collapse tool meta paths by directory 2025-12-03 10:24:41 +00:00
Peter Steinberger
341ecf3bbe Docs: note 1s tool coalescing window 2025-12-03 10:19:10 +00:00
Peter Steinberger
b6b5144ddf Verbose: slow tool batch window to 1s 2025-12-03 10:13:02 +00:00
Peter Steinberger
deac5ff585 Verbose: shorten home paths in tool meta 2025-12-03 10:12:27 +00:00
Peter Steinberger
38a03ff2c8 Verbose: batch rapid tool results 2025-12-03 10:11:41 +00:00
Peter Steinberger
527bed2b53 Verbose: include tool arg metadata in prefixes 2025-12-03 09:57:41 +00:00
Peter Steinberger
318166f8b0 Verbose: send tool result metadata only 2025-12-03 09:40:05 +00:00
Peter Steinberger
394c751d7d Tau RPC: resolve on agent_end 2025-12-03 09:39:26 +00:00
Peter Steinberger
86d707ad51 Docs: note streaming verbose tool results 2025-12-03 09:22:43 +00:00
Peter Steinberger
c3792db0e5 Auto-reply: stream verbose tool results via tau rpc 2025-12-03 09:21:31 +00:00
Peter Steinberger
16e42e6d6d Auto-reply: show tool results before main reply in verbose mode 2025-12-03 09:14:10 +00:00
Peter Steinberger
53c1674382 Chore: format + lint fixes 2025-12-03 09:09:34 +00:00
Peter Steinberger
85917d4769 Docs: mention verbose hints 2025-12-03 09:08:03 +00:00
Peter Steinberger
ae0d35c727 Auto-reply: add verbose session hint 2025-12-03 09:07:17 +00:00
Peter Steinberger
086dd284d6 Auto-reply: add /verbose directives and tool result replies 2025-12-03 09:04:37 +00:00
Peter Steinberger
8ba35a2dc3 Auto-reply: treat prefixed think directives as directive-only 2025-12-03 08:57:30 +00:00
Peter Steinberger
48dfb1c8ca Auto-reply: ack think directives 2025-12-03 08:54:38 +00:00
Peter Steinberger
5a83a44112 Docs: document thinking levels 2025-12-03 08:45:30 +00:00
Peter Steinberger
58520859e5 Auto-reply: add thinking directives 2025-12-03 08:45:23 +00:00
Peter Steinberger
4faba0fe8b Changelog: heartbeat array handling 2025-12-03 01:03:59 +00:00
Peter Steinberger
c4b0155cc2 Format: align thinking helpers 2025-12-03 01:02:10 +00:00
Peter Steinberger
38b18202fc Heartbeat: guard optional heartbeatCommand 2025-12-03 00:45:27 +00:00
Peter Steinberger
0f17a7d828 Heartbeat: normalize reply arrays for twilio/web 2025-12-03 00:43:28 +00:00
Peter Steinberger
9da5b9f4bb Heartbeat: normalize array replies 2025-12-03 00:40:19 +00:00
Peter Steinberger
a7fdc7b992 Auto-reply: allow array payloads in signature 2025-12-03 00:35:57 +00:00
Peter Steinberger
f519e22e6d CI: fix command-reply payload typing 2025-12-03 00:33:58 +00:00
Peter Steinberger
ecac4dd72a Auto-reply: format and lint fixes 2025-12-03 00:30:05 +00:00
Peter Steinberger
b6c45485bc Auto-reply: smarter chunking breaks 2025-12-03 00:25:01 +00:00
Peter Steinberger
ec46932259 web: handle multi-payload replies 2025-12-02 23:46:11 +00:00
Peter Steinberger
10182f1182 limits: chunk replies for twilio/web 2025-12-02 23:10:16 +00:00
Peter Steinberger
cfaec9d608 auto-reply: support multi-text RPC outputs 2025-12-02 23:03:55 +00:00
Peter Steinberger
0f6157a49d logging: emit agent/session meta at command start 2025-12-02 21:30:28 +00:00
Peter Steinberger
1df6373cb1 revert: mark system prompt sent on first turn 2025-12-02 21:23:56 +00:00
Peter Steinberger
ea32cd85fe chore: cut 1.3.1 in changelog 2025-12-02 21:13:47 +00:00
Peter Steinberger
716524c151 docs: note media cleanup and tau rpc typing 2025-12-02 21:13:21 +00:00
Peter Steinberger
96722bba08 ci: fix lint and tau rpc typing 2025-12-02 21:12:51 +00:00
Peter Steinberger
4e20a20927 fix(media): clean up files after response finishes 2025-12-02 21:10:18 +00:00
Peter Steinberger
a0d1004909 test(media): add redirect coverage and update changelog 2025-12-02 21:09:26 +00:00
Peter Steinberger
ccab950d16 Merge branch 'fix/media-replies' 2025-12-02 21:07:45 +00:00
Peter Steinberger
2018c90ae2 chore: tidy claude prompt and drop npm lock 2025-12-02 21:07:37 +00:00
Joao Lisboa
793360c5bb style: fix biome formatting 2025-12-02 21:07:13 +00:00
Joao Lisboa
d8b1a38350 style: fix biome lint errors 2025-12-02 21:07:13 +00:00
Joao Lisboa
499a3e3227 style: fix biome formatting 2025-12-02 21:07:13 +00:00
Joao Lisboa
73a9fdca2a fix: send Claude identity prefix on first session message
The systemSent variable was being set to true before being passed to
runCommandReply, causing the identity prefix to never be injected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:07:13 +00:00
Joao Lisboa
06dd9b8ed8 fix: follow redirects when downloading Twilio media
node:https request() doesn't follow redirects by default, causing
Twilio media URLs (which 302 to CDN) to save placeholder/metadata
instead of actual images.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:07:13 +00:00
Joao Lisboa
a86cb932cf chore: user-agnostic Claude identity and tests
- Use ~/Clawd instead of hardcoded /Users/steipete/clawd
- Add MEDIA: syntax instructions to identity prefix
- Update tests to check for 'scratchpad' instead of specific path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:07:13 +00:00
Joao Lisboa
2fae0a9f47 fix: media serving and id consistency
- server.ts: Replace sendFile with manual readFile+send to fix
  NotFoundError when serving media (sendFile failed even after stat)
- store.ts: Return id with file extension so it matches actual filename

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:07:13 +00:00
Joao Lisboa
2ec9192010 fix: use export type for type-only re-exports
Fixes build error with isolatedModules.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:06:27 +00:00
Peter Steinberger
202eff984d docs: update agent guidance and changelog 2025-12-02 20:10:43 +00:00
Peter Steinberger
b172b538fc perf(pi): reuse tau rpc for command auto-replies 2025-12-02 20:09:51 +00:00
Peter Steinberger
a34271adf9 chore: credit media fix contributor 2025-12-02 18:38:02 +00:00
Peter Steinberger
2cf134668c fix(media): block symlink traversal 2025-12-02 18:37:15 +00:00
Joao Lisboa
b94b220156 Fix path traversal vulnerability in media server
The /media/:id endpoint was vulnerable to path traversal attacks.
Since this endpoint is exposed via Tailscale Funnel (unlike the
WhatsApp webhook which requires Twilio signature validation),
attackers could directly request paths like /media/%2e%2e%2fwarelay.json
to access sensitive files in ~/.warelay/ (e.g. warelay.json), or even
escape further to the user's home directory via multiple ../ sequences.

Fix: validate resolved paths stay within the media directory.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 19:33:21 +01:00
Peter Steinberger
26921cbe68 chore(logs): rotate daily and prune after 24h 2025-12-02 17:11:43 +00:00
Peter Steinberger
8844674825 chore(security): purge session store on logout 2025-12-02 16:33:44 +00:00
Peter Steinberger
c9fbe2cb92 chore(security): harden ipc socket 2025-12-02 16:09:40 +00:00
Peter Steinberger
2b941ccc93 Changelog: note multi-agent and batching
Co-authored-by: RealSid08 <RealSid08@users.noreply.github.com>
2025-12-02 11:11:50 +00:00
Peter Steinberger
ed080ae988 Tests: cover agents and fix web defaults
Co-authored-by: RealSid08 <RealSid08@users.noreply.github.com>
2025-12-02 11:08:00 +00:00
Peter Steinberger
f31e89d5af Agents: add pluggable CLIs
Co-authored-by: RealSid08 <RealSid08@users.noreply.github.com>
2025-12-02 11:07:46 +00:00
Peter Steinberger
52c311e47f chore: bump version to 1.3.0 2025-12-02 07:54:49 +00:00
Peter Steinberger
5b54d4de7a feat(web): batch inbound messages 2025-12-02 07:54:13 +00:00
Peter Steinberger
96152f6577 Add typing indicator after IPC send
After sending via IPC, automatically show "composing" indicator so
user knows more messages may be coming from the running session.
2025-12-02 06:58:17 +00:00
Peter Steinberger
e881b3c5de Document exclamation mark escaping workaround for Claude Code
Add symlink CLAUDE.md -> AGENTS.md for Claude Code compatibility.
2025-12-02 06:52:56 +00:00
Peter Steinberger
e86b507da7 Add IPC to prevent Signal session corruption from concurrent connections
When the relay is running, `warelay send` and `warelay heartbeat` now
communicate via Unix socket IPC (~/.warelay/relay.sock) to send messages
through the relay's existing WhatsApp connection.

Previously, these commands created new Baileys sockets that wrote to the
same auth state files, corrupting the Signal session ratchet and causing
the relay's subsequent sends to fail silently.

Changes:
- Add src/web/ipc.ts with Unix socket server/client
- Relay starts IPC server after connecting
- send command tries IPC first, falls back to direct
- heartbeat uses sendWithIpcFallback helper
- inbound.ts exposes sendMessage on listener object
- Messages sent via IPC are added to echo detection set
2025-12-02 06:31:07 +00:00
Peter Steinberger
2fc3a822c8 web: isolate session fixtures and skip heartbeat when busy 2025-12-02 06:17:16 +00:00
Peter Steinberger
1b0e1edb08 Update changelog with error message and test isolation fixes 2025-12-02 05:59:31 +00:00
Peter Steinberger
d107b79c63 Fix test corrupting production sessions.json
The test 'falls back to most recent session when no to is provided' was
using resolveStorePath() which returns the real ~/.warelay/sessions.json.
This overwrote production session data with test values, causing session
fragmentation issues.

Changed to use a temp directory like other tests.
2025-12-02 05:54:31 +00:00
Peter Steinberger
c5ab442f46 Fix empty result JSON dump and missing heartbeat prefix
Bug fixes:
- Empty result field handling: Changed truthy check to explicit type
  check (`typeof parsed?.text === "string"`) in command-reply.ts.
  Previously, Claude CLI returning `result: ""` would cause raw JSON
  to be sent to WhatsApp.
- Response prefix on heartbeat: Apply `responsePrefix` to heartbeat
  alert messages in runReplyHeartbeat, matching behavior of regular
  message handler.
2025-12-02 04:29:17 +00:00
Peter Steinberger
c5677df56e Increase watchdog timeout to 30 minutes
Changed from 10 to 30 minutes to avoid false positives when
heartbeatMinutes is set to 10. The watchdog should be significantly
longer than the heartbeat interval to account for:
- Network latency
- Slow command responses
- Brief connection hiccups

With heartbeatMinutes=10, a 30-minute watchdog gives 3x buffer before
triggering auto-restart.
2025-11-30 18:03:19 +00:00
Peter Steinberger
21ba0fb8a4 Fix test isolation to prevent loading real user config
Tests were picking up real ~/.warelay/warelay.json with emojis and
prefixes (like "🦞"), causing test assertions to fail. Added proper
config mocks to all test files.

Changes:
- Mock loadConfig() in index.core.test.ts, inbound.media.test.ts,
  monitor-inbox.test.ts
- Update test-helpers.ts default mock to disable all prefixes
- Tests now use clean config: no messagePrefix, no responsePrefix,
  no timestamp, allowFrom=["*"]

This ensures tests validate core behavior without user-specific config.
The responsePrefix feature itself is already fully config-driven - this
only fixes test isolation.
2025-11-30 18:00:57 +00:00
Peter Steinberger
69319a0569 Add auto-recovery from stuck WhatsApp sessions
Fixes issue where unauthorized messages from +212652169245 (5elements spa)
triggered Bad MAC errors and silently killed the event emitter, preventing
all future message processing.

Changes:
1. Early allowFrom filtering in inbound.ts - blocks unauthorized senders
   before they trigger encryption errors
2. Message timeout watchdog - auto-restarts connection if no messages
   received for 10 minutes
3. Health monitoring in heartbeat - warns if >30 min without messages
4. Mock loadConfig in tests to handle new dependency

Root cause: Event emitter stopped firing after Bad MAC errors from
decryption attempts on messages from unauthorized senders. Connection
stayed alive but all subsequent messages.upsert events silently failed.
2025-11-30 17:53:32 +00:00
Peter Steinberger
37d8e55991 Skip responsePrefix for HEARTBEAT_OK responses
Preserve exact match so warelay recognizes heartbeat responses
and doesn't send them as messages.
2025-11-29 06:02:21 +00:00
Peter Steinberger
8d20edb028 Simplify timestampPrefix: bool or timezone string, default true
- timestampPrefix: true (UTC), false (off), or 'America/New_York'
- Removed separate timestampTimezone option
- Default is now enabled (true/UTC) unless explicitly false
2025-11-29 05:29:29 +00:00
Peter Steinberger
7564c4e7f4 Generalize prefix config: messagePrefix + responsePrefix
Replaces samePhoneMarker/samePhoneResponsePrefix with:
- messagePrefix: prefix for all inbound messages
  - Default: '[warelay]' if no allowFrom, else ''
- responsePrefix: prefix for all outbound replies

Also adds timestamp options:
- timestampPrefix: boolean to enable [Nov 29 06:30] format
- timestampTimezone: IANA timezone (default UTC)

Updated README with new config table entries.
2025-11-29 05:27:58 +00:00
Peter Steinberger
26e02a9b8b Add timestampPrefix config for datetime awareness
New config options:
- timestampPrefix: boolean - prepend timestamp to messages
- timestampTimezone: string - IANA timezone (default: UTC)

Format: [Nov 29 06:30] - compact but informative
Helps AI assistants stay aware of current date/time.
2025-11-29 05:25:53 +00:00
Peter Steinberger
25ec133574 Add samePhoneResponsePrefix config option
Automatically prefixes responses with a configurable string when in
same-phone mode. This helps distinguish bot replies from user messages
in the same chat bubble.

Example config:
  "samePhoneResponsePrefix": "🦞"

Will prefix all same-phone replies with the lobster emoji.
2025-11-29 05:24:01 +00:00
Peter Steinberger
d88ede92b9 feat: same-phone mode with echo detection and configurable marker
Adds full support for self-messaging setups where you chat with yourself
and an AI assistant replies in the same WhatsApp bubble.

Changes:
- Same-phone mode (from === to) always allowed, bypasses allowFrom check
- Echo detection via bounded Set (max 100) prevents infinite loops
- Configurable samePhoneMarker in config (default: "[same-phone]")
- Messages prefixed with marker so assistants know the context
- fromMe filter removed from inbound.ts (echo detection in auto-reply)
- Verbose logging for same-phone detection and echo skips

Tests:
- Same-phone allowed without/despite allowFrom configuration
- Body prefixed only when from === to
- Non-same-phone rejected when not in allowFrom
2025-11-29 04:52:21 +00:00
Peter Steinberger
5bafe9483d chore: release 1.2.2 2025-11-28 08:17:22 +01:00
Peter Steinberger
4e3663b4d4 chore: move heartbeat notes to unreleased 1.2.2 2025-11-28 08:14:51 +01:00
Peter Steinberger
12d7be7cad feat(heartbeat): allow manual message and dry-run for web/twilio 2025-11-28 08:14:07 +01:00
Peter Steinberger
84f2595349 docs: note changelog not needed for pure tests 2025-11-28 08:13:59 +01:00
Peter Steinberger
c11abc1134 chore: release 1.2.1 2025-11-28 08:11:07 +01:00
Peter Steinberger
f63bdda628 docs: document mime-first media handling 2025-11-28 08:07:53 +01:00
Peter Steinberger
7d6a4f5204 fix(media): sniff mime and keep extensions 2025-11-28 08:07:53 +01:00
Peter Steinberger
f871869c79 Fix broken link: claude-config.md -> clawd.md 2025-11-28 05:19:43 +00:00
Peter Steinberger
8ebe72951f docs: Add Twitter automation and music recognition examples
- Added Twitter automation patterns using Peekaboo + AppleScript
- Documented JS injection for reliable button clicks on Twitter's dynamic UI
- Added audd.io music recognition API example
- These are the techniques Clawd uses to reply to tweets autonomously
2025-11-27 21:00:28 +00:00
Peter Steinberger
8d4b31a301 Expand heartbeat capabilities in docs 2025-11-27 19:09:30 +01:00
Peter Steinberger
8912b3e035 Rename claude-config.md to clawd.md, update credits
- Renamed docs/claude-config.md → docs/clawd.md
- Credits now include Clawd (they/them) as co-author
2025-11-27 19:07:35 +01:00
Peter Steinberger
f5d7057042 Add browser-tools CLI and example tweets to docs
- Added browser-tools to CLI tools table (lightweight DevTools CLI)
- Added browser-tools usage section for web scraping
- Added "See It In Action" section with 3 example tweets
- Links to agent-scripts repo
2025-11-27 18:59:01 +01:00
Peter Steinberger
6d7e620430 Release 1.2.0 2025-11-27 18:52:26 +01:00
Peter Steinberger
0cc732dce3 Docs: refresh 1.2.0 changelog; fix webhook host import 2025-11-27 18:46:46 +01:00
Peter Steinberger
8acd82aa0d Add gowa WhatsApp MCP to power user add-ons 2025-11-27 18:45:05 +01:00
Peter Steinberger
7377c676fd Add WhatsApp screenshot to claude-config.md
Shows Clawd in action in the "Meet Clawd" section
2025-11-27 18:43:24 +01:00
Peter Steinberger
9b3c4db10d Heartbeat defaults and ws guard; format 2025-11-27 18:37:30 +01:00
Peter Steinberger
49ada54f6d Docs: add useful CLI tools section (spotify-player, TTS, etc.) 2025-11-27 18:33:38 +01:00
Peter Steinberger
c43cdc5ac3 Docs: new Clawd session intro with personality and powers 2025-11-27 18:32:47 +01:00
Peter Steinberger
e1bd9976b3 Docs: explain two-phone setup for dedicated AI number 2025-11-27 18:29:41 +01:00
Peter Steinberger
a888564251 Docs: mention Claude Code reuses existing subscription 2025-11-27 18:28:51 +01:00
Peter Steinberger
e2ccde6434 Fix: warelay lowercase 2025-11-27 18:27:09 +01:00
Peter Steinberger
e88ff78816 Add Peekaboo and mcporter links to recommended tools 2025-11-27 18:26:40 +01:00
Peter Steinberger
5bc151fdca Redact phone number from example config 2025-11-27 18:24:12 +01:00
Peter Steinberger
f0a5cdc6e4 Add warning disclaimer to claude-config.md 2025-11-27 18:23:56 +01:00
Peter Steinberger
85f53a4174 Fix WebSocket crash + heartbeat default 10min + docs refresh
- Wrap Baileys connection.update listeners in try-catch to prevent
  unhandled exceptions from crashing the relay process
- Add WebSocket-level error handlers in session.ts
- Add global unhandledRejection/uncaughtException handlers in index.ts
- Make listener.onClose error-safe with .catch() in auto-reply.ts
- Change default heartbeat from 30min to 10min
- Rewrite claude-config.md with personality, better explain personal
  assistant features, add recommended MCPs section
2025-11-27 18:21:14 +01:00
Peter Steinberger
549ad272fc Docs: link Clawd setup and current config 2025-11-27 18:17:17 +01:00
Peter Steinberger
537348d995 Update README.md 2025-11-27 18:14:54 +01:00
Peter Steinberger
d4580d1a31 Fix CI: type gaps and hasMedia check 2025-11-27 18:14:20 +01:00
Peter Steinberger
93a103dde5 Tests: cover identity prefix gating 2025-11-27 04:40:03 +01:00
Peter Steinberger
9e6ad97cfb Claude prompt: only prepend on first turn 2025-11-27 03:53:13 +01:00
Peter Steinberger
8d995a8529 Heartbeat: add ultrathink marker 2025-11-27 03:15:51 +01:00
Peter Steinberger
f869cd4b79 Heartbeat: shorten prompt to token 2025-11-27 02:48:23 +01:00
Peter Steinberger
26b087c1b4 Heartbeat: honor session override 2025-11-26 18:32:25 +01:00
Peter Steinberger
63bf4683c5 Heartbeat: allow session-id override (with test) 2025-11-26 18:28:02 +01:00
Peter Steinberger
73456a68d7 Fix heartbeat CLI import for recipients resolution 2025-11-26 18:22:28 +01:00
Peter Steinberger
aa6637b47a Heartbeat: session-id override and safer fallback 2025-11-26 18:19:54 +01:00
Peter Steinberger
8f6e43fd66 Changelog: bump to 1.2.0 unreleased 2025-11-26 18:18:13 +01:00
Peter Steinberger
ebce6ef263 Docs: show --all heartbeat example 2025-11-26 18:17:30 +01:00
Peter Steinberger
c20a266a11 Heartbeat: harden targeting and support lid mapping 2025-11-26 18:15:57 +01:00
Marcus Neves
b825f141f3 fix: add @lid format support and allowFrom wildcard handling
- Add support for WhatsApp Linked ID (@lid) format in jidToE164()
- Use existing lid-mapping-*_reverse.json files for LID resolution
- Fix allowFrom wildcard '*' to actually allow all senders
- Maintain backward compatibility with @s.whatsapp.net format

Fixes issues where:
- Messages from newer WhatsApp versions are silently dropped
- allowFrom: ['*'] configuration doesn't work as documented
2025-11-26 18:03:12 +01:00
Peter Steinberger
7e5b3958cc CLI: rename heartbeat tmux helper and log file path 2025-11-26 18:00:23 +01:00
Peter Steinberger
deded848ee Heartbeat: add relay helper and fix CLI tests 2025-11-26 17:49:34 +01:00
Peter Steinberger
117161e6ff docs: document heartbeat idle override and tests 2025-11-26 17:31:56 +01:00
Peter Steinberger
98d52edcc9 test: cover heartbeat skip preserving session timestamp 2025-11-26 17:29:12 +01:00
Peter Steinberger
135d930c99 feat: add heartbeat idle override and preserve session freshness 2025-11-26 17:26:17 +01:00
Peter Steinberger
e6c78df975 chore: add verbose heartbeat session logging 2025-11-26 17:21:59 +01:00
Peter Steinberger
3749797434 chore: log heartbeat session snapshot 2025-11-26 17:20:48 +01:00
Peter Steinberger
507ed25289 chore: log heartbeat fallback and add test 2025-11-26 17:12:28 +01:00
Peter Steinberger
0d5e5f8dee fix: heartbeat falls back to last session contact 2025-11-26 17:08:43 +01:00
Peter Steinberger
3998933b30 docs: document heartbeat triggers 2025-11-26 17:05:09 +01:00
Peter Steinberger
271004bf60 feat: add heartbeat cli and relay trigger 2025-11-26 17:04:43 +01:00
Peter Steinberger
c9e2d69bfb docs: open 1.1.x unreleased section 2025-11-26 03:33:44 +01:00
Peter Steinberger
c194247dab test(auto-reply): cover cwd timeout hint and queue meta 2025-11-26 03:03:13 +01:00
Peter Steinberger
a48420d85f docs: finalize web refactor and coverage 2025-11-26 02:54:43 +01:00
Peter Steinberger
5c66e8273b chore: update changelog and surface web relay settings 2025-11-26 02:43:24 +01:00
Peter Steinberger
5992e629c3 web: add reconnect logging + troubleshooting doc 2025-11-26 02:41:10 +01:00
Peter Steinberger
765d67cd18 web: extract reconnect helpers and add tests 2025-11-26 02:39:31 +01:00
Peter Steinberger
baf20af17f web: add heartbeat and bounded reconnect tuning 2025-11-26 02:34:43 +01:00
Peter Steinberger
e482e7768b chore: commit pending cli/web test tweaks 2025-11-26 02:19:45 +01:00
Peter Steinberger
8682352edb docs: trim changelog to user-facing auto-reply changes 2025-11-26 02:19:21 +01:00
Peter Steinberger
ef1222ff31 chore: drop refactor note 2025-11-26 02:18:57 +01:00
Peter Steinberger
0145f3a585 docs: note auto-reply helper split 2025-11-26 02:18:39 +01:00
Peter Steinberger
4a8bb56a1e chore(auto-reply): include cwd in timeout message 2025-11-26 02:18:16 +01:00
Peter Steinberger
ce5b02a9ad test(auto-reply): add helper coverage and docs 2025-11-26 02:09:50 +01:00
Peter Steinberger
5c8ce41e12 refactor(auto-reply): split reply helpers 2025-11-26 02:03:51 +01:00
Peter Steinberger
a2586b8b06 feat(web): add logout command and tests 2025-11-26 01:29:02 +01:00
Peter Steinberger
1fd4485716 Auto-reply: refresh typing indicator every 8s 2025-11-26 01:27:51 +01:00
Peter Steinberger
b029ab933e chore(tests): organize web test imports 2025-11-26 01:24:34 +01:00
Peter Steinberger
e0b28b6718 test(web): split provider web suite 2025-11-26 01:23:34 +01:00
Peter Steinberger
4dd2f3b7f7 refactor(web): split provider module 2025-11-26 01:16:54 +01:00
Peter Steinberger
e5f677803f chore: format to 2-space and bump changelog 2025-11-26 00:53:53 +01:00
Peter Steinberger
a67f4db5e2 chore: format + lint 2025-11-26 00:30:30 +01:00
Peter Steinberger
8a01dc7f4c style: normalize indentation to 2 spaces 2025-11-26 00:15:10 +01:00
Peter Steinberger
e107f115e2 chore: bump version to 1.1.0 2025-11-26 00:11:42 +01:00
Peter Steinberger
af8af4881b docs/tests: typing interval docs and coverage 2025-11-26 00:10:38 +01:00
Peter Steinberger
d871dad85f feat: keep typing indicators alive during commands 2025-11-26 00:05:11 +01:00
Peter Steinberger
5b83d30887 test: cover sendSystemOnce default 2025-11-25 23:57:41 +01:00
Peter Steinberger
2e3b8a03aa feat: send session prompt once 2025-11-25 23:52:38 +01:00
Peter Steinberger
d924b7d283 docs: document media caps and tidy web tests 2025-11-25 23:43:57 +01:00
Peter Steinberger
e0425ad3e1 feat: support audio/video/doc media caps and transcript context 2025-11-25 23:21:35 +01:00
Peter Steinberger
5dced02a20 docs: clarify transcript prompt and config 2025-11-25 23:14:23 +01:00
Peter Steinberger
e642f128ae feat: transcribe audio and surface transcript to prompts 2025-11-25 23:13:22 +01:00
Peter Steinberger
7d0ae151e8 feat: optional audio transcription via CLI 2025-11-25 23:06:54 +01:00
Peter Steinberger
f945e284e1 test: cover media formats + doc resize cap 2025-11-25 22:23:06 +01:00
Peter Steinberger
7166efef08 docs: document media resize config 2025-11-25 22:16:09 +01:00
Peter Steinberger
a81689e902 chore: approve build scripts 2025-11-25 20:11:36 +01:00
Peter Steinberger
0a0418b973 web: compress auto-reply media 2025-11-25 20:09:03 +01:00
1080 changed files with 192120 additions and 7466 deletions

View File

@@ -7,20 +7,56 @@ on:
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
runtime: [node, bun]
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: false
- name: Checkout submodules (retry)
run: |
set -euo pipefail
git submodule sync --recursive
for attempt in 1 2 3 4 5; do
if git -c protocol.version=2 submodule update --init --force --depth=1 --recursive; then
exit 0
fi
echo "Submodule update failed (attempt $attempt/5). Retrying…"
sleep $((attempt * 10))
done
exit 1
- name: Setup Node.js
if: matrix.runtime == 'node'
uses: actions/setup-node@v4
with:
node-version: 22
check-latest: true
- name: Node version
- name: Setup Bun
if: matrix.runtime == 'bun'
uses: oven-sh/setup-bun@v2
with:
# bun.sh downloads currently fail with:
# "Failed to list releases from GitHub: 401" -> "Unexpected HTTP response: 400"
bun-download-url: "https://github.com/oven-sh/bun/releases/latest/download/bun-linux-x64.zip"
- name: Setup Node.js (tooling for bun)
if: matrix.runtime == 'bun'
uses: actions/setup-node@v4
with:
node-version: 22
check-latest: true
- name: Runtime versions
run: |
node -v
npm -v
if [ "${{ matrix.runtime }}" = "bun" ]; then bun -v; fi
- name: Capture node path
run: echo "NODE_BIN=$(dirname \"$(node -p \"process.execPath\")\")" >> "$GITHUB_ENV"
@@ -41,11 +77,312 @@ jobs:
pnpm -v
pnpm install --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true || pnpm install --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true
- name: Lint
- name: Lint (node)
if: matrix.runtime == 'node'
run: pnpm lint
- name: Test
- name: Test (node)
if: matrix.runtime == 'node'
run: pnpm test
- name: Build
- name: Build (node)
if: matrix.runtime == 'node'
run: pnpm build
- name: Protocol check (node)
if: matrix.runtime == 'node'
run: pnpm protocol:check
- name: Lint (bun)
if: matrix.runtime == 'bun'
run: bunx biome check src
- name: Test (bun)
if: matrix.runtime == 'bun'
run: bunx vitest run
- name: Build (bun)
if: matrix.runtime == 'bun'
run: bunx tsc -p tsconfig.json
macos-app:
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: false
- name: Checkout submodules (retry)
run: |
set -euo pipefail
git submodule sync --recursive
for attempt in 1 2 3 4 5; do
if git -c protocol.version=2 submodule update --init --force --depth=1 --recursive; then
exit 0
fi
echo "Submodule update failed (attempt $attempt/5). Retrying…"
sleep $((attempt * 10))
done
exit 1
- name: Select Xcode 26.1
run: |
sudo xcode-select -s /Applications/Xcode_26.1.app
xcodebuild -version
- name: Install XcodeGen / SwiftLint / SwiftFormat
run: |
brew install xcodegen swiftlint swiftformat
- name: Show toolchain
run: |
sw_vers
xcodebuild -version
swift --version
- name: SwiftLint
run: swiftlint --config .swiftlint.yml
- name: SwiftFormat (lint mode)
run: swiftformat --lint apps/macos/Sources --config .swiftformat
- name: Swift build (release)
run: |
set -euo pipefail
for attempt in 1 2 3; do
if swift build --package-path apps/macos --configuration release; then
exit 0
fi
echo "swift build failed (attempt $attempt/3). Retrying…"
sleep $((attempt * 20))
done
exit 1
- name: Swift tests (coverage)
run: |
set -euo pipefail
for attempt in 1 2 3; do
if swift test --package-path apps/macos --parallel --enable-code-coverage --show-codecov-path; then
exit 0
fi
echo "swift test failed (attempt $attempt/3). Retrying…"
sleep $((attempt * 20))
done
exit 1
ios:
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: false
- name: Checkout submodules (retry)
run: |
set -euo pipefail
git submodule sync --recursive
for attempt in 1 2 3 4 5; do
if git -c protocol.version=2 submodule update --init --force --depth=1 --recursive; then
exit 0
fi
echo "Submodule update failed (attempt $attempt/5). Retrying…"
sleep $((attempt * 10))
done
exit 1
- name: Select Xcode 26.1
run: |
sudo xcode-select -s /Applications/Xcode_26.1.app
xcodebuild -version
- name: Install XcodeGen
run: brew install xcodegen
- name: Install SwiftLint / SwiftFormat
run: brew install swiftlint swiftformat
- name: Show toolchain
run: |
sw_vers
xcodebuild -version
swift --version
- name: Generate iOS project
run: |
cd apps/ios
xcodegen generate
- name: iOS tests
run: |
set -euo pipefail
RESULT_BUNDLE_PATH="$RUNNER_TEMP/Clawdis-iOS.xcresult"
DEST_ID="$(
python3 - <<'PY'
import json
import subprocess
import sys
import uuid
def sh(args: list[str]) -> str:
return subprocess.check_output(args, text=True).strip()
# Prefer an already-created iPhone simulator if it exists.
devices = json.loads(sh(["xcrun", "simctl", "list", "devices", "-j"]))
candidates: list[tuple[str, str]] = []
for runtime, devs in (devices.get("devices") or {}).items():
for dev in devs or []:
if not dev.get("isAvailable"):
continue
name = str(dev.get("name") or "")
udid = str(dev.get("udid") or "")
if not udid or not name.startswith("iPhone"):
continue
candidates.append((name, udid))
candidates.sort(key=lambda it: (0 if "iPhone 16" in it[0] else 1, it[0]))
if candidates:
print(candidates[0][1])
sys.exit(0)
# Otherwise, create one from the newest available iOS runtime.
runtimes = json.loads(sh(["xcrun", "simctl", "list", "runtimes", "-j"])).get("runtimes") or []
ios = [rt for rt in runtimes if rt.get("platform") == "iOS" and rt.get("isAvailable")]
if not ios:
print("No available iOS runtimes found.", file=sys.stderr)
sys.exit(1)
def version_key(rt: dict) -> tuple[int, ...]:
parts: list[int] = []
for p in str(rt.get("version") or "0").split("."):
try:
parts.append(int(p))
except ValueError:
parts.append(0)
return tuple(parts)
ios.sort(key=version_key, reverse=True)
runtime = ios[0]
runtime_id = str(runtime.get("identifier") or "")
if not runtime_id:
print("Missing iOS runtime identifier.", file=sys.stderr)
sys.exit(1)
supported = runtime.get("supportedDeviceTypes") or []
iphones = [dt for dt in supported if dt.get("productFamily") == "iPhone"]
if not iphones:
print("No iPhone device types for iOS runtime.", file=sys.stderr)
sys.exit(1)
iphones.sort(
key=lambda dt: (
0 if "iPhone 16" in str(dt.get("name") or "") else 1,
str(dt.get("name") or ""),
)
)
device_type_id = str(iphones[0].get("identifier") or "")
if not device_type_id:
print("Missing iPhone device type identifier.", file=sys.stderr)
sys.exit(1)
sim_name = f"CI iPhone {uuid.uuid4().hex[:8]}"
udid = sh(["xcrun", "simctl", "create", sim_name, device_type_id, runtime_id])
if not udid:
print("Failed to create iPhone simulator.", file=sys.stderr)
sys.exit(1)
print(udid)
PY
)"
echo "Using iOS Simulator id: $DEST_ID"
xcodebuild test \
-project apps/ios/Clawdis.xcodeproj \
-scheme Clawdis \
-destination "platform=iOS Simulator,id=$DEST_ID" \
-resultBundlePath "$RESULT_BUNDLE_PATH" \
-enableCodeCoverage YES
- name: iOS coverage summary
run: |
set -euo pipefail
RESULT_BUNDLE_PATH="$RUNNER_TEMP/Clawdis-iOS.xcresult"
xcrun xccov view --report --only-targets "$RESULT_BUNDLE_PATH"
- name: iOS coverage gate (43%)
run: |
set -euo pipefail
RESULT_BUNDLE_PATH="$RUNNER_TEMP/Clawdis-iOS.xcresult"
RESULT_BUNDLE_PATH="$RESULT_BUNDLE_PATH" python3 - <<'PY'
import json
import os
import subprocess
import sys
target_name = "Clawdis.app"
minimum = 0.43
report = json.loads(
subprocess.check_output(
["xcrun", "xccov", "view", "--report", "--json", os.environ["RESULT_BUNDLE_PATH"]],
text=True,
)
)
target_coverage = None
for target in report.get("targets", []):
if target.get("name") == target_name:
target_coverage = float(target["lineCoverage"])
break
if target_coverage is None:
print(f"Could not find coverage for target: {target_name}")
sys.exit(1)
print(f"{target_name} line coverage: {target_coverage * 100:.2f}% (min {minimum * 100:.2f}%)")
if target_coverage + 1e-12 < minimum:
sys.exit(1)
PY
android:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: false
- name: Checkout submodules (retry)
run: |
set -euo pipefail
git submodule sync --recursive
for attempt in 1 2 3 4 5; do
if git -c protocol.version=2 submodule update --init --force --depth=1 --recursive; then
exit 0
fi
echo "Submodule update failed (attempt $attempt/5). Retrying…"
sleep $((attempt * 10))
done
exit 1
- name: Setup Java
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 21
- name: Setup Android SDK
uses: android-actions/setup-android@v3
- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4
- name: Install Android SDK packages
run: |
yes | sdkmanager --licenses >/dev/null
sdkmanager --install \
"platform-tools" \
"platforms;android-36" \
"build-tools;36.0.0"
- name: Android unit tests + debug build
working-directory: apps/android
run: ./gradlew --no-daemon :app:testDebugUnitTest :app:assembleDebug

35
.gitignore vendored
View File

@@ -4,3 +4,38 @@ dist
pnpm-lock.yaml
coverage
.pnpm-store
.worktrees/
.DS_Store
**/.DS_Store
# Bun build artifacts
*.bun-build
apps/macos/.build/
apps/shared/ClawdisKit/.build/
bin/clawdis-mac
bin/docs-list
apps/macos/.build-local/
apps/macos/.swiftpm/
apps/shared/ClawdisKit/.swiftpm/
Core/
apps/ios/*.xcodeproj/
apps/ios/*.xcworkspace/
apps/ios/.swiftpm/
# Vendor build artifacts
vendor/a2ui/renderers/lit/dist/
# fastlane (iOS)
apps/ios/fastlane/README.md
apps/ios/fastlane/report.xml
apps/ios/fastlane/Preview.html
apps/ios/fastlane/screenshots/
apps/ios/fastlane/test_output/
apps/ios/fastlane/logs/
# fastlane build artifacts (local)
apps/ios/*.ipa
apps/ios/*.dSYM.zip
# provisioning profiles (local)
apps/ios/*.mobileprovision

4
.gitmodules vendored Normal file
View File

@@ -0,0 +1,4 @@
[submodule "Peekaboo"]
path = Peekaboo
url = https://github.com/steipete/Peekaboo.git
branch = main

1
.npmrc Normal file
View File

@@ -0,0 +1 @@
allow-build-scripts=@whiskeysockets/baileys,sharp,esbuild,protobufjs,fs-ext

51
.swiftformat Normal file
View File

@@ -0,0 +1,51 @@
# SwiftFormat configuration adapted from Peekaboo defaults (Swift 6 friendly)
--swiftversion 6.2
# Self handling
--self insert
--selfrequired
# Imports / extensions
--importgrouping testable-bottom
--extensionacl on-declarations
# Indentation
--indent 4
--indentcase false
--ifdef no-indent
--xcodeindentation enabled
# Line breaks
--linebreaks lf
--maxwidth 120
# Whitespace
--trimwhitespace always
--emptybraces no-space
--nospaceoperators ...,..<
--ranges no-space
--someAny true
--voidtype void
# Wrapping
--wraparguments before-first
--wrapparameters before-first
--wrapcollections before-first
--closingparen same-line
# Organization
--organizetypes class,struct,enum,extension
--extensionmark "MARK: - %t + %p"
--marktypes always
--markextensions always
--structthreshold 0
--enumthreshold 0
# Other
--stripunusedargs closure-only
--header ignore
--allman false
# Exclusions
--exclude .build,.swiftpm,DerivedData,node_modules,dist,coverage,xcuserdata,apps/macos/Sources/ClawdisProtocol

142
.swiftlint.yml Normal file
View File

@@ -0,0 +1,142 @@
# SwiftLint configuration adapted from Peekaboo defaults (Swift 6 friendly)
included:
- apps/macos/Sources
excluded:
- .build
- DerivedData
- "**/.build"
- "**/.swiftpm"
- "**/DerivedData"
- "**/Generated"
- "**/Resources"
- "**/Package.swift"
- "**/Tests/Resources"
- node_modules
- dist
- coverage
- "*.playground"
analyzer_rules:
- unused_declaration
- unused_import
opt_in_rules:
- array_init
- closure_spacing
- contains_over_first_not_nil
- empty_count
- empty_string
- explicit_init
- fallthrough
- fatal_error_message
- first_where
- joined_default_parameter
- last_where
- literal_expression_end_indentation
- multiline_arguments
- multiline_parameters
- operator_usage_whitespace
- overridden_super_call
- pattern_matching_keywords
- private_outlet
- prohibited_super_call
- redundant_nil_coalescing
- sorted_first_last
- switch_case_alignment
- unneeded_parentheses_in_closure_argument
- vertical_parameter_alignment_on_call
disabled_rules:
# SwiftFormat handles these
- trailing_whitespace
- trailing_newline
- trailing_comma
- vertical_whitespace
- indentation_width
# Style exclusions
- explicit_self
- identifier_name
- file_header
- explicit_top_level_acl
- explicit_acl
- explicit_type_interface
- missing_docs
- required_deinit
- prefer_nimble
- quick_discouraged_call
- quick_discouraged_focused_test
- quick_discouraged_pending_test
- anonymous_argument_in_multiline_closure
- no_extension_access_modifier
- no_grouping_extension
- switch_case_on_newline
- strict_fileprivate
- extension_access_modifier
- convenience_type
- no_magic_numbers
- one_declaration_per_file
- vertical_whitespace_between_cases
- vertical_whitespace_closing_braces
- superfluous_else
- number_separator
- prefixed_toplevel_constant
- opening_brace
- trailing_closure
- contrasted_opening_brace
- sorted_imports
- redundant_type_annotation
- shorthand_optional_binding
- untyped_error_in_catch
- file_name
- todo
force_cast: warning
force_try: warning
type_name:
min_length:
warning: 2
error: 1
max_length:
warning: 60
error: 80
function_body_length:
warning: 150
error: 300
file_length:
warning: 1500
error: 2500
ignore_comment_only_lines: true
type_body_length:
warning: 800
error: 1200
cyclomatic_complexity:
warning: 20
error: 120
large_tuple:
warning: 4
error: 5
nesting:
type_level:
warning: 4
error: 6
function_level:
warning: 5
error: 7
line_length:
warning: 120
error: 250
ignores_comments: true
ignores_urls: true
reporter: "xcode"

View File

@@ -1,13 +1,13 @@
# Repository Guidelines
## Project Structure & Module Organization
- Source code: `src/` (CLI wiring in `src/cli`, commands in `src/commands`, Twilio in `src/twilio`, Web provider in `src/provider-web.ts`, infra in `src/infra`, media pipeline in `src/media`).
- Tests: colocated `*.test.ts` plus e2e in `src/cli/relay.e2e.test.ts`.
- Docs: `docs/` (images, queue, Claude config). Built output lives in `dist/`.
- Source code: `src/` (CLI wiring in `src/cli`, commands in `src/commands`, web provider in `src/provider-web.ts`, infra in `src/infra`, media pipeline in `src/media`).
- Tests: colocated `*.test.ts`.
- Docs: `docs/` (images, queue, Pi config). Built output lives in `dist/`.
## Build, Test, and Development Commands
- Install deps: `pnpm install`
- Run CLI in dev: `pnpm warelay ...` (tsx entry) or `pnpm dev` for `src/index.ts`.
- Run CLI in dev: `pnpm clawdis ...` (tsx entry) or `pnpm dev` for `src/index.ts`.
- Type-check/build: `pnpm build` (tsc)
- Lint/format: `pnpm lint` (biome check), `pnpm format` (biome format)
- Tests: `pnpm test` (vitest); coverage: `pnpm test:coverage`
@@ -16,22 +16,56 @@
- Language: TypeScript (ESM). Prefer strict typing; avoid `any`.
- Formatting/linting via Biome; run `pnpm lint` before commits.
- Keep files concise; extract helpers instead of “V2” copies. Use existing patterns for CLI options and dependency injection via `createDefaultDeps`.
- Keep every file ≤ 500 LOC; refactor or split before exceeding and check frequently.
## Testing Guidelines
- Framework: Vitest with V8 coverage thresholds (70% lines/branches/functions/statements).
- Naming: match source names with `*.test.ts`; e2e in `*.e2e.test.ts`.
- Run `pnpm test` (or `pnpm test:coverage`) before pushing when you touch logic.
- Pure test additions/fixes generally do **not** need a changelog entry unless they alter user-facing behavior or the user asks for one.
## Commit & Pull Request Guidelines
- Create commits with `scripts/committer "<msg>" <file...>`; avoid manual `git add`/`git commit` so staging stays scoped.
- Follow concise, action-oriented commit messages (e.g., `CLI: add verbose flag to send`).
- Group related changes; avoid bundling unrelated refactors.
- PRs should summarize scope, note testing performed, and mention any user-facing changes or new flags.
## Security & Configuration Tips
- Environment: copy `.env.example`; set Twilio creds and WhatsApp sender (`TWILIO_WHATSAPP_FROM`).
- Web provider stores creds at `~/.warelay/credentials/`; rerun `warelay login` if logged out.
- Media hosting relies on Tailscale Funnel when using Twilio; use `warelay webhook --ingress tailscale` or `--serve-media` for local hosting.
- Web provider stores creds at `~/.clawdis/credentials/`; rerun `clawdis login` if logged out.
- Pi sessions live under `~/.clawdis/sessions/` by default; the base directory is not configurable.
- Never commit or publish real phone numbers, videos, or live configuration values. Use obviously fake placeholders in docs, tests, and examples.
## Agent-Specific Notes
- If the relay is running in tmux (`warelay-relay`), restart it after code changes: kill pane/session and run `pnpm warelay relay --verbose` inside tmux. Check tmux before editing; keep the watcher healthy if you start it.
- Gateway currently runs only as the menubar app (launchctl shows `application.com.steipete.clawdis.debug.*`), there is no separate LaunchAgent/helper label installed. Restart via the Clawdis Mac app or `scripts/restart-mac.sh`; to verify/kill use `launchctl print gui/$UID | grep clawdis` rather than expecting `com.steipete.clawdis`. **When debugging on macOS, start/stop the gateway via the app, not ad-hoc tmux sessions; kill any temporary tunnels before handoff.**
- macOS logs: use `./scripts/clawlog.sh` (aka `vtlog`) to query unified logs for subsystem `com.steipete.clawdis`; it supports follow/tail/category filters and expects passwordless sudo for `/usr/bin/log`.
- Also read the shared guardrails at `~/Projects/oracle/AGENTS.md` and `~/Projects/agent-scripts/AGENTS.MD` before making changes; align with any cross-repo rules noted there.
- SwiftUI state management (iOS/macOS): prefer the `Observation` framework (`@Observable`, `@Bindable`) over `ObservableObject`/`@StateObject`; dont introduce new `ObservableObject` unless required for compatibility, and migrate existing usages when touching related code.
- **Restart apps:** “restart iOS/Android apps” means rebuild (recompile/install) and relaunch, not just kill/launch.
- Notary key file lives at `~/Library/CloudStorage/Dropbox/Backup/AppStore/AuthKey_NJF3NFGTS3.p8` (Sparkle keys live under `~/Library/CloudStorage/Dropbox/Backup/Sparkle`).
- **Multi-agent safety:** do **not** create/apply/drop `git stash` entries unless Peter explicitly asks (this includes `git pull --rebase --autostash`). Assume other agents may be working; keep unrelated WIP untouched and avoid cross-cutting state changes.
- **Multi-agent safety:** when Peter says "push", you may `git pull --rebase` to integrate latest changes (never discard other agents' work). When Peter says "commit", scope to your changes only. When Peter says "commit all", commit everything in grouped chunks.
- **Multi-agent safety:** do **not** create/remove/modify `git worktree` checkouts (or edit `.worktrees/*`) unless Peter explicitly asks.
- **Multi-agent safety:** do **not** switch branches / check out a different branch unless Peter explicitly asks.
- When asked to open a “session” file, open the Pi session logs under `~/.clawdis/sessions/*.jsonl` (newest unless a specific ID is given), not the default `sessions.json`. If logs are needed from Mac Studio, SSH via Tailscale and read the same path there.
- Menubar dimming + restart flow mirrors Trimmy: use `scripts/restart-mac.sh` (kills all Clawdis variants, runs `swift build`, packages, relaunches). Icon dimming depends on MenuBarExtraAccess wiring in AppMain; keep `appearsDisabled` updates intact when touching the status item.
- Never send streaming/partial replies to external messaging surfaces (WhatsApp, Telegram); only final replies should be delivered there. Streaming/tool events may still go to internal UIs/control channel.
- Voice wake forwarding tips:
- Command template should stay `clawdis-mac agent --message "${text}" --thinking low`; `VoiceWakeForwarder` already shell-escapes `${text}`. Dont add extra quotes.
- launchd PATH is minimal; ensure the apps launch agent sets PATH to include `/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/steipete/Library/pnpm` so `pnpm`/`clawdis` binaries resolve when invoked via `clawdis-mac`.
- For manual `clawdis send` messages that include `!`, use the heredoc pattern noted below to avoid the Bash tools escaping.
## Exclamation Mark Escaping Workaround
The Claude Code Bash tool escapes `!` to `\\!` in command arguments. When using `clawdis send` with messages containing exclamation marks, use heredoc syntax:
```bash
# WRONG - will send "Hello\\!" with backslash
clawdis send --to "+1234" --message 'Hello!'
# CORRECT - use heredoc to avoid escaping
clawdis send --to "+1234" --message "$(cat <<'EOF'
Hello!
EOF
)"
```
This is a Claude Code quirk, not a clawdis bug.

View File

@@ -1,69 +1,232 @@
# Changelog
## [Unreleased] 1.0.5
## Unreleased — 2025-12-23
### Pending
- (add entries here)
### Fixes
- Telegram/WhatsApp: native replies now target the original inbound message; reply context is captured in `ReplyTo*` fields for templates. (Thanks @joshp123 for the PR and follow-up question.)
## 2.0.0-beta2 — 2025-12-21
Second beta focused on bundled gateway packaging, skills management, onboarding polish, and provider reliability.
### Highlights
- Bundled gateway packaging: bun-compiled embedded gateway, new `gateway-daemon` command, launchd support, DMG packaging (zip+DMG).
- Skills platform: managed/bundled skills, install metadata + installers (uv), skill search + website, media/transcription helpers.
- macOS app: new Connections settings w/ provider status + QR login, skills settings redesign w/ install targets, models list loaded from the Gateway, clearer local/remote gateway choices.
- Web/agent UX: tool summary streaming + runtime toggle, WhatsApp QR login tool, agent steering queue, voice wake routes to main session, workspace bootstrap ritual.
### Gateway & providers
- Gateway: `models.list`, provider status events + RPC coverage, tailscale auth + PAM, bind-mode config, enriched agent WS logs, safer upgrade socket handling, fixed handshake auth crash.
- WhatsApp Web: QR login flow improvements (logged-out clearing, wait flow), self-chat mode handling, removed batching delay, web inbox made non-blocking.
- Telegram: normalized chat IDs with clearer error reporting.
### Canvas & browser control
- Canvas host served on Gateway port; removed standalone canvasHost port config; restored action bridge; refreshed A2UI bundle + message context; bridge canvas host for nodes.
- A2UI full-screen gutters + status clearance after successful load to avoid overlay collisions.
- Browser control API simplified; added MCP tool dispatch + native actions; control server can start without Playwright; hook timeouts extended.
### macOS UI polish
- Onboarding chat UI: kickoff flow, bubble tails, spacing + bottom bar refinements, window sizing tweaks, show Dock icon during onboarding.
- Skills UI: stabilized action column, fixed install target access, refined list layout and sizing, always show CLI installer.
- Remote/local gateway: auto-enable local gateway, clearer labels, re-ensure remote tunnel, hide local bridge discovery in remote mode.
### Build, CI, deps
- Bundled playwright-core + chromium-bidi/long; bun gateway bytecode builds; swiftformat/biome CI fixes; iOS lint script updates; Android icon/compiler updates; ignored new ClawdisKit `.swiftpm` path.
### Docs
- README architecture refresh + npm header image fix; onboarding/bootstrap steps; skills install guidance + new skills; browser/canvas control docs; bundled gateway + DMG packaging notes.
## 2.0.0-beta1 — 2025-12-19
First Clawdis release post rebrand. This is a semver-major because we dropped legacy providers/agents and moved defaults to new paths while adding a full macOS companion app, a WebSocket Gateway, and an iOS node.
### Bug Fixes
- macOS: Voice Wake / push-to-talk no longer initialize `AVAudioEngine` at app launch, preventing Bluetooth headphones from switching into headset profile when voice features are unused. (Thanks @Nachx639)
### Breaking
- Renamed to **Clawdis**: defaults now live under `~/.clawdis` (sessions in `~/.clawdis/sessions/`, IPC at `~/.clawdis/clawdis.sock`, logs in `/tmp/clawdis`). Launchd labels and config filenames follow the new name; legacy stores are copied forward on first run.
- Pi only: `inbound.reply.agent.kind` accepts only `"pi"`, and the agent CLI/CLI flags for Claude/Codex/Gemini were removed. The Pi CLI runs in RPC mode with a persistent worker.
- WhatsApp Web is the only transport; Twilio support and related CLI flags/tests were removed.
- Direct chats now collapse into a single `main` session by default (no config needed); groups stay isolated as `group:<jid>`.
- Gateway is now a loopback-only WebSocket daemon (`ws://127.0.0.1:18789`) that owns all providers/state; clients (CLI, WebChat, macOS app, nodes) connect to it. Start it explicitly (`clawdis gateway …`) or via Clawdis.app; helper subcommands no longer auto-spawn a gateway.
### Gateway, nodes, and automation
- New typed Gateway WS protocol (JSON schema validated) with `clawdis gateway {health,status,send,agent,call}` helpers and structured presence/instance updates for all clients.
- Optional LAN-facing bridge (`tcp://0.0.0.0:18790`) keeps the Gateway loopback-only while enabling direct Bonjour-discovered connections for paired nodes.
- Node pairing + management via `clawdis nodes {pending,approve,reject,invoke}` (used by the iOS node and future remote nodes).
- Cron jobs are Gateway-owned (`clawdis cron …`) with run history stored as JSONL and support for “isolated summary” posting into the main session.
### macOS companion app
- **Clawdis.app menu bar companion**: packaged, signed bundle with gateway start/stop, launchd toggle, project-root and pnpm/node auto-resolution, live log shortcut, restart button, and status/recipient table plus badges/dimming for attention and paused states.
- **On-device Voice Wake**: Apple speech recognizer with wake-word table, language picker, live mic meter, “hold until silence,” animated ears/legs, and main-session routing that replies on the **last used surface** (WhatsApp/Telegram/WebChat). Delivery failures are logged, and the run remains visible via WebChat/session logs.
- **WebChat & Debugging**: bundled WebChat UI, Debug tab with heartbeat sliders, session-store picker, log opener (`clawlog`), gateway restart, health probes, and scrollable settings panes.
- **Browser control**: manage clawds dedicated Chrome/Chromium with tab listing/open/focus/close, screenshots, DOM query/dump, and “AI snapshots” (aria/domSnapshot/ai) via `clawdis browser …` and UI controls.
- **Remote gateway control**: Bonjour discovery for local masters plus SSH-tunnel fallback for remote control when multicast is unavailable.
### iOS node
- New iOS companion app that pairs to the Gateway bridge, reports presence as a node, and exposes a WKWebView “Canvas” for agent-driven UI.
- `clawdis nodes invoke` supports `canvas.eval` and `canvas.snapshot` to drive and verify the iOS Canvas (fails fast when the iOS node is backgrounded).
- Voice wake words are configurable in-app; the iOS node reconnects to the last bridge when credentials are still present in Keychain.
### WhatsApp & agent experience
- Group chats fully supported: mention-gated triggers (including media-only captions), sender attribution, session primer with subject/member roster, allowlist bypass when youre @mentioned, and safer handling of view-once/ephemeral media.
- Thinking/verbosity directives: `/think` and `/verbose` acknowledge and persist per session while allowing inline overrides; verbose mode streams tool metadata with emoji/args/previews and coalesces bursts to reduce WhatsApp noise.
- Heartbeats: configurable cadence with CLI/GUI toggles; directive acks suppressed during heartbeats; array/multi-payload replies normalized for Baileys.
- Reply quality: smarter chunking on words/newlines, fallback warnings when media fails to send, self-number mention detection, and primed group sessions send the roster on first turn.
- In-chat `/status`: prints agent readiness, session context usage %, current thinking/verbose options, and when the WhatsApp web creds were refreshed (helps decide when to re-scan QR); still available via `clawdis status` CLI for web session health.
### CLI, RPC, and health
- New `clawdis agent` command plus a persistent Pi RPC worker (auto-started) enables direct agent chats; `clawdis status` renders a colored session/recipient table.
- `clawdis health` probes WhatsApp link status, connect latency, heartbeat interval, session-store recency, and IPC socket presence (JSON mode for monitors).
- Added `--help`/`--version` flags; login/logout accept `--provider` (WhatsApp default). Console output is mirrored into pino logs under `/tmp/clawdis`.
- RPC stability: stdin/stdout loop for Pi, auto-restart worker, raw error surfacing, and deliver-via-RPC when JSON agent output is returned.
### Security & hardening
- Media server blocks symlink/path traversal, clears temporary downloads, and rotates logs daily (24h retention).
- Session store purged on logout; IPC socket directory permissions tightened (0700/0600).
- Launchd PATH and helper lookup hardened for packaged macOS builds; health probes surface missing binaries quickly.
### Docs
- Added `docs/telegram.md` outlining the Telegram Bot API provider (grammY) and how it shares the `main` session. Default grammY throttler keeps Bot API calls under rate limits.
- Gateway can run WhatsApp + Telegram together when configured; `clawdis send --provider telegram …` sends via the Telegram bot (webhook/proxy options documented).
## 1.5.0 — 2025-12-05
### Breaking
- Dropped all non-Pi agents (Claude, Codex, Gemini, Opencode); `inbound.reply.agent.kind` now only accepts `"pi"` and related CLI helpers have been removed.
- Removed Twilio support and all related commands/options (webhook/up/provider flags/wait-poll); CLAWDIS is Baileys Web-only.
### Changes
- Default agent handling now favors Pi RPC while falling back to plain command execution for non-Pi invocations, keeping heartbeat/session plumbing intact.
- Documentation updated to reflect Pi-only support and to mark legacy Claude paths as historical.
- Status command reports web session health + session recipients; config paths are locked to `~/.clawdis` with session metadata stored under `~/.clawdis/sessions/`.
- Simplified send/agent/gateway/heartbeat to web-only delivery; removed Twilio mocks/tests and dead code.
- Pi RPC timeout is now inactivity-based (5m without events) and error messages show seconds only.
- Pi sessions now write to `~/.clawdis/sessions/` by default (legacy session logs from older installs are copied over when present).
- Directive triggers (`/think`, `/verbose`, `/stop` et al.) now reply immediately using normalized bodies (timestamps/group prefixes stripped) without waiting for the agent.
- Directive/system acks carry a `⚙️` prefix and verbose parsing rejects typoed `/ver*` strings so unrelated text doesnt flip verbosity.
- Batched history blocks no longer trip directive parsing; `/think` in prior messages won't emit stray acknowledgements.
- RPC fallbacks no longer echo the user's prompt (e.g., pasting a link) when the agent returns no assistant text.
- Heartbeat prompts with `/think` no longer send directive acks; heartbeat replies stay silent on settings.
- `clawdis sessions` now renders a colored table (a la oracle) with context usage shown in k tokens and percent of the context window.
## 1.4.1 — 2025-12-04
### Changes
- Added `clawdis agent` CLI command to talk directly to the configured agent using existing session handling (no WhatsApp send), with JSON output and delivery option.
- `/new` reset trigger now works even when inbound messages have timestamp prefixes (e.g., `[Dec 4 17:35]`).
- WhatsApp mention parsing accepts nullable arrays and flattens safely to avoid missed mentions.
## 1.4.0 — 2025-12-03
### Highlights
- **Thinking directives & state:** `/t|/think|/thinking <level>` (aliases off|minimal|low|medium|high|max/highest). Inline applies to that message; directive-only message pins the level for the session; `/think:off` clears. Resolution: inline > session override > `inbound.reply.thinkingDefault` > off. Pi gets `--thinking <level>` (except off); other agents append cue words (`think``think hard``think harder``ultrathink`). Heartbeat probe uses `HEARTBEAT /think:high`.
- **Group chats (web provider):** Clawdis now fully supports WhatsApp groups: mention-gated triggers (including image-only @ mentions), recent group history injection, per-group sessions, sender attribution, and a first-turn primer with group subject/member roster; heartbeats are skipped for groups.
- **Group session primer:** The first turn of a group session now tells the agent it is in a WhatsApp group and lists known members/subject so it can address the right speaker.
- **Media failures are surfaced:** When a web auto-reply media fetch/send fails (e.g., HTTP 404), we now append a warning to the fallback text so you know the attachment was skipped.
- **Verbose directives + session hints:** `/v|/verbose on|full|off` mirrors thinking: inline > session > config default. Directive-only replies with an acknowledgement; invalid levels return a hint. When enabled, tool results from JSON-emitting agents (Pi, etc.) are forwarded as metadata-only `[🛠️ <tool-name> <arg>]` messages (now streamed as they happen), and new sessions surface a `🧭 New session: <id>` hint.
- **Verbose tool coalescing:** successive tool results of the same tool within ~1s are batched into one `[🛠️ tool] arg1, arg2` message to reduce WhatsApp noise.
- **Directive confirmations:** Directive-only messages now reply with an acknowledgement (`Thinking level set to high.` / `Thinking disabled.`) and reject unknown levels with a helpful hint (state is unchanged).
- **Pi stability:** RPC replies buffered until the assistant turn finishes; parsers return consistent `texts[]`; web auto-replies keep a warm Pi RPC process to avoid cold starts.
- **Claude prompt flow:** One-time `sessionIntro` with per-message `/think:high` bodyPrefix; system prompt always sent on first turn even with `sendSystemOnce`.
- **Heartbeat UX:** Backpressure skips reply heartbeats while other commands run; skips dont refresh session `updatedAt`; web heartbeats normalize array payloads and optional `heartbeatCommand`.
- **Control via WhatsApp:** Send `/restart` to restart the launchd service (`com.steipete.clawdis`) from your allowed numbers.
- **Pi completion signal:** RPC now resolves on Pis `agent_end` (or process exit) so late assistant messages arent truncated; 5-minute hard cap only as a failsafe.
### Reliability & UX
- Outbound chunking prefers newlines/word boundaries and enforces caps (~4000 chars for web/WhatsApp).
- Web auto-replies fall back to caption-only if media send fails; hosted media MIME-sniffed and cleaned up immediately.
- IPC gateway send shows typing indicator; batched inbound messages keep timestamps; watchdog restarts WhatsApp after long inactivity.
- Early `allowFrom` filtering prevents decryption errors; same-phone mode supported with echo suppression.
- All console output is now mirrored into pino logs (still printed to stdout/stderr), so verbose runs keep full traces.
- `--verbose` now forces log level `trace` (was `debug`) to capture every event.
- Verbose tool messages now include emoji + args + a short result preview for bash/read/edit/write/attach (derived from RPC tool start/end events).
### Security / Hardening
- IPC socket hardened (0700 dir / 0600 socket, no symlinks/foreign owners); `clawdis logout` also prunes session store.
- Media server blocks symlinks and enforces path containment; logging rotates daily and prunes >24h.
### Bug Fixes
- Web group chats now bypass the second `allowFrom` check (we still enforce it on the group participant at inbox ingest), so mentioned group messages reply even when the group JID isnt in your allowlist.
- `logVerbose` also writes to the configured Pino logger at debug level (without breaking stdout).
- Group auto-replies now append the triggering sender (`[from: Name (+E164)]`) to the batch body so agents can address the right person in group chats.
- Media-only pings now pick up mentions inside captions (image/video/etc.), so @-mentions on media-only messages trigger replies.
- MIME sniffing and redirect handling for downloads/hosted media.
- Response prefix applied to heartbeat alerts; heartbeat array payloads handled for both providers.
- Pi RPC typing exposes `signal`/`killed`; NDJSON parsers normalized across agents.
- Pi session resumes now append `--continue`, so existing history/think level are reloaded instead of starting empty.
### Testing
- Fixtures isolate session stores; added coverage for thinking directives, stateful levels, heartbeat backpressure, and agent parsing.
## 1.3.0 — 2025-12-02
### Highlights
- **Pluggable agents (Claude, Pi, Codex, Opencode):** `inbound.reply.agent` selects CLI/parser; per-agent argv builders and NDJSON parsers enable swapping without template changes.
- **Safety stop words:** `stop|esc|abort|wait|exit` immediately reply “Agent was aborted.” and mark the session so the next prompt is prefixed with an abort reminder.
- **Agent session reliability:** Only Claude returns a stable `session_id`; others may reset between runs.
### Bug Fixes
- Empty `result` fields no longer leak raw JSON to users.
- Heartbeat alerts now honor `responsePrefix`.
- Command failures return user-friendly messages.
- Test session isolation to avoid touching real `sessions.json`.
- (Removed in 2.0.0) IPC reuse for `clawdis send/heartbeat` prevents Signal/WhatsApp session corruption.
- Web send respects media kind (image/audio/video/document) with correct limits.
### Changes
- (Removed in 2.0.0) IPC gateway socket at `~/.clawdis/ipc/gateway.sock` with automatic CLI fallback.
- Batched inbound messages with timestamps; typing indicator after sends.
- Watchdog restarts WhatsApp after long inactivity; heartbeat logging includes minutes since last message.
- Early `allowFrom` filtering before decryption.
- Same-phone mode with echo detection and optional `inbound.samePhoneMarker`.
## 1.2.2 — 2025-11-28
### Changes
- Manual heartbeat sends: `clawdis heartbeat --message/--body` (web provider only); `--dry-run` previews payloads.
## 1.2.1 — 2025-11-28
### Changes
- Media MIME-first handling; hosted media extensions derived from detected MIME with tests.
### Planned / in progress (from prior notes)
- Heartbeat targeting quality: clearer recipient resolution and verbose logs.
- Heartbeat delivery preview (Claude path) dry-run.
- Simulated inbound hook for local testing.
## 1.2.0 — 2025-11-27
### Changes
- Heartbeat interval default 10m for command mode; prompt `HEARTBEAT /think:high`; skips dont refresh session; session `heartbeatIdleMinutes` support.
- Heartbeat tooling: `--session-id`, `--heartbeat-now` (inline flag on `gateway`) for immediate startup probes.
- Prompt structure: `sessionIntro` plus per-message `/think:high`; session idle up to 7 days.
- Thinking directives: `/think:<level>`; Pi uses `--thinking`; others append cue; `/think:off` no-op.
- Robustness: Baileys/WebSocket guards; global unhandled error handlers; WhatsApp LID mapping; hosted media MIME-sniffing and cleanup.
- Docs: README Clawd setup; `docs/claude-config.md` for live config.
## 1.1.0 — 2025-11-26
### Changes
- Web auto-replies resize/recompress media and honor `inbound.reply.mediaMaxMb`.
- Detect media kind, enforce provider caps (images ≤6MB, audio/video ≤16MB, docs ≤100MB).
- `session.sendSystemOnce` and optional `sessionIntro`.
- Typing indicator refresh during commands; configurable via `inbound.reply.typingIntervalSeconds`.
- Optional audio transcription via external CLI.
- Command replies return structured payload/meta; respect `mediaMaxMb`; log Claude metadata; include `cwd` in timeout messages.
- Web provider refactor; logout command; web-only gateway start helper.
- Structured reconnect/heartbeat logging; bounded backoff with CLI/config knobs; troubleshooting guide.
- Relay help prints effective heartbeat/backoff when in web mode.
## 1.0.4 — 2025-11-25
### Changes
- Auto-replies now send a WhatsApp fallback message when a command/Claude run hits the timeout, including up to 800 chars of partial stdout so the user still sees progress.
- Added tests covering the new timeout fallback behavior and partial-output truncation.
- Web relay auto-reconnects after Baileys/WebSocket drops (with log-out detection) and exposes close events for monitoring; added tests for close propagation and reconnect loop.
- Timeout fallbacks send partial stdout (≤800 chars) to the user instead of silence; tests added.
- Web gateway auto-reconnects after Baileys/WebSocket drops; close propagation tests.
## 0.1.3 — 2025-11-25
### Features
- Added `cwd` option to command reply config for setting the working directory where commands execute. Essential for Claude Code to have proper project context.
- Added configurable file-based logging (default `/tmp/warelay/warelay.log`) with log level set via `logging.level` in `~/.warelay/warelay.json`; verbose still forces debug.
### Developer notes
- Command auto-replies now pass `{ timeoutMs, cwd }` into the command runner; custom runners/tests that stub `runCommandWithTimeout` should accept the options object as well as the legacy numeric timeout.
## 0.1.2 — 2025-11-25
### CI/build fix
- Fixed commander help configuration (`subcommandTerm`) so TypeScript builds pass in CI.
## 0.1.1 — 2025-11-25
### CLI polish
- Added a proper executable shim so `npx warelay@0.1.x --help` runs the CLI directly.
- Help/version banner now uses the README tagline with color, and the help footer includes colored examples with short explanations.
- `send` and `status` gained a `--verbose` flag for consistent noisy output when debugging.
- Lowercased branding in docs/UA; web provider UA is `warelay/cli/0.1.1`.
## 0.1.0 — 2025-11-25
### CLI & Providers
- Bundles a single `warelay` CLI with commands for `send`, `relay`, `status`, `webhook`, `login`, and tmux helpers `relay:tmux` / `relay:tmux:attach` (see `src/cli/program.ts`); `webhook` accepts `--ingress tailscale|none`.
- Supports two messaging backends: **Twilio** (default) and **personal WhatsApp Web**; `relay --provider auto` selects Web when a cached login exists, otherwise falls back to Twilio polling (`provider-web.ts`, `cli/program.ts`).
- `send` can target either provider, optionally wait for delivery status (Twilio only), output JSON, dry-run payloads, and attach media (`commands/send.ts`).
- `status` merges inbound + outbound Twilio traffic with formatted lines or JSON output (`commands/status.ts`, `twilio/messages.ts`).
### Webhook, Funnel & Port Management
- `webhook` starts an Express server for inbound Twilio callbacks, logs requests, and optionally auto-replies with static text or config-driven replies (`twilio/webhook.ts`, `commands/webhook.ts`).
- `webhook --ingress tailscale` automates end-to-end webhook setup: ensures required binaries, enables Tailscale Funnel, starts the webhook on the chosen port/path, discovers the WhatsApp sender SID, and updates Twilio webhook URLs with multiple fallbacks (`commands/up.ts`, `infra/tailscale.ts`, `twilio/update-webhook.ts`, `twilio/senders.ts`).
- Guardrails detect busy ports with helpful diagnostics and aborts when conflicts are found (`infra/ports.ts`).
### Auto-Reply Engine
- Configurable via `~/.warelay/warelay.json` (JSON5) with allowlist support, text or command-driven replies, templating (`{{Body}}`, `{{From}}`, `{{MediaPath}}`, etc.), optional body prefixes, and per-sender or global conversation sessions with `/new` resets and idle expiry (`auto-reply/reply.ts`, `config/config.ts`, `config/sessions.ts`, `auto-reply/templating.ts`).
- Command replies run through a process-wide FIFO queue to avoid concurrent executions across webhook, poller, and web listener flows (`process/command-queue.ts`); verbose mode surfaces wait times.
- Claude CLI integration auto-injects identity, output-format flags, session args, and parses JSON output while preserving metadata (`auto-reply/claude.ts`, `auto-reply/reply.ts`).
- Typing indicators fire before replies for Twilio, and Web provider sends “composing/available” presence when possible (`twilio/typing.ts`, `provider-web.ts`).
### Media Pipeline
- `send --media` works on both providers: Web accepts local paths or URLs; Twilio requires HTTPS and transparently hosts local files (≤5MB) via the Funnel/webhook media endpoint, auto-spawning a short-lived media server when `--serve-media` is requested (`commands/send.ts`, `media/host.ts`, `media/server.ts`).
- Auto-replies may include `mediaUrl` from config or command output (`MEDIA:` token extraction) and will host local media when needed before sending (`auto-reply/reply.ts`, `media/parse.ts`, `media/host.ts`).
- Inbound media from Twilio or Web is downloaded to `~/.warelay/media` with TTL cleanup and passed to commands via `MediaPath`/`MediaType` for richer prompts (`twilio/webhook.ts`, `provider-web.ts`, `media/store.ts`).
### Relay & Monitoring
- `relay` polls Twilio on an interval with exponential-backoff resilience, auto-replying to inbound messages, or listens live via WhatsApp Web with automatic read receipts and presence updates (`cli/program.ts`, `twilio/monitor.ts`, `provider-web.ts`).
- `send` + `waitForFinalStatus` polls Twilio until a terminal delivery state (delivered/read) or timeout, with clear failure surfaces (`twilio/send.ts`).
### Developer & Ops Ergonomics
- `relay:tmux` helper restarts/attaches to a dedicated `warelay-relay` tmux session for long-running relays (`cli/relay_tmux.ts`).
- Environment validation enforces Twilio credentials early and supports either auth token or API key/secret pairs (`env.ts`).
- Shared logging utilities, binary checks, and runtime abstractions keep CLI output consistent (`globals.ts`, `logger.ts`, `infra/binaries.ts`).
### Changes
- Auto-replies send a WhatsApp fallback message on command/Claude timeout with truncated stdout.
- Added tests for timeout fallback and partial-output truncation.

1
CLAUDE.md Symbolic link
View File

@@ -0,0 +1 @@
AGENTS.md

1
Peekaboo Submodule

Submodule Peekaboo added at 9db365b73c

273
README.md
View File

@@ -1,151 +1,202 @@
# 📡 warelay — Send, receive, and auto-reply on WhatsApp.
# 🦞 CLAWDIS — Personal AI Assistant
<p align="center">
<img src="README-header.png" alt="warelay header" width="640">
<img src="https://raw.githubusercontent.com/steipete/clawdis/main/docs/whatsapp-clawd.jpg" alt="CLAWDIS" width="400">
</p>
<p align="center">
<a href="https://github.com/steipete/warelay/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/steipete/warelay/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://www.npmjs.com/package/warelay"><img src="https://img.shields.io/npm/v/warelay.svg?style=for-the-badge" alt="npm version"></a>
<strong>EXFOLIATE! EXFOLIATE!</strong>
</p>
<p align="center">
<a href="https://github.com/steipete/clawdis/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/steipete/clawdis/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://github.com/steipete/clawdis/releases"><img src="https://img.shields.io/github/v/release/steipete/clawdis?include_prereleases&style=for-the-badge" alt="GitHub release"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
</p>
Send, receive, auto-reply, and inspect WhatsApp messages over **Twilio** or your personal **WhatsApp Web** session. Ships with a one-command webhook setup (Tailscale Funnel + Twilio callback) and a configurable auto-reply engine (plain text or command/Claude driven).
**Clawdis** is a *personal AI assistant* you run on your own devices.
It answers you on the surfaces you already use (WhatsApp, Telegram, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.
## Quick Start (pick your engine)
Install from npm (global): `npm install -g warelay` (Node 22+). Then choose **one** path:
If you want a private, single-user assistant that feels local, fast, and always-on, this is it.
**A) Personal WhatsApp Web (preferred: no Twilio creds, fastest setup)**
1. Link your account: `warelay login` (scan the QR).
2. Send a message: `warelay send --to +12345550000 --message "Hi from warelay"` (add `--provider web` if you want to force the web session).
3. Stay online & auto-reply: `warelay relay --verbose` (defaults to Web when logged in, falls back to Twilio otherwise).
```
Your surfaces
┌───────────────────────────────┐
│ Gateway │ ws://127.0.0.1:18789
│ (control plane) │ tcp://0.0.0.0:18790 (optional Bridge)
└──────────────┬────────────────┘
├─ Pi agent (RPC)
├─ CLI (clawdis …)
├─ WebChat (browser)
├─ macOS app (Clawdis.app)
└─ iOS node (Canvas + voice)
```
**B) Twilio WhatsApp number (for delivery status + webhooks)**
1. Copy `.env.example``.env`; set `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN` **or** `TWILIO_API_KEY`/`TWILIO_API_SECRET`, and `TWILIO_WHATSAPP_FROM=whatsapp:+19995550123` (optional `TWILIO_SENDER_SID`).
2. Send a message: `warelay send --to +12345550000 --message "Hi from warelay"`.
3. Receive replies:
- Polling (no ingress): `warelay relay --provider twilio --interval 5 --lookback 10`
- Webhook + public URL via Tailscale Funnel: `warelay webhook --ingress tailscale --port 42873 --path /webhook/whatsapp --verbose`
## What Clawdis does
> Already developing locally? You can still run `pnpm install` and `pnpm warelay ...` from the repo, but end users only need the npm package.
- **Personal assistant** — one user, one identity, one memory surface.
- **Multi-surface inbox** — WhatsApp, Telegram, WebChat, macOS, iOS.
- **Voice wake + push-to-talk** — local speech recognition on macOS/iOS.
- **Canvas** — a live visual workspace you can drive from the agent.
- **Automation-ready** — browser control, media handling, and tool streaming.
- **Local-first control plane** — the Gateway owns state, everything else connects.
- **Group chats** — mention-based by default, `/activation always|mention` per group (owner-only).
## Main Features
- **Two providers:** Twilio (default) for reliable delivery + status; Web provider for quick personal sends/receives via QR login.
- **Auto-replies:** Static templates or external commands (Claude-aware), with per-sender or global sessions and `/new` resets.
- Claude setup guide: see `docs/claude-config.md` for the exact Claude CLI configuration we support.
- **Webhook in one go:** `warelay webhook --ingress tailscale` enables Tailscale Funnel, runs the webhook server, and updates the Twilio sender callback URL.
- **Polling fallback:** `relay` polls Twilio when webhooks arent available; works headless.
- **Status + delivery tracking:** `status` shows recent inbound/outbound; `send` can wait for final Twilio status.
## How it works (short)
## Command Cheat Sheet
| Command | What it does | Core flags |
| --- | --- | --- |
| `warelay send` | Send a WhatsApp message (Twilio or Web) | `--to <e164>` `--message <text>` `--wait <sec>` `--poll <sec>` `--provider twilio\|web` `--json` `--dry-run` `--verbose` |
| `warelay relay` | Auto-reply loop (poll Twilio or listen on Web) | `--provider <auto\|twilio\|web>` `--interval <sec>` `--lookback <min>` `--verbose` |
| `warelay status` | Show recent sent/received messages | `--limit <n>` `--lookback <min>` `--json` `--verbose` |
| `warelay webhook` | Run inbound webhook (`ingress=tailscale` updates Twilio; `none` is local-only) | `--ingress tailscale\|none` `--port <port>` `--path <path>` `--reply <text>` `--verbose` `--yes` `--dry-run` |
| `warelay login` | Link personal WhatsApp Web via QR | `--verbose` |
- **Gateway** is the single source of truth for sessions/providers.
- **Loopback-first**: `ws://127.0.0.1:18789` by default.
- **Bridge** (optional) exposes a paired-node port for iOS/Android.
- **Agent runtime** is **Pi** in RPC mode.
### Sending images
- Twilio: `warelay send --to +1... --message "Hi" --media ./pic.jpg --serve-media` (needs `warelay webhook --ingress tailscale` or `--serve-media` to auto-host via Funnel; max 5MB).
- Web: `warelay send --provider web --media ./pic.jpg --message "Hi"` (local path or URL; no hosting needed).
- Auto-replies can attach `mediaUrl` in `~/.warelay/warelay.json` (used alongside `text` when present).
## Quick start (from source)
## Providers
- **Twilio (default):** needs `.env` creds + WhatsApp-enabled number; supports delivery tracking, polling, webhooks, and auto-reply typing indicators.
- **Web (`--provider web`):** uses your personal WhatsApp via Baileys; supports send/receive + auto-reply, but no delivery-status wait; cache lives in `~/.warelay/credentials/` (rerun `login` if logged out).
- **Auto-select (`relay` only):** `--provider auto` uses Web when logged in, otherwise Twilio polling.
Runtime: **Node ≥22** + **pnpm**.
Best practice: use a dedicated WhatsApp account (separate SIM/eSIM or business account) for automation instead of your primary personal account to avoid unexpected logouts or rate limits.
```bash
pnpm install
pnpm build
pnpm ui:build
# Link WhatsApp (stores creds in ~/.clawdis/credentials)
pnpm clawdis login
# Start the gateway
pnpm clawdis gateway --port 18789 --verbose
# Send a message
pnpm clawdis send --to +1234567890 --message "Hello from Clawdis"
# Talk to the assistant (optionally deliver back to WhatsApp/Telegram)
pnpm clawdis agent --message "Ship checklist" --thinking high
```
If you run from source, prefer `pnpm clawdis …` (not global `clawdis`).
## Chat commands
Send these in WhatsApp/Telegram/WebChat (group commands are owner-only):
- `/status` — health + session info (group shows activation mode)
- `/new` or `/reset` — reset the session
- `/think <level>` — off|minimal|low|medium|high
- `/verbose on|off`
- `/restart` — restart the gateway (owner-only in groups)
- `/activation mention|always` — group activation toggle (groups only)
## Architecture
### TypeScript Gateway (src/gateway/server.ts)
- **Single HTTP+WS server** on `ws://127.0.0.1:18789` (bind policy: loopback/lan/tailnet/auto). The first frame must be `connect`; AJV validates frames against TypeBox schemas (`src/gateway/protocol`).
- **Single source of truth** for sessions, providers, cron, voice wake, and presence. Methods cover `send`, `agent`, `chat.*`, `sessions.*`, `config.*`, `cron.*`, `voicewake.*`, `node.*`, `system-*`, `wake`.
- **Events + snapshot**: handshake returns a snapshot (presence/health) and declares event types; runtime events include `agent`, `chat`, `presence`, `tick`, `health`, `heartbeat`, `cron`, `node.pair.*`, `voicewake.changed`, `shutdown`.
- **Idempotency & safety**: `send`/`agent`/`chat.send` require idempotency keys with a TTL cache (5 min, cap 1000) to avoid doublesends on reconnects; payload sizes are capped per connection.
- **Bridge for nodes**: optional TCP bridge (`src/infra/bridge/server.ts`) is newlinedelimited JSON frames (`hello`, pairing, RPC, `invoke`); node connect/disconnect is surfaced into presence.
- **Control UI + Canvas Host**: HTTP serves `/ui` assets (if built) and can host a livereload Canvas host for nodes (`src/canvas-host/server.ts`), injecting the A2UI postMessage bridge.
### iOS app (apps/ios)
- **Discovery + pairing**: Bonjour discovery via `BridgeDiscoveryModel` (NWBrowser). `BridgeConnectionController` autoconnects using Keychain token or allows manual host/port.
- **Node runtime**: `BridgeSession` (actor) maintains the `NWConnection`, hello handshake, ping/pong, RPC requests, and `invoke` callbacks.
- **Capabilities + commands**: advertises `canvas`, `screen`, `camera`, `voiceWake` (settingsdriven) and executes `canvas.*`, `canvas.a2ui.*`, `camera.*`, `screen.record` (`NodeAppModel.handleInvoke`).
- **Canvas**: `WKWebView` with bundled Canvas scaffold + A2UI, JS eval, snapshot capture, and `clawdis://` deeplink interception (`ScreenController`).
- **Voice + deep links**: voice wake sends `voice.transcript` events; `clawdis://agent` links emit `agent.request`. Voice wake triggers sync via `voicewake.get` + `voicewake.changed`.
## Companion apps
The **macOS app is critical**: it runs the menubar control plane, owns local permissions (TCC), hosts Voice Wake, exposes WebChat/debug tools, and coordinates local/remote gateway mode. Most “assistant” UX lives here.
### macOS (Clawdis.app)
- Menu bar control for the Gateway and health.
- Voice Wake + push-to-talk overlay.
- WebChat + debug tools.
- Remote gateway control over SSH.
Build/run: `./scripts/restart-mac.sh` (packages + launches).
### iOS node (internal)
- Pairs as a node via the Bridge.
- Voice trigger forwarding + Canvas surface.
- Controlled via `clawdis nodes …`.
Runbook: `docs/ios/connect.md`.
### Android node (internal)
- Pairs via the same Bridge + pairing flow as iOS.
- Exposes Canvas, Camera, and Screen capture commands.
- Runbook: `docs/android/connect.md`.
## Agent workspace + skills
- Workspace root: `~/clawd` (configurable via `inbound.workspace`).
- Injected prompt files: `AGENTS.md`, `SOUL.md`, `TOOLS.md`.
- Skills: `~/clawd/skills/<skill>/SKILL.md`.
## Configuration
### Environment (.env)
| Variable | Required | Description |
| --- | --- | --- |
| `TWILIO_ACCOUNT_SID` | Yes (Twilio provider) | Twilio Account SID |
| `TWILIO_AUTH_TOKEN` | Yes* | Auth token (or use API key/secret) |
| `TWILIO_API_KEY` | Yes* | API key if not using auth token |
| `TWILIO_API_SECRET` | Yes* | API secret paired with `TWILIO_API_KEY` |
| `TWILIO_WHATSAPP_FROM` | Yes (Twilio provider) | WhatsApp-enabled sender, e.g. `whatsapp:+19995550123` |
| `TWILIO_SENDER_SID` | Optional | Overrides auto-discovery of the sender SID |
(*Provide either auth token OR api key/secret.)
### Auto-reply config (`~/.warelay/warelay.json`, JSON5)
- Controls who is allowed to trigger replies (`allowFrom`), reply mode (`text` or `command`), templates, and session behavior.
- Example (Claude command):
Minimal `~/.clawdis/clawdis.json`:
```json5
{
inbound: {
allowFrom: ["+12345550000"],
reply: {
mode: "command",
bodyPrefix: "You are a concise WhatsApp assistant.\n\n",
command: ["claude", "--dangerously-skip-permissions", "{{BodyStripped}}"],
claudeOutputFormat: "text",
session: { scope: "per-sender", resetTriggers: ["/new"], idleMinutes: 60 }
}
allowFrom: ["+1234567890"]
}
}
```
### Logging (optional)
- File logs are written to `/tmp/warelay/warelay.log` by default. Levels: `silent | fatal | error | warn | info | debug | trace` (CLI `--verbose` forces `debug`). Web-provider inbound/outbound entries include message bodies and auto-reply text for easier auditing.
- Override in `~/.warelay/warelay.json`:
### WhatsApp
- Link the device: `pnpm clawdis login` (stores creds in `~/.clawdis/credentials`).
- Allowlist who can talk to the assistant via `inbound.allowFrom`.
### Telegram
- Set `TELEGRAM_BOT_TOKEN` or `telegram.botToken` (env wins).
- Optional: set `telegram.requireMention`, `telegram.allowFrom`, or `telegram.webhookUrl` as needed.
```json5
{
logging: {
level: "warn",
file: "/tmp/warelay/custom.log"
telegram: {
botToken: "123456:ABCDEF"
}
}
```
### Claude CLI setup (how we run it)
1) Install the official Claude CLI (e.g., `brew install anthropic-ai/cli/claude` or follow the Anthropic docs) and run `claude login` so it can read your API key.
2) In `warelay.json`, set `reply.mode` to `"command"` and point `command[0]` to `"claude"`; set `claudeOutputFormat` to `"text"` (or `"json"`/`"stream-json"` if you want warelay to parse and trim the JSON output).
3) (Optional) Add `bodyPrefix` to inject a system prompt and `session` settings to keep multi-turn context (`/new` resets by default).
4) Run `pnpm warelay relay --provider auto` (or `--provider web|twilio`) and send a WhatsApp message; warelay will queue the Claude call, stream typing indicators (Twilio provider), parse the result, and send back the text.
Browser control (optional):
### Auto-reply parameter table (compact)
| Key | Type & default | Notes |
| --- | --- | --- |
| `inbound.allowFrom` | `string[]` (default: empty) | E.164 numbers allowed to trigger auto-reply (no `whatsapp:`). |
| `inbound.reply.mode` | `"text"` \| `"command"` (default: —) | Reply style. |
| `inbound.reply.text` | `string` (default: —) | Used when `mode=text`; templating supported. |
| `inbound.reply.command` | `string[]` (default: —) | Argv for `mode=command`; each element templated. Stdout (trimmed) is sent. |
| `inbound.reply.template` | `string` (default: —) | Injected as argv[1] (prompt prefix) before the body. |
| `inbound.reply.bodyPrefix` | `string` (default: —) | Prepended to `Body` before templating (great for system prompts). |
| `inbound.reply.timeoutSeconds` | `number` (default: `600`) | Command timeout. |
| `inbound.reply.claudeOutputFormat` | `"text"`\|`"json"`\|`"stream-json"` (default: —) | When command starts with `claude`, auto-adds `--output-format` + `-p/--print` and trims reply text. |
| `inbound.reply.session.scope` | `"per-sender"`\|`"global"` (default: `per-sender`) | Session bucket for conversation memory. |
| `inbound.reply.session.resetTriggers` | `string[]` (default: `["/new"]`) | Exact match or prefix (`/new hi`) resets session. |
| `inbound.reply.session.idleMinutes` | `number` (default: `60`) | Session expires after idle period. |
| `inbound.reply.session.store` | `string` (default: `~/.warelay/sessions.json`) | Custom session store path. |
| `inbound.reply.session.sessionArgNew` | `string[]` (default: `["--session-id","{{SessionId}}"]`) | Args injected for a new session run. |
| `inbound.reply.session.sessionArgResume` | `string[]` (default: `["--resume","{{SessionId}}"]`) | Args for resumed sessions. |
| `inbound.reply.session.sessionArgBeforeBody` | `boolean` (default: `true`) | Place session args before final body arg. |
```json5
{
browser: {
enabled: true,
controlUrl: "http://127.0.0.1:18791",
color: "#FF4500"
}
}
```
Templating tokens: `{{Body}}`, `{{BodyStripped}}`, `{{From}}`, `{{To}}`, `{{MessageSid}}`, plus `{{SessionId}}` and `{{IsNewSession}}` when sessions are enabled.
## Docs
## Webhook & Tailscale Flow
- `warelay webhook --ingress none` starts the local Express server on your chosen port/path; add `--reply "Got it"` for a static reply when no config file is present.
- `warelay webhook --ingress tailscale` enables Tailscale Funnel, prints the public URL (`https://<tailnet-host><path>`), starts the webhook, discovers the WhatsApp sender SID, and updates Twilio callbacks to the Funnel URL.
- If Funnel is not allowed on your tailnet, the CLI exits with guidance; you can still use `relay --provider twilio` to poll without webhooks.
- [`docs/index.md`](docs/index.md) (overview)
- [`docs/configuration.md`](docs/configuration.md)
- [`docs/group-messages.md`](docs/group-messages.md)
- [`docs/gateway.md`](docs/gateway.md)
- [`docs/web.md`](docs/web.md)
- [`docs/discovery.md`](docs/discovery.md)
- [`docs/agent.md`](docs/agent.md)
- [`docs/security.md`](docs/security.md)
- [`docs/troubleshooting.md`](docs/troubleshooting.md)
- [`docs/ios/connect.md`](docs/ios/connect.md)
- [`docs/clawdis-mac.md`](docs/clawdis-mac.md)
## Troubleshooting Tips
- Send/receive issues: run `pnpm warelay status --limit 20 --lookback 240 --json` to inspect recent traffic.
- Auto-reply not firing: ensure sender is in `allowFrom` (or unset), and confirm `.env` + `warelay.json` are loaded (reload shell after edits).
- Web provider dropped: rerun `pnpm warelay login`; credentials live in `~/.warelay/credentials/`.
- Tailscale Funnel errors: update tailscale/tailscaled; check admin console that Funnel is enabled for this device.
## Clawd
## FAQ & Safety
- Twilio errors: **63016 “permission to send an SMS has not been enabled”** → ensure your number is WhatsApp-enabled; **63007 template not approved** → send a free-form session message within 24h or use an approved template; **63112 policy violation** → adjust content, shorten to <1600 chars, avoid links that trigger spam filters. Re-run `pnpm warelay status` to see the exact Twilio response body.
- Does this store my messages? warelay only writes `~/.warelay/warelay.json` (config), `~/.warelay/credentials/` (WhatsApp Web auth), and `~/.warelay/sessions.json` (session IDs + timestamps). It does **not** persist message bodies beyond the session store. Logs stream to stdout/stderr and also `/tmp/warelay/warelay.log` (configurable via `logging.file`).
- Personal WhatsApp safety: Automation on personal accounts can be rate-limited or logged out by WhatsApp. Use `--provider web` sparingly, keep messages human-like, and re-run `login` if the session is dropped.
- Limits to remember: WhatsApp text limit ~1600 chars; avoid rapid bursts—space sends by a few seconds; keep webhook replies under a couple seconds for good UX; command auto-replies time out after 600s by default.
- Deploy / keep running: Use `tmux` or `screen` for ad-hoc (`tmux new -s warelay -- pnpm warelay relay --provider twilio`). For long-running hosts, wrap `pnpm warelay relay ...` or `pnpm warelay webhook --ingress tailscale ...` in a systemd service or macOS LaunchAgent; ensure environment variables are loaded in that context.
- Rotating credentials: Update `.env` (Twilio keys), rerun your process; for Web provider, delete `~/.warelay/credentials/` and rerun `pnpm warelay login` to relink.
Clawdis was built for **Clawd**, a space lobster AI assistant.
- https://clawd.me
- https://soul.md
- https://steipete.me

54
Swabble/.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
name: CI
on:
push:
branches: [main]
pull_request:
jobs:
build-and-test:
runs-on: macos-latest
defaults:
run:
shell: bash
working-directory: swabble
steps:
- name: Checkout swabble
uses: actions/checkout@v4
with:
path: swabble
- name: Select Xcode 26.1 (prefer 26.1.1)
run: |
set -euo pipefail
# pick the newest installed 26.1.x, fallback to newest 26.x
CANDIDATE="$(ls -d /Applications/Xcode_26.1*.app 2>/dev/null | sort -V | tail -1 || true)"
if [[ -z "$CANDIDATE" ]]; then
CANDIDATE="$(ls -d /Applications/Xcode_26*.app 2>/dev/null | sort -V | tail -1 || true)"
fi
if [[ -z "$CANDIDATE" ]]; then
echo "No Xcode 26.x found on runner" >&2
exit 1
fi
echo "Selecting $CANDIDATE"
sudo xcode-select -s "$CANDIDATE"
xcodebuild -version
- name: Show Swift version
run: swift --version
- name: Install tooling
run: |
brew update
brew install swiftlint swiftformat
- name: Format check
run: |
./scripts/format.sh
git diff --exit-code
- name: Lint
run: ./scripts/lint.sh
- name: Test
run: swift test --parallel

33
Swabble/.gitignore vendored Normal file
View File

@@ -0,0 +1,33 @@
# macOS
.DS_Store
# SwiftPM / Build
/.build
/.swiftpm
/DerivedData
xcuserdata/
*.xcuserstate
# Editors
/.vscode
.idea/
# Xcode artifacts
*.hmap
*.ipa
*.dSYM.zip
*.dSYM
# Playgrounds
*.xcplayground
playground.xcworkspace
timeline.xctimeline
# Carthage
Carthage/Build/
# fastlane
fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots/**/*.png
fastlane/test_output

8
Swabble/.swiftformat Normal file
View File

@@ -0,0 +1,8 @@
--swiftversion 6.2
--indent 4
--maxwidth 120
--wraparguments before-first
--wrapcollections before-first
--stripunusedargs closure-only
--self remove
--header ""

43
Swabble/.swiftlint.yml Normal file
View File

@@ -0,0 +1,43 @@
# SwiftLint for swabble
included:
- Sources
excluded:
- .build
- DerivedData
- "**/.swiftpm"
- "**/.build"
- "**/DerivedData"
- "**/.DS_Store"
opt_in_rules:
- array_init
- closure_spacing
- explicit_init
- fatal_error_message
- first_where
- joined_default_parameter
- last_where
- literal_expression_end_indentation
- multiline_arguments
- multiline_parameters
- operator_usage_whitespace
- redundant_nil_coalescing
- sorted_first_last
- switch_case_alignment
- vertical_parameter_alignment_on_call
- vertical_whitespace_opening_braces
- vertical_whitespace_closing_braces
disabled_rules:
- trailing_whitespace
- trailing_newline
- indentation_width
- identifier_name
- explicit_self
- file_header
- todo
line_length:
warning: 140
error: 180
reporter: "xcode"

11
Swabble/CHANGELOG.md Normal file
View File

@@ -0,0 +1,11 @@
# Changelog
## 0.2.0 — 2025-12-23
### Highlights
- Added `SwabbleKit` (multi-platform wake-word gate utilities with segment-aware gap detection).
- Swabble package now supports iOS + macOS consumers; CLI remains macOS 26-only.
### Changes
- CLI wake-word matching/stripping routed through `SwabbleKit` helpers.
- Speech pipeline types now explicitly gated to macOS 26 / iOS 26 availability.

21
Swabble/LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Peter Steinberger
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

33
Swabble/Package.resolved Normal file
View File

@@ -0,0 +1,33 @@
{
"originHash" : "3018b2c8c183d55b57ad0c4526b2380ac3b957d13a3a86e1b2845e81323c443a",
"pins" : [
{
"identity" : "commander",
"kind" : "remoteSourceControl",
"location" : "https://github.com/steipete/Commander.git",
"state" : {
"revision" : "8b8cb4f34315ce9e5307b3a2bcd77ff73f586a02",
"version" : "0.2.0"
}
},
{
"identity" : "swift-syntax",
"kind" : "remoteSourceControl",
"location" : "https://github.com/swiftlang/swift-syntax.git",
"state" : {
"revision" : "0687f71944021d616d34d922343dcef086855920",
"version" : "600.0.1"
}
},
{
"identity" : "swift-testing",
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-testing",
"state" : {
"revision" : "399f76dcd91e4c688ca2301fa24a8cc6d9927211",
"version" : "0.99.0"
}
}
],
"version" : 3
}

55
Swabble/Package.swift Normal file
View File

@@ -0,0 +1,55 @@
// swift-tools-version: 6.2
import PackageDescription
let package = Package(
name: "swabble",
platforms: [
.macOS(.v15),
.iOS(.v17),
],
products: [
.library(name: "Swabble", targets: ["Swabble"]),
.library(name: "SwabbleKit", targets: ["SwabbleKit"]),
.executable(name: "swabble", targets: ["SwabbleCLI"]),
],
dependencies: [
.package(url: "https://github.com/steipete/Commander.git", from: "0.2.0"),
.package(url: "https://github.com/apple/swift-testing", from: "0.99.0"),
],
targets: [
.target(
name: "Swabble",
path: "Sources/SwabbleCore",
swiftSettings: []),
.target(
name: "SwabbleKit",
path: "Sources/SwabbleKit",
swiftSettings: [
.enableUpcomingFeature("StrictConcurrency"),
]),
.executableTarget(
name: "SwabbleCLI",
dependencies: [
"Swabble",
"SwabbleKit",
.product(name: "Commander", package: "Commander"),
],
path: "Sources/swabble"),
.testTarget(
name: "SwabbleKitTests",
dependencies: [
"SwabbleKit",
.product(name: "Testing", package: "swift-testing"),
],
swiftSettings: [
.enableUpcomingFeature("StrictConcurrency"),
.enableExperimentalFeature("SwiftTesting"),
]),
.testTarget(
name: "swabbleTests",
dependencies: [
"Swabble",
.product(name: "Testing", package: "swift-testing"),
]),
],
swiftLanguageModes: [.v6])

111
Swabble/README.md Normal file
View File

@@ -0,0 +1,111 @@
# 🎙️ swabble — Speech.framework wake-word hook daemon (macOS 26)
swabble is a Swift 6.2 wake-word hook daemon. The CLI targets macOS 26 (SpeechAnalyzer + SpeechTranscriber). The shared `SwabbleKit` target is multi-platform and exposes wake-word gating utilities for iOS/macOS apps.
- **Local-only**: Speech.framework on-device models; zero network usage.
- **Wake word**: Default `clawd` (aliases `claude`), optional `--no-wake` bypass.
- **SwabbleKit**: Shared wake gate utilities (gap-based gating when you provide speech segments).
- **Hooks**: Run any command with prefix/env, cooldown, min_chars, timeout.
- **Services**: launchd helper stubs for start/stop/install.
- **File transcribe**: TXT or SRT with time ranges (using AttributedString splits).
## Quick start
```bash
# Install deps
brew install swiftformat swiftlint
# Build
swift build
# Write default config (~/.config/swabble/config.json)
swift run swabble setup
# Run foreground daemon
swift run swabble serve
# Test your hook
swift run swabble test-hook "hello world"
# Transcribe a file to SRT
swift run swabble transcribe /path/to/audio.m4a --format srt --output out.srt
```
## Use as a library
Add swabble as a SwiftPM dependency and import the `Swabble` or `SwabbleKit` product:
```swift
// Package.swift
dependencies: [
.package(url: "https://github.com/steipete/swabble.git", branch: "main"),
],
targets: [
.target(name: "MyApp", dependencies: [
.product(name: "Swabble", package: "swabble"), // Speech pipeline (macOS 26+ / iOS 26+)
.product(name: "SwabbleKit", package: "swabble"), // Wake-word gate utilities (iOS 17+ / macOS 15+)
]),
]
```
## CLI
- `serve` — foreground loop (mic → wake → hook)
- `transcribe <file>` — offline transcription (txt|srt)
- `test-hook "text"` — invoke configured hook
- `mic list|set <index>` — enumerate/select input device
- `setup` — write default config JSON
- `doctor` — check Speech auth & device availability
- `health` — prints `ok`
- `tail-log` — last 10 transcripts
- `status` — show wake state + recent transcripts
- `service install|uninstall|status` — user launchd plist (stub: prints launchctl commands)
- `start|stop|restart` — placeholders until full launchd wiring
All commands accept Commander runtime flags (`-v/--verbose`, `--json-output`, `--log-level`), plus `--config` where applicable.
## Config
`~/.config/swabble/config.json` (auto-created by `setup`):
```json
{
"audio": {"deviceName": "", "deviceIndex": -1, "sampleRate": 16000, "channels": 1},
"wake": {"enabled": true, "word": "clawd", "aliases": ["claude"]},
"hook": {
"command": "",
"args": [],
"prefix": "Voice swabble from ${hostname}: ",
"cooldownSeconds": 1,
"minCharacters": 24,
"timeoutSeconds": 5,
"env": {}
},
"logging": {"level": "info", "format": "text"},
"transcripts": {"enabled": true, "maxEntries": 50},
"speech": {"localeIdentifier": "en_US", "etiquetteReplacements": false}
}
```
- Config path override: `--config /path/to/config.json` on relevant commands.
- Transcripts persist to `~/Library/Application Support/swabble/transcripts.log`.
## Hook protocol
When a wake-gated transcript passes min_chars & cooldown, swabble runs:
```
<command> <args...> "<prefix><text>"
```
Environment variables:
- `SWABBLE_TEXT` — stripped transcript (wake word removed)
- `SWABBLE_PREFIX` — rendered prefix (hostname substituted)
- plus any `hook.env` key/values
## Speech pipeline
- `AVAudioEngine` tap → `BufferConverter``AnalyzerInput``SpeechAnalyzer` with a `SpeechTranscriber` module.
- Requests volatile + final results; the CLI uses text-only wake gating today.
- Authorization requested at first start; requires macOS 26 + new Speech.framework APIs.
## Development
- Format: `./scripts/format.sh` (uses ../peekaboo/.swiftformat if present)
- Lint: `./scripts/lint.sh` (uses ../peekaboo/.swiftlint.yml if present)
- Tests: `swift test` (uses swift-testing package)
## Roadmap
- launchd control (load/bootout, PID + status socket)
- JSON logging + PII redaction toggle
- Stronger wake-word detection and control socket status/health

View File

@@ -0,0 +1,77 @@
import Foundation
public struct SwabbleConfig: Codable, Sendable {
public struct Audio: Codable, Sendable {
public var deviceName: String = ""
public var deviceIndex: Int = -1
public var sampleRate: Double = 16000
public var channels: Int = 1
}
public struct Wake: Codable, Sendable {
public var enabled: Bool = true
public var word: String = "clawd"
public var aliases: [String] = ["claude"]
}
public struct Hook: Codable, Sendable {
public var command: String = ""
public var args: [String] = []
public var prefix: String = "Voice swabble from ${hostname}: "
public var cooldownSeconds: Double = 1
public var minCharacters: Int = 24
public var timeoutSeconds: Double = 5
public var env: [String: String] = [:]
}
public struct Logging: Codable, Sendable {
public var level: String = "info"
public var format: String = "text" // text|json placeholder
}
public struct Transcripts: Codable, Sendable {
public var enabled: Bool = true
public var maxEntries: Int = 50
}
public struct Speech: Codable, Sendable {
public var localeIdentifier: String = Locale.current.identifier
public var etiquetteReplacements: Bool = false
}
public var audio = Audio()
public var wake = Wake()
public var hook = Hook()
public var logging = Logging()
public var transcripts = Transcripts()
public var speech = Speech()
public static let defaultPath = FileManager.default
.homeDirectoryForCurrentUser
.appendingPathComponent(".config/swabble/config.json")
public init() {}
}
public enum ConfigError: Error {
case missingConfig
}
public enum ConfigLoader {
public static func load(at path: URL?) throws -> SwabbleConfig {
let url = path ?? SwabbleConfig.defaultPath
if !FileManager.default.fileExists(atPath: url.path) {
throw ConfigError.missingConfig
}
let data = try Data(contentsOf: url)
return try JSONDecoder().decode(SwabbleConfig.self, from: data)
}
public static func save(_ config: SwabbleConfig, at path: URL?) throws {
let url = path ?? SwabbleConfig.defaultPath
let dir = url.deletingLastPathComponent()
try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
let data = try JSONEncoder().encode(config)
try data.write(to: url)
}
}

View File

@@ -0,0 +1,75 @@
import Foundation
public struct HookJob: Sendable {
public let text: String
public let timestamp: Date
public init(text: String, timestamp: Date) {
self.text = text
self.timestamp = timestamp
}
}
public actor HookExecutor {
private let config: SwabbleConfig
private var lastRun: Date?
private let hostname: String
public init(config: SwabbleConfig) {
self.config = config
hostname = Host.current().localizedName ?? "host"
}
public func shouldRun() -> Bool {
guard config.hook.cooldownSeconds > 0 else { return true }
if let lastRun, Date().timeIntervalSince(lastRun) < config.hook.cooldownSeconds {
return false
}
return true
}
public func run(job: HookJob) async throws {
guard shouldRun() else { return }
guard !config.hook.command.isEmpty else { throw NSError(
domain: "Hook",
code: 1,
userInfo: [NSLocalizedDescriptionKey: "hook command not set"]) }
let prefix = config.hook.prefix.replacingOccurrences(of: "${hostname}", with: hostname)
let payload = prefix + job.text
let process = Process()
process.executableURL = URL(fileURLWithPath: config.hook.command)
process.arguments = config.hook.args + [payload]
var env = ProcessInfo.processInfo.environment
env["SWABBLE_TEXT"] = job.text
env["SWABBLE_PREFIX"] = prefix
for (k, v) in config.hook.env {
env[k] = v
}
process.environment = env
let pipe = Pipe()
process.standardOutput = pipe
process.standardError = pipe
try process.run()
let timeoutNanos = UInt64(max(config.hook.timeoutSeconds, 0.1) * 1_000_000_000)
try await withThrowingTaskGroup(of: Void.self) { group in
group.addTask {
process.waitUntilExit()
}
group.addTask {
try await Task.sleep(nanoseconds: timeoutNanos)
if process.isRunning {
process.terminate()
}
}
try await group.next()
group.cancelAll()
}
lastRun = Date()
}
}

View File

@@ -0,0 +1,50 @@
@preconcurrency import AVFoundation
import Foundation
final class BufferConverter {
private final class Box<T>: @unchecked Sendable { var value: T; init(_ value: T) { self.value = value } }
enum ConverterError: Swift.Error {
case failedToCreateConverter
case failedToCreateConversionBuffer
case conversionFailed(NSError?)
}
private var converter: AVAudioConverter?
func convert(_ buffer: AVAudioPCMBuffer, to format: AVAudioFormat) throws -> AVAudioPCMBuffer {
let inputFormat = buffer.format
if inputFormat == format {
return buffer
}
if converter == nil || converter?.outputFormat != format {
converter = AVAudioConverter(from: inputFormat, to: format)
converter?.primeMethod = .none
}
guard let converter else { throw ConverterError.failedToCreateConverter }
let sampleRateRatio = converter.outputFormat.sampleRate / converter.inputFormat.sampleRate
let scaledInputFrameLength = Double(buffer.frameLength) * sampleRateRatio
let frameCapacity = AVAudioFrameCount(scaledInputFrameLength.rounded(.up))
guard let conversionBuffer = AVAudioPCMBuffer(pcmFormat: converter.outputFormat, frameCapacity: frameCapacity)
else {
throw ConverterError.failedToCreateConversionBuffer
}
var nsError: NSError?
let consumed = Box(false)
let inputBuffer = buffer
let status = converter.convert(to: conversionBuffer, error: &nsError) { _, statusPtr in
if consumed.value {
statusPtr.pointee = .noDataNow
return nil
}
consumed.value = true
statusPtr.pointee = .haveData
return inputBuffer
}
if status == .error {
throw ConverterError.conversionFailed(nsError)
}
return conversionBuffer
}
}

View File

@@ -0,0 +1,114 @@
import AVFoundation
import Foundation
import Speech
@available(macOS 26.0, iOS 26.0, *)
public struct SpeechSegment: Sendable {
public let text: String
public let isFinal: Bool
}
@available(macOS 26.0, iOS 26.0, *)
public enum SpeechPipelineError: Error {
case authorizationDenied
case analyzerFormatUnavailable
case transcriberUnavailable
}
/// Live microphone SpeechAnalyzer SpeechTranscriber pipeline.
@available(macOS 26.0, iOS 26.0, *)
public actor SpeechPipeline {
private struct UnsafeBuffer: @unchecked Sendable { let buffer: AVAudioPCMBuffer }
private var engine = AVAudioEngine()
private var transcriber: SpeechTranscriber?
private var analyzer: SpeechAnalyzer?
private var inputContinuation: AsyncStream<AnalyzerInput>.Continuation?
private var resultTask: Task<Void, Never>?
private let converter = BufferConverter()
public init() {}
public func start(localeIdentifier: String, etiquette: Bool) async throws -> AsyncStream<SpeechSegment> {
let auth = await requestAuthorizationIfNeeded()
guard auth == .authorized else { throw SpeechPipelineError.authorizationDenied }
let transcriberModule = SpeechTranscriber(
locale: Locale(identifier: localeIdentifier),
transcriptionOptions: etiquette ? [.etiquetteReplacements] : [],
reportingOptions: [.volatileResults],
attributeOptions: [])
transcriber = transcriberModule
guard let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith: [transcriberModule])
else {
throw SpeechPipelineError.analyzerFormatUnavailable
}
analyzer = SpeechAnalyzer(modules: [transcriberModule])
let (stream, continuation) = AsyncStream<AnalyzerInput>.makeStream()
inputContinuation = continuation
let inputNode = engine.inputNode
let inputFormat = inputNode.outputFormat(forBus: 0)
inputNode.removeTap(onBus: 0)
inputNode.installTap(onBus: 0, bufferSize: 2048, format: inputFormat) { [weak self] buffer, _ in
guard let self else { return }
let boxed = UnsafeBuffer(buffer: buffer)
Task { await self.handleBuffer(boxed.buffer, targetFormat: analyzerFormat) }
}
engine.prepare()
try engine.start()
try await analyzer?.start(inputSequence: stream)
guard let transcriberForStream = transcriber else {
throw SpeechPipelineError.transcriberUnavailable
}
return AsyncStream { continuation in
self.resultTask = Task {
do {
for try await result in transcriberForStream.results {
let seg = SpeechSegment(text: String(result.text.characters), isFinal: result.isFinal)
continuation.yield(seg)
}
} catch {
// swallow errors and finish
}
continuation.finish()
}
continuation.onTermination = { _ in
Task { await self.stop() }
}
}
}
public func stop() async {
resultTask?.cancel()
inputContinuation?.finish()
engine.inputNode.removeTap(onBus: 0)
engine.stop()
try? await analyzer?.finalizeAndFinishThroughEndOfInput()
}
private func handleBuffer(_ buffer: AVAudioPCMBuffer, targetFormat: AVAudioFormat) async {
do {
let converted = try converter.convert(buffer, to: targetFormat)
let input = AnalyzerInput(buffer: converted)
inputContinuation?.yield(input)
} catch {
// drop on conversion failure
}
}
private func requestAuthorizationIfNeeded() async -> SFSpeechRecognizerAuthorizationStatus {
let current = SFSpeechRecognizer.authorizationStatus()
guard current == .notDetermined else { return current }
return await withCheckedContinuation { continuation in
SFSpeechRecognizer.requestAuthorization { status in
continuation.resume(returning: status)
}
}
}
}

View File

@@ -0,0 +1,63 @@
import CoreMedia
import Foundation
import NaturalLanguage
extension AttributedString {
public func sentences(maxLength: Int? = nil) -> [AttributedString] {
let tokenizer = NLTokenizer(unit: .sentence)
let string = String(characters)
tokenizer.string = string
let sentenceRanges = tokenizer.tokens(for: string.startIndex..<string.endIndex).map {
(
$0,
AttributedString.Index($0.lowerBound, within: self)!
..<
AttributedString.Index($0.upperBound, within: self)!)
}
let ranges = sentenceRanges.flatMap { sentenceStringRange, sentenceRange in
let sentence = self[sentenceRange]
guard let maxLength, sentence.characters.count > maxLength else {
return [sentenceRange]
}
let wordTokenizer = NLTokenizer(unit: .word)
wordTokenizer.string = string
var wordRanges = wordTokenizer.tokens(for: sentenceStringRange).map {
AttributedString.Index($0.lowerBound, within: self)!
..<
AttributedString.Index($0.upperBound, within: self)!
}
guard !wordRanges.isEmpty else { return [sentenceRange] }
wordRanges[0] = sentenceRange.lowerBound..<wordRanges[0].upperBound
wordRanges[wordRanges.count - 1] = wordRanges[wordRanges.count - 1].lowerBound..<sentenceRange.upperBound
var ranges: [Range<AttributedString.Index>] = []
for wordRange in wordRanges {
if let lastRange = ranges.last,
self[lastRange].characters.count + self[wordRange].characters.count <= maxLength
{
ranges[ranges.count - 1] = lastRange.lowerBound..<wordRange.upperBound
} else {
ranges.append(wordRange)
}
}
return ranges
}
return ranges.compactMap { range in
let audioTimeRanges = self[range].runs.filter {
!String(self[$0.range].characters)
.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
}.compactMap(\.audioTimeRange)
guard !audioTimeRanges.isEmpty else { return nil }
let start = audioTimeRanges.first!.start
let end = audioTimeRanges.last!.end
var attributes = AttributeContainer()
attributes[AttributeScopes.SpeechAttributes.TimeRangeAttribute.self] = CMTimeRange(
start: start,
end: end)
return AttributedString(self[range].characters, attributes: attributes)
}
}
}

View File

@@ -0,0 +1,41 @@
import Foundation
public enum LogLevel: String, Comparable, CaseIterable, Sendable {
case trace, debug, info, warn, error
var rank: Int {
switch self {
case .trace: 0
case .debug: 1
case .info: 2
case .warn: 3
case .error: 4
}
}
public static func < (lhs: LogLevel, rhs: LogLevel) -> Bool { lhs.rank < rhs.rank }
}
public struct Logger: Sendable {
public let level: LogLevel
public init(level: LogLevel) { self.level = level }
public func log(_ level: LogLevel, _ message: String) {
guard level >= self.level else { return }
let ts = ISO8601DateFormatter().string(from: Date())
print("[\(level.rawValue.uppercased())] \(ts) | \(message)")
}
public func trace(_ msg: String) { log(.trace, msg) }
public func debug(_ msg: String) { log(.debug, msg) }
public func info(_ msg: String) { log(.info, msg) }
public func warn(_ msg: String) { log(.warn, msg) }
public func error(_ msg: String) { log(.error, msg) }
}
extension LogLevel {
public init?(configValue: String) {
self.init(rawValue: configValue.lowercased())
}
}

View File

@@ -0,0 +1,45 @@
import CoreMedia
import Foundation
public enum OutputFormat: String {
case txt
case srt
public var needsAudioTimeRange: Bool {
switch self {
case .srt: true
default: false
}
}
public func text(for transcript: AttributedString, maxLength: Int) -> String {
switch self {
case .txt:
return String(transcript.characters)
case .srt:
func format(_ timeInterval: TimeInterval) -> String {
let ms = Int(timeInterval.truncatingRemainder(dividingBy: 1) * 1000)
let s = Int(timeInterval) % 60
let m = (Int(timeInterval) / 60) % 60
let h = Int(timeInterval) / 60 / 60
return String(format: "%0.2d:%0.2d:%0.2d,%0.3d", h, m, s, ms)
}
return transcript.sentences(maxLength: maxLength).compactMap { (sentence: AttributedString) -> (
CMTimeRange,
String)? in
guard let timeRange = sentence.audioTimeRange else { return nil }
return (timeRange, String(sentence.characters))
}.enumerated().map { index, run in
let (timeRange, text) = run
return """
\(index + 1)
\(format(timeRange.start.seconds)) --> \(format(timeRange.end.seconds))
\(text.trimmingCharacters(in: .whitespacesAndNewlines))
"""
}.joined().trimmingCharacters(in: .whitespacesAndNewlines)
}
}
}

View File

@@ -0,0 +1,46 @@
import Foundation
public actor TranscriptsStore {
public static let shared = TranscriptsStore()
private var entries: [String] = []
private let limit = 100
private let fileURL: URL
public init() {
let dir = FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent("Library/Application Support/swabble", isDirectory: true)
try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
fileURL = dir.appendingPathComponent("transcripts.log")
if let data = try? Data(contentsOf: fileURL),
let text = String(data: data, encoding: .utf8)
{
entries = text.split(separator: "\n").map(String.init).suffix(limit)
}
}
public func append(text: String) {
entries.append(text)
if entries.count > limit {
entries.removeFirst(entries.count - limit)
}
let body = entries.joined(separator: "\n")
try? body.write(to: fileURL, atomically: false, encoding: .utf8)
}
public func latest() -> [String] { entries }
}
extension String {
private func appendLine(to url: URL) throws {
let data = (self + "\n").data(using: .utf8) ?? Data()
if FileManager.default.fileExists(atPath: url.path) {
let handle = try FileHandle(forWritingTo: url)
try handle.seekToEnd()
try handle.write(contentsOf: data)
try handle.close()
} else {
try data.write(to: url)
}
}
}

View File

@@ -0,0 +1,202 @@
import Foundation
public struct WakeWordSegment: Sendable, Equatable {
public let text: String
public let start: TimeInterval
public let duration: TimeInterval
public let range: Range<String.Index>?
public init(text: String, start: TimeInterval, duration: TimeInterval, range: Range<String.Index>? = nil) {
self.text = text
self.start = start
self.duration = duration
self.range = range
}
public var end: TimeInterval { self.start + self.duration }
}
public struct WakeWordGateConfig: Sendable, Equatable {
public var triggers: [String]
public var minPostTriggerGap: TimeInterval
public var minCommandLength: Int
public init(
triggers: [String],
minPostTriggerGap: TimeInterval = 0.45,
minCommandLength: Int = 1)
{
self.triggers = triggers
self.minPostTriggerGap = minPostTriggerGap
self.minCommandLength = minCommandLength
}
}
public struct WakeWordGateMatch: Sendable, Equatable {
public let triggerEndTime: TimeInterval
public let postGap: TimeInterval
public let command: String
public init(triggerEndTime: TimeInterval, postGap: TimeInterval, command: String) {
self.triggerEndTime = triggerEndTime
self.postGap = postGap
self.command = command
}
}
public enum WakeWordGate {
private struct Token {
let normalized: String
let start: TimeInterval
let end: TimeInterval
let range: Range<String.Index>?
let text: String
}
private struct TriggerTokens {
let tokens: [String]
}
public static func match(
transcript: String,
segments: [WakeWordSegment],
config: WakeWordGateConfig)
-> WakeWordGateMatch? {
let triggerTokens = self.normalizeTriggers(config.triggers)
guard !triggerTokens.isEmpty else { return nil }
let tokens = self.normalizeSegments(segments)
guard !tokens.isEmpty else { return nil }
var bestIndex: Int?
var bestTriggerEnd: TimeInterval = 0
var bestGap: TimeInterval = 0
for trigger in triggerTokens {
let count = trigger.tokens.count
guard count > 0, tokens.count > count else { continue }
for i in 0...(tokens.count - count - 1) {
var matched = true
for t in 0..<count {
if tokens[i + t].normalized != trigger.tokens[t] {
matched = false
break
}
}
if !matched { continue }
let triggerEnd = tokens[i + count - 1].end
let nextToken = tokens[i + count]
let gap = nextToken.start - triggerEnd
if gap < config.minPostTriggerGap { continue }
if let bestIndex, i <= bestIndex { continue }
bestIndex = i
bestTriggerEnd = triggerEnd
bestGap = gap
}
}
guard let bestIndex else { return nil }
let command = self.commandText(transcript: transcript, segments: segments, triggerEndTime: bestTriggerEnd)
.trimmingCharacters(in: Self.whitespaceAndPunctuation)
guard command.count >= config.minCommandLength else { return nil }
return WakeWordGateMatch(triggerEndTime: bestTriggerEnd, postGap: bestGap, command: command)
}
public static func commandText(
transcript: String,
segments: [WakeWordSegment],
triggerEndTime: TimeInterval)
-> String {
let threshold = triggerEndTime + 0.001
for segment in segments where segment.start >= threshold {
if normalizeToken(segment.text).isEmpty { continue }
if let range = segment.range {
let slice = transcript[range.lowerBound...]
return String(slice).trimmingCharacters(in: Self.whitespaceAndPunctuation)
}
break
}
let text = segments
.filter { $0.start >= threshold && !self.normalizeToken($0.text).isEmpty }
.map(\.text)
.joined(separator: " ")
return text.trimmingCharacters(in: Self.whitespaceAndPunctuation)
}
public static func matchesTextOnly(text: String, triggers: [String]) -> Bool {
guard !text.isEmpty else { return false }
let normalized = text.lowercased()
for trigger in triggers {
let token = trigger.trimmingCharacters(in: self.whitespaceAndPunctuation).lowercased()
if token.isEmpty { continue }
if normalized.contains(token) { return true }
}
return false
}
public static func stripWake(text: String, triggers: [String]) -> String {
var out = text
for trigger in triggers {
let token = trigger.trimmingCharacters(in: self.whitespaceAndPunctuation)
guard !token.isEmpty else { continue }
out = out.replacingOccurrences(of: token, with: "", options: [.caseInsensitive])
}
return out.trimmingCharacters(in: self.whitespaceAndPunctuation)
}
private static func normalizeTriggers(_ triggers: [String]) -> [TriggerTokens] {
var output: [TriggerTokens] = []
for trigger in triggers {
let tokens = trigger
.split(whereSeparator: { $0.isWhitespace })
.map { self.normalizeToken(String($0)) }
.filter { !$0.isEmpty }
if tokens.isEmpty { continue }
output.append(TriggerTokens(tokens: tokens))
}
return output
}
private static func normalizeSegments(_ segments: [WakeWordSegment]) -> [Token] {
segments.compactMap { segment in
let normalized = self.normalizeToken(segment.text)
guard !normalized.isEmpty else { return nil }
return Token(
normalized: normalized,
start: segment.start,
end: segment.end,
range: segment.range,
text: segment.text)
}
}
private static func normalizeToken(_ token: String) -> String {
token
.trimmingCharacters(in: self.whitespaceAndPunctuation)
.lowercased()
}
private static let whitespaceAndPunctuation = CharacterSet.whitespacesAndNewlines
.union(.punctuationCharacters)
}
#if canImport(Speech)
import Speech
public enum WakeWordSpeechSegments {
public static func from(transcription: SFTranscription, transcript: String) -> [WakeWordSegment] {
transcription.segments.map { segment in
let range = Range(segment.substringRange, in: transcript)
return WakeWordSegment(
text: segment.substring,
start: segment.timestamp,
duration: segment.duration,
range: range)
}
}
}
#endif

View File

@@ -0,0 +1,71 @@
import Commander
import Foundation
@available(macOS 26.0, *)
@MainActor
enum CLIRegistry {
static var descriptors: [CommandDescriptor] {
let serveDesc = descriptor(for: ServeCommand.self)
let transcribeDesc = descriptor(for: TranscribeCommand.self)
let testHookDesc = descriptor(for: TestHookCommand.self)
let micList = descriptor(for: MicList.self)
let micSet = descriptor(for: MicSet.self)
let micRoot = CommandDescriptor(
name: "mic",
abstract: "Microphone management",
discussion: nil,
signature: CommandSignature(),
subcommands: [micList, micSet])
let serviceRoot = CommandDescriptor(
name: "service",
abstract: "launchd helper",
discussion: nil,
signature: CommandSignature(),
subcommands: [
descriptor(for: ServiceInstall.self),
descriptor(for: ServiceUninstall.self),
descriptor(for: ServiceStatus.self),
])
let doctorDesc = descriptor(for: DoctorCommand.self)
let setupDesc = descriptor(for: SetupCommand.self)
let healthDesc = descriptor(for: HealthCommand.self)
let tailLogDesc = descriptor(for: TailLogCommand.self)
let startDesc = descriptor(for: StartCommand.self)
let stopDesc = descriptor(for: StopCommand.self)
let restartDesc = descriptor(for: RestartCommand.self)
let statusDesc = descriptor(for: StatusCommand.self)
let rootSignature = CommandSignature().withStandardRuntimeFlags()
let root = CommandDescriptor(
name: "swabble",
abstract: "Speech hook daemon",
discussion: "Local wake-word → SpeechTranscriber → hook",
signature: rootSignature,
subcommands: [
serveDesc,
transcribeDesc,
testHookDesc,
micRoot,
serviceRoot,
doctorDesc,
setupDesc,
healthDesc,
tailLogDesc,
startDesc,
stopDesc,
restartDesc,
statusDesc,
])
return [root]
}
private static func descriptor(for type: any ParsableCommand.Type) -> CommandDescriptor {
let sig = CommandSignature.describe(type.init()).withStandardRuntimeFlags()
return CommandDescriptor(
name: type.commandDescription.commandName ?? "",
abstract: type.commandDescription.abstract,
discussion: type.commandDescription.discussion,
signature: sig,
subcommands: [])
}
}

View File

@@ -0,0 +1,37 @@
import Commander
import Foundation
import Speech
import Swabble
@MainActor
struct DoctorCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "doctor", abstract: "Check Speech permission and config")
}
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
init() {}
init(parsed: ParsedValues) {
self.init()
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
let auth = await SFSpeechRecognizer.authorizationStatus()
print("Speech auth: \(auth)")
do {
_ = try ConfigLoader.load(at: configURL)
print("Config: OK")
} catch {
print("Config missing or invalid; run setup")
}
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: [.microphone, .external],
mediaType: .audio,
position: .unspecified)
print("Mics found: \(session.devices.count)")
}
private var configURL: URL? { configPath.map { URL(fileURLWithPath: $0) } }
}

View File

@@ -0,0 +1,16 @@
import Commander
import Foundation
@MainActor
struct HealthCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "health", abstract: "Health probe")
}
init() {}
init(parsed: ParsedValues) {}
mutating func run() async throws {
print("ok")
}
}

View File

@@ -0,0 +1,62 @@
import AVFoundation
import Commander
import Foundation
import Swabble
@MainActor
struct MicCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(
commandName: "mic",
abstract: "Microphone management",
subcommands: [MicList.self, MicSet.self])
}
}
@MainActor
struct MicList: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "list", abstract: "List input devices")
}
init() {}
init(parsed: ParsedValues) {}
mutating func run() async throws {
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: [.microphone, .external],
mediaType: .audio,
position: .unspecified)
let devices = session.devices
if devices.isEmpty { print("no audio inputs found"); return }
for (idx, device) in devices.enumerated() {
print("[\(idx)] \(device.localizedName)")
}
}
}
@MainActor
struct MicSet: ParsableCommand {
@Argument(help: "Device index from list") var index: Int = 0
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
static var commandDescription: CommandDescription {
CommandDescription(commandName: "set", abstract: "Set default input device index")
}
init() {}
init(parsed: ParsedValues) {
self.init()
if let value = parsed.positional.first, let intVal = Int(value) { index = intVal }
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
var cfg = try ConfigLoader.load(at: configURL)
cfg.audio.deviceIndex = index
try ConfigLoader.save(cfg, at: configURL)
print("saved device index \(index)")
}
private var configURL: URL? { configPath.map { URL(fileURLWithPath: $0) } }
}

View File

@@ -0,0 +1,81 @@
import Commander
import Foundation
import Swabble
import SwabbleKit
@available(macOS 26.0, *)
@MainActor
struct ServeCommand: ParsableCommand {
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
@Flag(name: .long("no-wake"), help: "Disable wake word") var noWake: Bool = false
static var commandDescription: CommandDescription {
CommandDescription(
commandName: "serve",
abstract: "Run swabble in the foreground")
}
init() {}
init(parsed: ParsedValues) {
self.init()
if parsed.flags.contains("noWake") { noWake = true }
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
var cfg: SwabbleConfig
do {
cfg = try ConfigLoader.load(at: configURL)
} catch {
cfg = SwabbleConfig()
try ConfigLoader.save(cfg, at: configURL)
}
if noWake {
cfg.wake.enabled = false
}
let logger = Logger(level: LogLevel(configValue: cfg.logging.level) ?? .info)
logger.info("swabble serve starting (wake: \(cfg.wake.enabled ? cfg.wake.word : "disabled"))")
let pipeline = SpeechPipeline()
do {
let stream = try await pipeline.start(
localeIdentifier: cfg.speech.localeIdentifier,
etiquette: cfg.speech.etiquetteReplacements)
for await seg in stream {
if cfg.wake.enabled {
guard Self.matchesWake(text: seg.text, cfg: cfg) else { continue }
}
let stripped = Self.stripWake(text: seg.text, cfg: cfg)
let job = HookJob(text: stripped, timestamp: Date())
let executor = HookExecutor(config: cfg)
try await executor.run(job: job)
if cfg.transcripts.enabled {
await TranscriptsStore.shared.append(text: stripped)
}
if seg.isFinal {
logger.info("final: \(stripped)")
} else {
logger.debug("partial: \(stripped)")
}
}
} catch {
logger.error("serve error: \(error)")
throw error
}
}
private var configURL: URL? {
configPath.map { URL(fileURLWithPath: $0) }
}
private static func matchesWake(text: String, cfg: SwabbleConfig) -> Bool {
let triggers = [cfg.wake.word] + cfg.wake.aliases
return WakeWordGate.matchesTextOnly(text: text, triggers: triggers)
}
private static func stripWake(text: String, cfg: SwabbleConfig) -> String {
let triggers = [cfg.wake.word] + cfg.wake.aliases
return WakeWordGate.stripWake(text: text, triggers: triggers)
}
}

View File

@@ -0,0 +1,77 @@
import Commander
import Foundation
@MainActor
struct ServiceRootCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(
commandName: "service",
abstract: "Manage launchd agent",
subcommands: [ServiceInstall.self, ServiceUninstall.self, ServiceStatus.self])
}
}
private enum LaunchdHelper {
static let label = "com.swabble.agent"
static var plistURL: URL {
FileManager.default
.homeDirectoryForCurrentUser
.appendingPathComponent("Library/LaunchAgents/\(label).plist")
}
static func writePlist(executable: String) throws {
let plist: [String: Any] = [
"Label": label,
"ProgramArguments": [executable, "serve"],
"RunAtLoad": true,
"KeepAlive": true,
]
let data = try PropertyListSerialization.data(fromPropertyList: plist, format: .xml, options: 0)
try data.write(to: plistURL)
}
static func removePlist() throws {
try? FileManager.default.removeItem(at: plistURL)
}
}
@MainActor
struct ServiceInstall: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "install", abstract: "Install user launch agent")
}
mutating func run() async throws {
let exe = CommandLine.arguments.first ?? "/usr/local/bin/swabble"
try LaunchdHelper.writePlist(executable: exe)
print("launchctl load -w \(LaunchdHelper.plistURL.path)")
}
}
@MainActor
struct ServiceUninstall: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "uninstall", abstract: "Remove launch agent")
}
mutating func run() async throws {
try LaunchdHelper.removePlist()
print("launchctl bootout gui/$(id -u)/\(LaunchdHelper.label)")
}
}
@MainActor
struct ServiceStatus: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "status", abstract: "Show launch agent status")
}
mutating func run() async throws {
if FileManager.default.fileExists(atPath: LaunchdHelper.plistURL.path) {
print("plist present at \(LaunchdHelper.plistURL.path)")
} else {
print("launchd plist not installed")
}
}
}

View File

@@ -0,0 +1,26 @@
import Commander
import Foundation
import Swabble
@MainActor
struct SetupCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "setup", abstract: "Write default config")
}
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
init() {}
init(parsed: ParsedValues) {
self.init()
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
let cfg = SwabbleConfig()
try ConfigLoader.save(cfg, at: configURL)
print("wrote config to \(configURL?.path ?? SwabbleConfig.defaultPath.path)")
}
private var configURL: URL? { configPath.map { URL(fileURLWithPath: $0) } }
}

View File

@@ -0,0 +1,35 @@
import Commander
import Foundation
@MainActor
struct StartCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "start", abstract: "Start swabble (foreground placeholder)")
}
mutating func run() async throws {
print("start: launchd helper not implemented; run 'swabble serve' instead")
}
}
@MainActor
struct StopCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "stop", abstract: "Stop swabble (placeholder)")
}
mutating func run() async throws {
print("stop: launchd helper not implemented yet")
}
}
@MainActor
struct RestartCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "restart", abstract: "Restart swabble (placeholder)")
}
mutating func run() async throws {
print("restart: launchd helper not implemented yet")
}
}

View File

@@ -0,0 +1,34 @@
import Commander
import Foundation
import Swabble
@MainActor
struct StatusCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "status", abstract: "Show daemon state")
}
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
init() {}
init(parsed: ParsedValues) {
self.init()
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
let cfg = try? ConfigLoader.load(at: configURL)
let wake = cfg?.wake.word ?? "clawd"
let wakeEnabled = cfg?.wake.enabled ?? false
let latest = await TranscriptsStore.shared.latest().suffix(3)
print("wake: \(wakeEnabled ? wake : "disabled")")
if latest.isEmpty {
print("transcripts: (none yet)")
} else {
print("last transcripts:")
latest.forEach { print("- \($0)") }
}
}
private var configURL: URL? { configPath.map { URL(fileURLWithPath: $0) } }
}

View File

@@ -0,0 +1,20 @@
import Commander
import Foundation
import Swabble
@MainActor
struct TailLogCommand: ParsableCommand {
static var commandDescription: CommandDescription {
CommandDescription(commandName: "tail-log", abstract: "Tail recent transcripts")
}
init() {}
init(parsed: ParsedValues) {}
mutating func run() async throws {
let latest = await TranscriptsStore.shared.latest()
for line in latest.suffix(10) {
print(line)
}
}
}

View File

@@ -0,0 +1,30 @@
import Commander
import Foundation
import Swabble
@MainActor
struct TestHookCommand: ParsableCommand {
@Argument(help: "Text to send to hook") var text: String
@Option(name: .long("config"), help: "Path to config JSON") var configPath: String?
static var commandDescription: CommandDescription {
CommandDescription(commandName: "test-hook", abstract: "Invoke the configured hook with text")
}
init() {}
init(parsed: ParsedValues) {
self.init()
if let positional = parsed.positional.first { text = positional }
if let cfg = parsed.options["config"]?.last { configPath = cfg }
}
mutating func run() async throws {
let cfg = try ConfigLoader.load(at: configURL)
let executor = HookExecutor(config: cfg)
try await executor.run(job: HookJob(text: text, timestamp: Date()))
print("hook invoked")
}
private var configURL: URL? { configPath.map { URL(fileURLWithPath: $0) } }
}

View File

@@ -0,0 +1,61 @@
import AVFoundation
import Commander
import Foundation
import Speech
import Swabble
@MainActor
struct TranscribeCommand: ParsableCommand {
@Argument(help: "Path to audio/video file") var inputFile: String = ""
@Option(name: .long("locale"), help: "Locale identifier", parsing: .singleValue) var locale: String = Locale.current
.identifier
@Flag(help: "Censor etiquette-sensitive content") var censor: Bool = false
@Option(name: .long("output"), help: "Output file path") var outputFile: String?
@Option(name: .long("format"), help: "Output format txt|srt") var format: String = "txt"
@Option(name: .long("max-length"), help: "Max sentence length for srt") var maxLength: Int = 40
static var commandDescription: CommandDescription {
CommandDescription(
commandName: "transcribe",
abstract: "Transcribe a media file locally")
}
init() {}
init(parsed: ParsedValues) {
self.init()
if let positional = parsed.positional.first { inputFile = positional }
if let loc = parsed.options["locale"]?.last { locale = loc }
if parsed.flags.contains("censor") { censor = true }
if let out = parsed.options["output"]?.last { outputFile = out }
if let fmt = parsed.options["format"]?.last { format = fmt }
if let len = parsed.options["maxLength"]?.last, let intVal = Int(len) { maxLength = intVal }
}
mutating func run() async throws {
let fileURL = URL(fileURLWithPath: inputFile)
let audioFile = try AVAudioFile(forReading: fileURL)
let outputFormat = OutputFormat(rawValue: format) ?? .txt
let transcriber = SpeechTranscriber(
locale: Locale(identifier: locale),
transcriptionOptions: censor ? [.etiquetteReplacements] : [],
reportingOptions: [],
attributeOptions: outputFormat.needsAudioTimeRange ? [.audioTimeRange] : [])
let analyzer = SpeechAnalyzer(modules: [transcriber])
try await analyzer.start(inputAudioFile: audioFile, finishAfterFile: true)
var transcript: AttributedString = ""
for try await result in transcriber.results {
transcript += result.text
}
let output = outputFormat.text(for: transcript, maxLength: maxLength)
if let path = outputFile {
try output.write(to: URL(fileURLWithPath: path), atomically: false, encoding: .utf8)
} else {
print(output)
}
}
}

View File

@@ -0,0 +1,106 @@
import Commander
import Foundation
@available(macOS 26.0, *)
@MainActor
private func runCLI() async -> Int32 {
do {
let descriptors = CLIRegistry.descriptors
let program = Program(descriptors: descriptors)
let invocation = try program.resolve(argv: CommandLine.arguments)
try await dispatch(invocation: invocation)
return 0
} catch {
fputs("error: \(error)\n", stderr)
return 1
}
}
@available(macOS 26.0, *)
@MainActor
private func dispatch(invocation: CommandInvocation) async throws {
let parsed = invocation.parsedValues
let path = invocation.path
guard let first = path.first else { throw CommanderProgramError.missingCommand }
switch first {
case "swabble":
guard path.count >= 2 else { throw CommanderProgramError.missingSubcommand(command: "swabble") }
let sub = path[1]
switch sub {
case "serve":
var cmd = ServeCommand(parsed: parsed)
try await cmd.run()
case "transcribe":
var cmd = TranscribeCommand(parsed: parsed)
try await cmd.run()
case "test-hook":
var cmd = TestHookCommand(parsed: parsed)
try await cmd.run()
case "mic":
guard path.count >= 3 else { throw CommanderProgramError.missingSubcommand(command: "mic") }
let micSub = path[2]
if micSub == "list" {
var cmd = MicList(parsed: parsed)
try await cmd.run()
} else if micSub == "set" {
var cmd = MicSet(parsed: parsed)
try await cmd.run()
} else {
throw CommanderProgramError.unknownSubcommand(command: "mic", name: micSub)
}
case "service":
guard path.count >= 3 else { throw CommanderProgramError.missingSubcommand(command: "service") }
let svcSub = path[2]
switch svcSub {
case "install":
var cmd = ServiceInstall()
try await cmd.run()
case "uninstall":
var cmd = ServiceUninstall()
try await cmd.run()
case "status":
var cmd = ServiceStatus()
try await cmd.run()
default:
throw CommanderProgramError.unknownSubcommand(command: "service", name: svcSub)
}
case "doctor":
var cmd = DoctorCommand(parsed: parsed)
try await cmd.run()
case "setup":
var cmd = SetupCommand(parsed: parsed)
try await cmd.run()
case "health":
var cmd = HealthCommand(parsed: parsed)
try await cmd.run()
case "tail-log":
var cmd = TailLogCommand(parsed: parsed)
try await cmd.run()
case "start":
var cmd = StartCommand()
try await cmd.run()
case "stop":
var cmd = StopCommand()
try await cmd.run()
case "restart":
var cmd = RestartCommand()
try await cmd.run()
case "status":
var cmd = StatusCommand()
try await cmd.run()
default:
throw CommanderProgramError.unknownSubcommand(command: "swabble", name: sub)
}
default:
throw CommanderProgramError.unknownCommand(first)
}
}
if #available(macOS 26.0, *) {
let exitCode = await runCLI()
exit(exitCode)
} else {
fputs("error: swabble requires macOS 26 or newer\n", stderr)
exit(1)
}

View File

@@ -0,0 +1,63 @@
import Foundation
import Testing
import SwabbleKit
@Suite struct WakeWordGateTests {
@Test func matchRequiresGapAfterTrigger() {
let transcript = "hey clawd do thing"
let segments = makeSegments(
transcript: transcript,
words: [
("hey", 0.0, 0.1),
("clawd", 0.2, 0.1),
("do", 0.35, 0.1),
("thing", 0.5, 0.1),
])
let config = WakeWordGateConfig(triggers: ["clawd"], minPostTriggerGap: 0.3)
#expect(WakeWordGate.match(transcript: transcript, segments: segments, config: config) == nil)
}
@Test func matchAllowsGapAndExtractsCommand() {
let transcript = "hey clawd do thing"
let segments = makeSegments(
transcript: transcript,
words: [
("hey", 0.0, 0.1),
("clawd", 0.2, 0.1),
("do", 0.9, 0.1),
("thing", 1.1, 0.1),
])
let config = WakeWordGateConfig(triggers: ["clawd"], minPostTriggerGap: 0.3)
let match = WakeWordGate.match(transcript: transcript, segments: segments, config: config)
#expect(match?.command == "do thing")
}
@Test func matchHandlesMultiWordTriggers() {
let transcript = "hey clawd do it"
let segments = makeSegments(
transcript: transcript,
words: [
("hey", 0.0, 0.1),
("clawd", 0.2, 0.1),
("do", 0.8, 0.1),
("it", 1.0, 0.1),
])
let config = WakeWordGateConfig(triggers: ["hey clawd"], minPostTriggerGap: 0.3)
let match = WakeWordGate.match(transcript: transcript, segments: segments, config: config)
#expect(match?.command == "do it")
}
}
private func makeSegments(
transcript: String,
words: [(String, TimeInterval, TimeInterval)])
-> [WakeWordSegment] {
var searchStart = transcript.startIndex
var output: [WakeWordSegment] = []
for (word, start, duration) in words {
let range = transcript.range(of: word, range: searchStart..<transcript.endIndex)
output.append(WakeWordSegment(text: word, start: start, duration: duration, range: range))
if let range { searchStart = range.upperBound }
}
return output
}

View File

@@ -0,0 +1,23 @@
import Foundation
import Testing
@testable import Swabble
@Test
func configRoundTrip() throws {
var cfg = SwabbleConfig()
cfg.wake.word = "robot"
let url = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString + ".json")
defer { try? FileManager.default.removeItem(at: url) }
try ConfigLoader.save(cfg, at: url)
let loaded = try ConfigLoader.load(at: url)
#expect(loaded.wake.word == "robot")
#expect(loaded.hook.prefix.contains("Voice swabble"))
}
@Test
func configMissingThrows() {
#expect(throws: ConfigError.missingConfig) {
_ = try ConfigLoader.load(at: FileManager.default.temporaryDirectory.appendingPathComponent("nope.json"))
}
}

33
Swabble/docs/spec.md Normal file
View File

@@ -0,0 +1,33 @@
# swabble — macOS 26 speech hook daemon (Swift 6.2)
Goal: brabble-style always-on voice hook for macOS 26 using Apple Speech.framework (SpeechAnalyzer + SpeechTranscriber) instead of whisper.cpp. Local-only, wake word gated, dispatches a shell hook with the transcript. Shared wake-gate utilities live in `SwabbleKit` for reuse by other apps (iOS/macOS).
## Requirements
- macOS 26+, Swift 6.2, Speech.framework with on-device assets.
- Local only; no network calls during transcription.
- Wake word gating (default "clawd" plus aliases) with bypass flag `--no-wake`.
- `SwabbleKit` target (multi-platform) providing wake-word gating helpers that can use speech segment timing to require a post-trigger gap.
- Hook execution with cooldown, min_chars, timeout, prefix, env vars.
- Simple config at `~/.config/swabble/config.json` (JSON, Codable) — no TOML.
- CLI implemented with Commander (SwiftPM package `steipete/Commander`); core types are available via the SwiftPM library product `Swabble` for embedding.
- Foreground `serve`; later launchd helper for start/stop/restart.
- File transcription command emitting txt or srt.
- Basic status/health surfaces and mic selection stubs.
## Architecture
- **CLI layer (Commander)**: Root command `swabble` with subcommands `serve`, `transcribe`, `test-hook`, `mic list|set`, `doctor`, `health`, `tail-log`. Runtime flags from Commander (`-v/--verbose`, `--json-output`, `--log-level`). Custom `--config` path applies everywhere.
- **Config**: `SwabbleConfig` Codable. Fields: audio device name/index, wake (enabled/word/aliases/sensitivity placeholder), hook (command/args/prefix/cooldown/min_chars/timeout/env), logging (level, format), transcripts (enabled, max kept), speech (locale, enableEtiquetteReplacements flag). Stored JSON; default written by `setup`.
- **Audio + Speech pipeline**: `SpeechPipeline` wraps `AVAudioEngine` input → `SpeechAnalyzer` with `SpeechTranscriber` module. Emits partial/final transcripts via async stream. Requests `.audioTimeRange` when transcripts enabled. Handles Speech permission and asset download prompts ahead of capture.
- **Wake gate**: CLI currently uses text-only keyword match; shared `SwabbleKit` gate can enforce a minimum pause between the wake word and the next token when speech segments are available. `--no-wake` disables gating.
- **Hook executor**: async `HookExecutor` spawns `Process` with configured args, prefix substitution `${hostname}`. Enforces cooldown + timeout; injects env `SWABBLE_TEXT`, `SWABBLE_PREFIX` plus user env map.
- **Transcripts store**: in-memory ring buffer; optional persisted JSON lines under `~/Library/Application Support/swabble/transcripts.log`.
- **Logging**: simple structured logger to stderr; respects log level.
## Out of scope (initial cut)
- Model management (Speech handles assets).
- Launchd helper (planned follow-up).
- Advanced wake-word detector (segment-aware gate now lives in `SwabbleKit`; CLI still text-only until segment timing is plumbed through).
## Open decisions
- Whether to expose a UNIX control socket for `status`/`health` (currently planned as stdin/out direct calls).
- Hook redaction (PII) parity with brabble — placeholder boolean, no implementation yet.

10
Swabble/scripts/format.sh Executable file
View File

@@ -0,0 +1,10 @@
#!/bin/bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PEEKABOO_ROOT="${ROOT}/../peekaboo"
if [ -f "${PEEKABOO_ROOT}/.swiftformat" ]; then
CONFIG="${PEEKABOO_ROOT}/.swiftformat"
else
CONFIG="${ROOT}/.swiftformat"
fi
swiftformat --config "$CONFIG" "$ROOT/Sources"

14
Swabble/scripts/lint.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PEEKABOO_ROOT="${ROOT}/../peekaboo"
if [ -f "${PEEKABOO_ROOT}/.swiftlint.yml" ]; then
CONFIG="${PEEKABOO_ROOT}/.swiftlint.yml"
else
CONFIG="$ROOT/.swiftlint.yml"
fi
if ! command -v swiftlint >/dev/null; then
echo "swiftlint not installed" >&2
exit 1
fi
swiftlint --config "$CONFIG"

32
appcast.xml Normal file
View File

@@ -0,0 +1,32 @@
<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0"
xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<channel>
<title>Clawdis Updates</title>
<link>https://raw.githubusercontent.com/steipete/clawdis/main/appcast.xml</link>
<description>Signed update feed for the Clawdis macOS companion app.</description>
<item>
<title>Clawdis 2.0.0-beta2</title>
<sparkle:releaseNotesLink>https://github.com/steipete/clawdis/releases/tag/v2.0.0-beta2</sparkle:releaseNotesLink>
<pubDate>Sun, 21 Dec 2025 02:25:39 +0000</pubDate>
<enclosure url="https://github.com/steipete/clawdis/releases/download/v2.0.0-beta2/Clawdis-2.0.0-beta2.zip"
sparkle:edSignature="voRWLh2Cbg/i2KtUV6ci/MW3b7hK/u1ZPoiryKs+S36ua3xnc51R97JGwmIaToCfTHg2mgFWF7M6qppfe7YsAw=="
sparkle:version="2.0.0-beta2"
sparkle:shortVersionString="2.0.0-beta2"
length="67435891"
type="application/octet-stream" />
</item>
<item>
<title>Clawdis 2.0.0-beta1</title>
<sparkle:releaseNotesLink>https://github.com/steipete/clawdis/releases/tag/v2.0.0-beta1</sparkle:releaseNotesLink>
<pubDate>Fri, 19 Dec 2025 17:19:50 +0000</pubDate>
<enclosure url="https://github.com/steipete/clawdis/releases/download/v2.0.0-beta1/Clawdis-2.0.0-beta1.zip"
sparkle:edSignature="oEpGD46U4ZyBBSY9/piUIFDJU+KlFB751JIWOW2yS0sRNHKszyG5khDHg9o9bV9Zo8DOCNF/HOi88jmtHJAaCQ=="
sparkle:version="2.0.0-beta1"
sparkle:shortVersionString="2.0.0-beta1"
length="72410016"
type="application/octet-stream" />
</item>
</channel>
</rss>

5
apps/android/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
.gradle/
**/build/
local.properties
.idea/
**/*.iml

51
apps/android/README.md Normal file
View File

@@ -0,0 +1,51 @@
## Clawdis Node (Android) (internal)
Modern Android node app: connects to the **Gateway-owned bridge** (`_clawdis-bridge._tcp`) over TCP and exposes **Canvas + Chat + Camera**.
Notes:
- The node keeps the connection alive via a **foreground service** (persistent notification with a Disconnect action).
- Chat always uses the shared session key **`main`** (same session across iOS/macOS/WebChat/Android).
- Supports modern Android only (`minSdk 31`, Kotlin + Jetpack Compose).
## Open in Android Studio
- Open the folder `apps/android`.
## Build / Run
```bash
cd apps/android
./gradlew :app:assembleDebug
./gradlew :app:installDebug
./gradlew :app:testDebugUnitTest
```
`gradlew` auto-detects the Android SDK at `~/Library/Android/sdk` (macOS default) if `ANDROID_SDK_ROOT` / `ANDROID_HOME` are unset.
## Connect / Pair
1) Start the gateway (on your “master” machine):
```bash
pnpm clawdis gateway --port 18789 --verbose
```
2) In the Android app:
- Open **Settings**
- Either select a discovered bridge under **Discovered Bridges**, or use **Advanced → Manual Bridge** (host + port).
3) Approve pairing (on the gateway machine):
```bash
clawdis nodes pending
clawdis nodes approve <requestId>
```
More details: `docs/android/connect.md`.
## Permissions
- Discovery:
- Android 13+ (`API 33+`): `NEARBY_WIFI_DEVICES`
- Android 12 and below: `ACCESS_FINE_LOCATION` (required for NSD scanning)
- Foreground service notification (Android 13+): `POST_NOTIFICATIONS`
- Camera:
- `CAMERA` for `camera.snap` and `camera.clip`
- `RECORD_AUDIO` for `camera.clip` when `includeAudio=true`

View File

@@ -0,0 +1,95 @@
plugins {
id("com.android.application")
id("org.jetbrains.kotlin.android")
id("org.jetbrains.kotlin.plugin.compose")
id("org.jetbrains.kotlin.plugin.serialization")
}
android {
namespace = "com.steipete.clawdis.node"
compileSdk = 36
sourceSets {
getByName("main") {
assets.srcDir(file("../../shared/ClawdisKit/Sources/ClawdisKit/Resources"))
}
}
defaultConfig {
applicationId = "com.steipete.clawdis.node"
minSdk = 31
targetSdk = 36
versionCode = 1
versionName = "0.1"
}
buildTypes {
release {
isMinifyEnabled = false
}
}
buildFeatures {
compose = true
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
packaging {
resources {
excludes += "/META-INF/{AL2.0,LGPL2.1}"
}
}
lint {
disable += setOf("IconLauncherShape")
}
}
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
}
}
dependencies {
val composeBom = platform("androidx.compose:compose-bom:2025.12.00")
implementation(composeBom)
androidTestImplementation(composeBom)
implementation("androidx.core:core-ktx:1.17.0")
implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.10.0")
implementation("androidx.activity:activity-compose:1.12.2")
implementation("androidx.compose.ui:ui")
implementation("androidx.compose.ui:ui-tooling-preview")
implementation("androidx.compose.material3:material3")
implementation("androidx.compose.material:material-icons-extended")
implementation("androidx.navigation:navigation-compose:2.9.6")
debugImplementation("androidx.compose.ui:ui-tooling")
// Material Components (XML theme + resources)
implementation("com.google.android.material:material:1.13.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2")
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.9.0")
implementation("androidx.security:security-crypto:1.1.0")
// CameraX (for node.invoke camera.* parity)
implementation("androidx.camera:camera-core:1.5.2")
implementation("androidx.camera:camera-camera2:1.5.2")
implementation("androidx.camera:camera-lifecycle:1.5.2")
implementation("androidx.camera:camera-video:1.5.2")
implementation("androidx.camera:camera-view:1.5.2")
// Unicast DNS-SD (Wide-Area Bonjour) for tailnet discovery domains.
implementation("dnsjava:dnsjava:3.6.3")
testImplementation("junit:junit:4.13.2")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.10.2")
}

View File

@@ -0,0 +1,48 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission
android:name="android.permission.NEARBY_WIFI_DEVICES"
android:usesPermissionFlags="neverForLocation" />
<uses-permission
android:name="android.permission.ACCESS_FINE_LOCATION"
android:maxSdkVersion="32" />
<uses-permission
android:name="android.permission.ACCESS_COARSE_LOCATION"
android:maxSdkVersion="32" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-feature
android:name="android.hardware.camera"
android:required="false" />
<application
android:name=".NodeApp"
android:allowBackup="true"
android:dataExtractionRules="@xml/data_extraction_rules"
android:fullBackupContent="@xml/backup_rules"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name"
android:supportsRtl="true"
android:networkSecurityConfig="@xml/network_security_config"
android:theme="@style/Theme.ClawdisNode">
<service
android:name=".NodeForegroundService"
android:exported="false"
android:foregroundServiceType="dataSync|microphone|mediaProjection" />
<activity
android:name=".MainActivity"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>

View File

@@ -0,0 +1,15 @@
package com.steipete.clawdis.node
enum class CameraHudKind {
Photo,
Recording,
Success,
Error,
}
data class CameraHudState(
val token: Long,
val kind: CameraHudKind,
val message: String,
)

View File

@@ -0,0 +1,26 @@
package com.steipete.clawdis.node
import android.content.Context
import android.os.Build
import android.provider.Settings
object DeviceNames {
fun bestDefaultNodeName(context: Context): String {
val deviceName =
runCatching {
Settings.Global.getString(context.contentResolver, "device_name")
}
.getOrNull()
?.trim()
.orEmpty()
if (deviceName.isNotEmpty()) return deviceName
val model =
listOfNotNull(Build.MANUFACTURER?.takeIf { it.isNotBlank() }, Build.MODEL?.takeIf { it.isNotBlank() })
.joinToString(" ")
.trim()
return model.ifEmpty { "Android Node" }
}
}

View File

@@ -0,0 +1,129 @@
package com.steipete.clawdis.node
import android.Manifest
import android.content.pm.ApplicationInfo
import android.os.Bundle
import android.os.Build
import android.view.WindowManager
import android.webkit.WebView
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.viewModels
import androidx.compose.material3.Surface
import androidx.compose.ui.Modifier
import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.lifecycleScope
import androidx.lifecycle.repeatOnLifecycle
import com.steipete.clawdis.node.ui.RootScreen
import com.steipete.clawdis.node.ui.ClawdisTheme
import kotlinx.coroutines.launch
class MainActivity : ComponentActivity() {
private val viewModel: MainViewModel by viewModels()
private lateinit var permissionRequester: PermissionRequester
private lateinit var screenCaptureRequester: ScreenCaptureRequester
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
val isDebuggable = (applicationInfo.flags and ApplicationInfo.FLAG_DEBUGGABLE) != 0
WebView.setWebContentsDebuggingEnabled(isDebuggable)
applyImmersiveMode()
requestDiscoveryPermissionsIfNeeded()
requestNotificationPermissionIfNeeded()
NodeForegroundService.start(this)
permissionRequester = PermissionRequester(this)
screenCaptureRequester = ScreenCaptureRequester(this)
viewModel.camera.attachLifecycleOwner(this)
viewModel.camera.attachPermissionRequester(permissionRequester)
viewModel.screenRecorder.attachScreenCaptureRequester(screenCaptureRequester)
viewModel.screenRecorder.attachPermissionRequester(permissionRequester)
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.STARTED) {
viewModel.preventSleep.collect { enabled ->
if (enabled) {
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
} else {
window.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
}
}
}
}
setContent {
ClawdisTheme {
Surface(modifier = Modifier) {
RootScreen(viewModel = viewModel)
}
}
}
}
override fun onResume() {
super.onResume()
applyImmersiveMode()
}
override fun onWindowFocusChanged(hasFocus: Boolean) {
super.onWindowFocusChanged(hasFocus)
if (hasFocus) {
applyImmersiveMode()
}
}
override fun onStart() {
super.onStart()
viewModel.setForeground(true)
}
override fun onStop() {
viewModel.setForeground(false)
super.onStop()
}
private fun applyImmersiveMode() {
WindowCompat.setDecorFitsSystemWindows(window, false)
val controller = WindowInsetsControllerCompat(window, window.decorView)
controller.systemBarsBehavior =
WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
controller.hide(WindowInsetsCompat.Type.systemBars())
}
private fun requestDiscoveryPermissionsIfNeeded() {
if (Build.VERSION.SDK_INT >= 33) {
val ok =
ContextCompat.checkSelfPermission(
this,
Manifest.permission.NEARBY_WIFI_DEVICES,
) == android.content.pm.PackageManager.PERMISSION_GRANTED
if (!ok) {
requestPermissions(arrayOf(Manifest.permission.NEARBY_WIFI_DEVICES), 100)
}
} else {
val ok =
ContextCompat.checkSelfPermission(
this,
Manifest.permission.ACCESS_FINE_LOCATION,
) == android.content.pm.PackageManager.PERMISSION_GRANTED
if (!ok) {
requestPermissions(arrayOf(Manifest.permission.ACCESS_FINE_LOCATION), 101)
}
}
}
private fun requestNotificationPermissionIfNeeded() {
if (Build.VERSION.SDK_INT < 33) return
val ok =
ContextCompat.checkSelfPermission(
this,
Manifest.permission.POST_NOTIFICATIONS,
) == android.content.pm.PackageManager.PERMISSION_GRANTED
if (!ok) {
requestPermissions(arrayOf(Manifest.permission.POST_NOTIFICATIONS), 102)
}
}
}

View File

@@ -0,0 +1,141 @@
package com.steipete.clawdis.node
import android.app.Application
import androidx.lifecycle.AndroidViewModel
import com.steipete.clawdis.node.bridge.BridgeEndpoint
import com.steipete.clawdis.node.chat.OutgoingAttachment
import com.steipete.clawdis.node.node.CameraCaptureManager
import com.steipete.clawdis.node.node.CanvasController
import com.steipete.clawdis.node.node.ScreenRecordManager
import kotlinx.coroutines.flow.StateFlow
class MainViewModel(app: Application) : AndroidViewModel(app) {
private val runtime: NodeRuntime = (app as NodeApp).runtime
val canvas: CanvasController = runtime.canvas
val camera: CameraCaptureManager = runtime.camera
val screenRecorder: ScreenRecordManager = runtime.screenRecorder
val bridges: StateFlow<List<BridgeEndpoint>> = runtime.bridges
val discoveryStatusText: StateFlow<String> = runtime.discoveryStatusText
val isConnected: StateFlow<Boolean> = runtime.isConnected
val statusText: StateFlow<String> = runtime.statusText
val serverName: StateFlow<String?> = runtime.serverName
val remoteAddress: StateFlow<String?> = runtime.remoteAddress
val cameraHud: StateFlow<CameraHudState?> = runtime.cameraHud
val cameraFlashToken: StateFlow<Long> = runtime.cameraFlashToken
val instanceId: StateFlow<String> = runtime.instanceId
val displayName: StateFlow<String> = runtime.displayName
val cameraEnabled: StateFlow<Boolean> = runtime.cameraEnabled
val preventSleep: StateFlow<Boolean> = runtime.preventSleep
val wakeWords: StateFlow<List<String>> = runtime.wakeWords
val voiceWakeMode: StateFlow<VoiceWakeMode> = runtime.voiceWakeMode
val voiceWakeStatusText: StateFlow<String> = runtime.voiceWakeStatusText
val voiceWakeIsListening: StateFlow<Boolean> = runtime.voiceWakeIsListening
val manualEnabled: StateFlow<Boolean> = runtime.manualEnabled
val manualHost: StateFlow<String> = runtime.manualHost
val manualPort: StateFlow<Int> = runtime.manualPort
val canvasDebugStatusEnabled: StateFlow<Boolean> = runtime.canvasDebugStatusEnabled
val chatSessionKey: StateFlow<String> = runtime.chatSessionKey
val chatSessionId: StateFlow<String?> = runtime.chatSessionId
val chatMessages = runtime.chatMessages
val chatError: StateFlow<String?> = runtime.chatError
val chatHealthOk: StateFlow<Boolean> = runtime.chatHealthOk
val chatThinkingLevel: StateFlow<String> = runtime.chatThinkingLevel
val chatStreamingAssistantText: StateFlow<String?> = runtime.chatStreamingAssistantText
val chatPendingToolCalls = runtime.chatPendingToolCalls
val chatSessions = runtime.chatSessions
val pendingRunCount: StateFlow<Int> = runtime.pendingRunCount
fun setForeground(value: Boolean) {
runtime.setForeground(value)
}
fun setDisplayName(value: String) {
runtime.setDisplayName(value)
}
fun setCameraEnabled(value: Boolean) {
runtime.setCameraEnabled(value)
}
fun setPreventSleep(value: Boolean) {
runtime.setPreventSleep(value)
}
fun setManualEnabled(value: Boolean) {
runtime.setManualEnabled(value)
}
fun setManualHost(value: String) {
runtime.setManualHost(value)
}
fun setManualPort(value: Int) {
runtime.setManualPort(value)
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
runtime.setCanvasDebugStatusEnabled(value)
}
fun setWakeWords(words: List<String>) {
runtime.setWakeWords(words)
}
fun resetWakeWordsDefaults() {
runtime.resetWakeWordsDefaults()
}
fun setVoiceWakeMode(mode: VoiceWakeMode) {
runtime.setVoiceWakeMode(mode)
}
fun connect(endpoint: BridgeEndpoint) {
runtime.connect(endpoint)
}
fun connectManual() {
runtime.connectManual()
}
fun disconnect() {
runtime.disconnect()
}
fun handleCanvasA2UIActionFromWebView(payloadJson: String) {
runtime.handleCanvasA2UIActionFromWebView(payloadJson)
}
fun loadChat(sessionKey: String = "main") {
runtime.loadChat(sessionKey)
}
fun refreshChat() {
runtime.refreshChat()
}
fun refreshChatSessions(limit: Int? = null) {
runtime.refreshChatSessions(limit = limit)
}
fun setChatThinkingLevel(level: String) {
runtime.setChatThinkingLevel(level)
}
fun switchChatSession(sessionKey: String) {
runtime.switchChatSession(sessionKey)
}
fun abortChat() {
runtime.abortChat()
}
fun sendChat(message: String, thinking: String, attachments: List<OutgoingAttachment>) {
runtime.sendChat(message = message, thinking = thinking, attachments = attachments)
}
}

View File

@@ -0,0 +1,8 @@
package com.steipete.clawdis.node
import android.app.Application
class NodeApp : Application() {
val runtime: NodeRuntime by lazy { NodeRuntime(this) }
}

View File

@@ -0,0 +1,163 @@
package com.steipete.clawdis.node
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.Service
import android.app.PendingIntent
import android.Manifest
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.content.pm.ServiceInfo
import androidx.core.app.NotificationCompat
import androidx.core.content.ContextCompat
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.launch
class NodeForegroundService : Service() {
private val scope: CoroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
private var notificationJob: Job? = null
private var lastRequiresMic = false
private var didStartForeground = false
override fun onCreate() {
super.onCreate()
ensureChannel()
val initial = buildNotification(title = "Clawdis Node", text = "Starting…")
startForegroundWithTypes(notification = initial, requiresMic = false)
val runtime = (application as NodeApp).runtime
notificationJob =
scope.launch {
combine(
runtime.statusText,
runtime.serverName,
runtime.isConnected,
runtime.voiceWakeMode,
runtime.voiceWakeIsListening,
) { status, server, connected, voiceMode, voiceListening ->
Quint(status, server, connected, voiceMode, voiceListening)
}.collect { (status, server, connected, voiceMode, voiceListening) ->
val title = if (connected) "Clawdis Node · Connected" else "Clawdis Node"
val voiceSuffix =
if (voiceMode == VoiceWakeMode.Always) {
if (voiceListening) " · Voice Wake: Listening" else " · Voice Wake: Paused"
} else {
""
}
val text = (server?.let { "$status · $it" } ?: status) + voiceSuffix
val requiresMic =
voiceMode == VoiceWakeMode.Always && hasRecordAudioPermission()
startForegroundWithTypes(
notification = buildNotification(title = title, text = text),
requiresMic = requiresMic,
)
}
}
}
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
when (intent?.action) {
ACTION_STOP -> {
(application as NodeApp).runtime.disconnect()
stopSelf()
return START_NOT_STICKY
}
}
// Keep running; connection is managed by NodeRuntime (auto-reconnect + manual).
return START_STICKY
}
override fun onDestroy() {
notificationJob?.cancel()
scope.cancel()
super.onDestroy()
}
override fun onBind(intent: Intent?) = null
private fun ensureChannel() {
val mgr = getSystemService(NotificationManager::class.java)
val channel =
NotificationChannel(
CHANNEL_ID,
"Connection",
NotificationManager.IMPORTANCE_LOW,
).apply {
description = "Clawdis node connection status"
setShowBadge(false)
}
mgr.createNotificationChannel(channel)
}
private fun buildNotification(title: String, text: String): Notification {
val stopIntent = Intent(this, NodeForegroundService::class.java).setAction(ACTION_STOP)
val flags = PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
val stopPending = PendingIntent.getService(this, 2, stopIntent, flags)
return NotificationCompat.Builder(this, CHANNEL_ID)
.setSmallIcon(R.mipmap.ic_launcher)
.setContentTitle(title)
.setContentText(text)
.setOngoing(true)
.setOnlyAlertOnce(true)
.setForegroundServiceBehavior(NotificationCompat.FOREGROUND_SERVICE_IMMEDIATE)
.addAction(0, "Disconnect", stopPending)
.build()
}
private fun updateNotification(notification: Notification) {
val mgr = getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
mgr.notify(NOTIFICATION_ID, notification)
}
private fun startForegroundWithTypes(notification: Notification, requiresMic: Boolean) {
if (didStartForeground && requiresMic == lastRequiresMic) {
updateNotification(notification)
return
}
lastRequiresMic = requiresMic
val types =
if (requiresMic) {
ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC or ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
} else {
ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC
}
startForeground(NOTIFICATION_ID, notification, types)
didStartForeground = true
}
private fun hasRecordAudioPermission(): Boolean {
return (
ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
)
}
companion object {
private const val CHANNEL_ID = "connection"
private const val NOTIFICATION_ID = 1
private const val ACTION_STOP = "com.steipete.clawdis.node.action.STOP"
fun start(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java)
context.startForegroundService(intent)
}
fun stop(context: Context) {
val intent = Intent(context, NodeForegroundService::class.java).setAction(ACTION_STOP)
context.startService(intent)
}
}
}
private data class Quint<A, B, C, D, E>(val first: A, val second: B, val third: C, val fourth: D, val fifth: E)

View File

@@ -0,0 +1,907 @@
package com.steipete.clawdis.node
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import android.os.SystemClock
import androidx.core.content.ContextCompat
import com.steipete.clawdis.node.chat.ChatController
import com.steipete.clawdis.node.chat.ChatMessage
import com.steipete.clawdis.node.chat.ChatPendingToolCall
import com.steipete.clawdis.node.chat.ChatSessionEntry
import com.steipete.clawdis.node.chat.OutgoingAttachment
import com.steipete.clawdis.node.bridge.BridgeDiscovery
import com.steipete.clawdis.node.bridge.BridgeEndpoint
import com.steipete.clawdis.node.bridge.BridgePairingClient
import com.steipete.clawdis.node.bridge.BridgeSession
import com.steipete.clawdis.node.node.CameraCaptureManager
import com.steipete.clawdis.node.node.CanvasController
import com.steipete.clawdis.node.node.ScreenRecordManager
import com.steipete.clawdis.node.protocol.ClawdisCapability
import com.steipete.clawdis.node.protocol.ClawdisCameraCommand
import com.steipete.clawdis.node.protocol.ClawdisCanvasA2UIAction
import com.steipete.clawdis.node.protocol.ClawdisCanvasA2UICommand
import com.steipete.clawdis.node.protocol.ClawdisCanvasCommand
import com.steipete.clawdis.node.protocol.ClawdisScreenCommand
import com.steipete.clawdis.node.voice.VoiceWakeManager
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.collect
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import java.util.concurrent.atomic.AtomicLong
class NodeRuntime(context: Context) {
private val appContext = context.applicationContext
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val prefs = SecurePrefs(appContext)
val canvas = CanvasController()
val camera = CameraCaptureManager(appContext)
val screenRecorder = ScreenRecordManager(appContext)
private val json = Json { ignoreUnknownKeys = true }
private val externalAudioCaptureActive = MutableStateFlow(false)
private val voiceWake: VoiceWakeManager by lazy {
VoiceWakeManager(
context = appContext,
scope = scope,
onCommand = { command ->
session.sendEvent(
event = "agent.request",
payloadJson =
buildJsonObject {
put("message", JsonPrimitive(command))
put("sessionKey", JsonPrimitive("main"))
put("thinking", JsonPrimitive(chatThinkingLevel.value))
put("deliver", JsonPrimitive(false))
}.toString(),
)
},
)
}
val voiceWakeIsListening: StateFlow<Boolean>
get() = voiceWake.isListening
val voiceWakeStatusText: StateFlow<String>
get() = voiceWake.statusText
private val discovery = BridgeDiscovery(appContext, scope = scope)
val bridges: StateFlow<List<BridgeEndpoint>> = discovery.bridges
val discoveryStatusText: StateFlow<String> = discovery.statusText
private val _isConnected = MutableStateFlow(false)
val isConnected: StateFlow<Boolean> = _isConnected.asStateFlow()
private val _statusText = MutableStateFlow("Offline")
val statusText: StateFlow<String> = _statusText.asStateFlow()
private val cameraHudSeq = AtomicLong(0)
private val _cameraHud = MutableStateFlow<CameraHudState?>(null)
val cameraHud: StateFlow<CameraHudState?> = _cameraHud.asStateFlow()
private val _cameraFlashToken = MutableStateFlow(0L)
val cameraFlashToken: StateFlow<Long> = _cameraFlashToken.asStateFlow()
private val _serverName = MutableStateFlow<String?>(null)
val serverName: StateFlow<String?> = _serverName.asStateFlow()
private val _remoteAddress = MutableStateFlow<String?>(null)
val remoteAddress: StateFlow<String?> = _remoteAddress.asStateFlow()
private val _isForeground = MutableStateFlow(true)
val isForeground: StateFlow<Boolean> = _isForeground.asStateFlow()
private var lastAutoA2uiUrl: String? = null
private val session =
BridgeSession(
scope = scope,
onConnected = { name, remote ->
_statusText.value = "Connected"
_serverName.value = name
_remoteAddress.value = remote
_isConnected.value = true
scope.launch { refreshWakeWordsFromGateway() }
maybeNavigateToA2uiOnConnect()
},
onDisconnected = { message -> handleSessionDisconnected(message) },
onEvent = { event, payloadJson ->
handleBridgeEvent(event, payloadJson)
},
onInvoke = { req ->
handleInvoke(req.command, req.paramsJson)
},
)
private val chat = ChatController(scope = scope, session = session, json = json)
private fun handleSessionDisconnected(message: String) {
_statusText.value = message
_serverName.value = null
_remoteAddress.value = null
_isConnected.value = false
chat.onDisconnected(message)
showLocalCanvasOnDisconnect()
}
private fun maybeNavigateToA2uiOnConnect() {
val a2uiUrl = resolveA2uiHostUrl() ?: return
val current = canvas.currentUrl()?.trim().orEmpty()
if (current.isEmpty() || current == lastAutoA2uiUrl) {
lastAutoA2uiUrl = a2uiUrl
canvas.navigate(a2uiUrl)
}
}
private fun showLocalCanvasOnDisconnect() {
lastAutoA2uiUrl = null
canvas.navigate("")
}
val instanceId: StateFlow<String> = prefs.instanceId
val displayName: StateFlow<String> = prefs.displayName
val cameraEnabled: StateFlow<Boolean> = prefs.cameraEnabled
val preventSleep: StateFlow<Boolean> = prefs.preventSleep
val wakeWords: StateFlow<List<String>> = prefs.wakeWords
val voiceWakeMode: StateFlow<VoiceWakeMode> = prefs.voiceWakeMode
val manualEnabled: StateFlow<Boolean> = prefs.manualEnabled
val manualHost: StateFlow<String> = prefs.manualHost
val manualPort: StateFlow<Int> = prefs.manualPort
val lastDiscoveredStableId: StateFlow<String> = prefs.lastDiscoveredStableId
val canvasDebugStatusEnabled: StateFlow<Boolean> = prefs.canvasDebugStatusEnabled
private var didAutoConnect = false
private var suppressWakeWordsSync = false
private var wakeWordsSyncJob: Job? = null
val chatSessionKey: StateFlow<String> = chat.sessionKey
val chatSessionId: StateFlow<String?> = chat.sessionId
val chatMessages: StateFlow<List<ChatMessage>> = chat.messages
val chatError: StateFlow<String?> = chat.errorText
val chatHealthOk: StateFlow<Boolean> = chat.healthOk
val chatThinkingLevel: StateFlow<String> = chat.thinkingLevel
val chatStreamingAssistantText: StateFlow<String?> = chat.streamingAssistantText
val chatPendingToolCalls: StateFlow<List<ChatPendingToolCall>> = chat.pendingToolCalls
val chatSessions: StateFlow<List<ChatSessionEntry>> = chat.sessions
val pendingRunCount: StateFlow<Int> = chat.pendingRunCount
init {
scope.launch {
combine(
voiceWakeMode,
isForeground,
externalAudioCaptureActive,
wakeWords,
) { mode, foreground, externalAudio, words ->
Quad(mode, foreground, externalAudio, words)
}.distinctUntilChanged()
.collect { (mode, foreground, externalAudio, words) ->
voiceWake.setTriggerWords(words)
val shouldListen =
when (mode) {
VoiceWakeMode.Off -> false
VoiceWakeMode.Foreground -> foreground
VoiceWakeMode.Always -> true
} && !externalAudio
if (!shouldListen) {
voiceWake.stop(statusText = if (mode == VoiceWakeMode.Off) "Off" else "Paused")
return@collect
}
if (!hasRecordAudioPermission()) {
voiceWake.stop(statusText = "Microphone permission required")
return@collect
}
voiceWake.start()
}
}
scope.launch(Dispatchers.Default) {
bridges.collect { list ->
if (list.isNotEmpty()) {
// Persist the last discovered bridge (best-effort UX parity with iOS).
prefs.setLastDiscoveredStableId(list.last().stableId)
}
if (didAutoConnect) return@collect
if (_isConnected.value) return@collect
val token = prefs.loadBridgeToken()
if (token.isNullOrBlank()) return@collect
if (manualEnabled.value) {
val host = manualHost.value.trim()
val port = manualPort.value
if (host.isNotEmpty() && port in 1..65535) {
didAutoConnect = true
connect(BridgeEndpoint.manual(host = host, port = port))
}
return@collect
}
val targetStableId = lastDiscoveredStableId.value.trim()
if (targetStableId.isEmpty()) return@collect
val target = list.firstOrNull { it.stableId == targetStableId } ?: return@collect
didAutoConnect = true
connect(target)
}
}
scope.launch {
combine(
canvasDebugStatusEnabled,
statusText,
serverName,
remoteAddress,
) { debugEnabled, status, server, remote ->
Quad(debugEnabled, status, server, remote)
}.distinctUntilChanged()
.collect { (debugEnabled, status, server, remote) ->
canvas.setDebugStatusEnabled(debugEnabled)
if (!debugEnabled) return@collect
canvas.setDebugStatus(status, server ?: remote)
}
}
}
fun setForeground(value: Boolean) {
_isForeground.value = value
}
fun setDisplayName(value: String) {
prefs.setDisplayName(value)
}
fun setCameraEnabled(value: Boolean) {
prefs.setCameraEnabled(value)
}
fun setPreventSleep(value: Boolean) {
prefs.setPreventSleep(value)
}
fun setManualEnabled(value: Boolean) {
prefs.setManualEnabled(value)
}
fun setManualHost(value: String) {
prefs.setManualHost(value)
}
fun setManualPort(value: Int) {
prefs.setManualPort(value)
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
prefs.setCanvasDebugStatusEnabled(value)
}
fun setWakeWords(words: List<String>) {
prefs.setWakeWords(words)
scheduleWakeWordsSyncIfNeeded()
}
fun resetWakeWordsDefaults() {
setWakeWords(SecurePrefs.defaultWakeWords)
}
fun setVoiceWakeMode(mode: VoiceWakeMode) {
prefs.setVoiceWakeMode(mode)
}
fun connect(endpoint: BridgeEndpoint) {
scope.launch {
_statusText.value = "Connecting…"
val storedToken = prefs.loadBridgeToken()
val modelIdentifier = listOfNotNull(Build.MANUFACTURER, Build.MODEL)
.joinToString(" ")
.trim()
.ifEmpty { null }
val invokeCommands =
buildList {
add(ClawdisCanvasCommand.Present.rawValue)
add(ClawdisCanvasCommand.Hide.rawValue)
add(ClawdisCanvasCommand.Navigate.rawValue)
add(ClawdisCanvasCommand.Eval.rawValue)
add(ClawdisCanvasCommand.Snapshot.rawValue)
add(ClawdisCanvasA2UICommand.Push.rawValue)
add(ClawdisCanvasA2UICommand.PushJSONL.rawValue)
add(ClawdisCanvasA2UICommand.Reset.rawValue)
add(ClawdisScreenCommand.Record.rawValue)
if (cameraEnabled.value) {
add(ClawdisCameraCommand.Snap.rawValue)
add(ClawdisCameraCommand.Clip.rawValue)
}
}
val resolved =
if (storedToken.isNullOrBlank()) {
_statusText.value = "Pairing…"
val caps = buildList {
add(ClawdisCapability.Canvas.rawValue)
add(ClawdisCapability.Screen.rawValue)
if (cameraEnabled.value) add(ClawdisCapability.Camera.rawValue)
if (voiceWakeMode.value != VoiceWakeMode.Off && hasRecordAudioPermission()) {
add(ClawdisCapability.VoiceWake.rawValue)
}
}
BridgePairingClient().pairAndHello(
endpoint = endpoint,
hello =
BridgePairingClient.Hello(
nodeId = instanceId.value,
displayName = displayName.value,
token = null,
platform = "Android",
version = "dev",
deviceFamily = "Android",
modelIdentifier = modelIdentifier,
caps = caps,
commands = invokeCommands,
),
)
} else {
BridgePairingClient.PairResult(ok = true, token = storedToken.trim())
}
if (!resolved.ok || resolved.token.isNullOrBlank()) {
_statusText.value = "Failed: pairing required"
return@launch
}
val authToken = requireNotNull(resolved.token).trim()
prefs.saveBridgeToken(authToken)
session.connect(
endpoint = endpoint,
hello =
BridgeSession.Hello(
nodeId = instanceId.value,
displayName = displayName.value,
token = authToken,
platform = "Android",
version = "dev",
deviceFamily = "Android",
modelIdentifier = modelIdentifier,
caps =
buildList {
add(ClawdisCapability.Canvas.rawValue)
add(ClawdisCapability.Screen.rawValue)
if (cameraEnabled.value) add(ClawdisCapability.Camera.rawValue)
if (voiceWakeMode.value != VoiceWakeMode.Off && hasRecordAudioPermission()) {
add(ClawdisCapability.VoiceWake.rawValue)
}
},
commands = invokeCommands,
),
)
}
}
private fun hasRecordAudioPermission(): Boolean {
return (
ContextCompat.checkSelfPermission(appContext, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
)
}
fun connectManual() {
val host = manualHost.value.trim()
val port = manualPort.value
if (host.isEmpty() || port <= 0 || port > 65535) {
_statusText.value = "Failed: invalid manual host/port"
return
}
connect(BridgeEndpoint.manual(host = host, port = port))
}
fun disconnect() {
session.disconnect()
}
fun handleCanvasA2UIActionFromWebView(payloadJson: String) {
scope.launch {
val trimmed = payloadJson.trim()
if (trimmed.isEmpty()) return@launch
val root =
try {
json.parseToJsonElement(trimmed).asObjectOrNull() ?: return@launch
} catch (_: Throwable) {
return@launch
}
val userActionObj = (root["userAction"] as? JsonObject) ?: root
val actionId = (userActionObj["id"] as? JsonPrimitive)?.content?.trim().orEmpty().ifEmpty {
java.util.UUID.randomUUID().toString()
}
val name = ClawdisCanvasA2UIAction.extractActionName(userActionObj) ?: return@launch
val surfaceId =
(userActionObj["surfaceId"] as? JsonPrimitive)?.content?.trim().orEmpty().ifEmpty { "main" }
val sourceComponentId =
(userActionObj["sourceComponentId"] as? JsonPrimitive)?.content?.trim().orEmpty().ifEmpty { "-" }
val contextJson = (userActionObj["context"] as? JsonObject)?.toString()
val sessionKey = "main"
val message =
ClawdisCanvasA2UIAction.formatAgentMessage(
actionName = name,
sessionKey = sessionKey,
surfaceId = surfaceId,
sourceComponentId = sourceComponentId,
host = displayName.value,
instanceId = instanceId.value.lowercase(),
contextJson = contextJson,
)
val connected = isConnected.value
var error: String? = null
if (connected) {
try {
session.sendEvent(
event = "agent.request",
payloadJson =
buildJsonObject {
put("message", JsonPrimitive(message))
put("sessionKey", JsonPrimitive(sessionKey))
put("thinking", JsonPrimitive("low"))
put("deliver", JsonPrimitive(false))
put("key", JsonPrimitive(actionId))
}.toString(),
)
} catch (e: Throwable) {
error = e.message ?: "send failed"
}
} else {
error = "bridge not connected"
}
try {
canvas.eval(
ClawdisCanvasA2UIAction.jsDispatchA2UIActionStatus(
actionId = actionId,
ok = connected && error == null,
error = error,
),
)
} catch (_: Throwable) {
// ignore
}
}
}
fun loadChat(sessionKey: String = "main") {
chat.load(sessionKey)
}
fun refreshChat() {
chat.refresh()
}
fun refreshChatSessions(limit: Int? = null) {
chat.refreshSessions(limit = limit)
}
fun setChatThinkingLevel(level: String) {
chat.setThinkingLevel(level)
}
fun switchChatSession(sessionKey: String) {
chat.switchSession(sessionKey)
}
fun abortChat() {
chat.abort()
}
fun sendChat(message: String, thinking: String, attachments: List<OutgoingAttachment>) {
chat.sendMessage(message = message, thinkingLevel = thinking, attachments = attachments)
}
private fun handleBridgeEvent(event: String, payloadJson: String?) {
if (event == "voicewake.changed") {
if (payloadJson.isNullOrBlank()) return
try {
val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
return
}
chat.handleBridgeEvent(event, payloadJson)
}
private fun applyWakeWordsFromGateway(words: List<String>) {
suppressWakeWordsSync = true
prefs.setWakeWords(words)
suppressWakeWordsSync = false
}
private fun scheduleWakeWordsSyncIfNeeded() {
if (suppressWakeWordsSync) return
if (!_isConnected.value) return
val snapshot = prefs.wakeWords.value
wakeWordsSyncJob?.cancel()
wakeWordsSyncJob =
scope.launch {
delay(650)
val jsonList = snapshot.joinToString(separator = ",") { it.toJsonString() }
val params = """{"triggers":[$jsonList]}"""
try {
session.request("voicewake.set", params)
} catch (_: Throwable) {
// ignore
}
}
}
private suspend fun refreshWakeWordsFromGateway() {
if (!_isConnected.value) return
try {
val res = session.request("voicewake.get", "{}")
val payload = json.parseToJsonElement(res).asObjectOrNull() ?: return
val array = payload["triggers"] as? JsonArray ?: return
val triggers = array.mapNotNull { it.asStringOrNull() }
applyWakeWordsFromGateway(triggers)
} catch (_: Throwable) {
// ignore
}
}
private suspend fun handleInvoke(command: String, paramsJson: String?): BridgeSession.InvokeResult {
if (
command.startsWith(ClawdisCanvasCommand.NamespacePrefix) ||
command.startsWith(ClawdisCanvasA2UICommand.NamespacePrefix) ||
command.startsWith(ClawdisCameraCommand.NamespacePrefix) ||
command.startsWith(ClawdisScreenCommand.NamespacePrefix)
) {
if (!isForeground.value) {
return BridgeSession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: canvas/camera/screen commands require foreground",
)
}
}
if (command.startsWith(ClawdisCameraCommand.NamespacePrefix) && !cameraEnabled.value) {
return BridgeSession.InvokeResult.error(
code = "CAMERA_DISABLED",
message = "CAMERA_DISABLED: enable Camera in Settings",
)
}
return when (command) {
ClawdisCanvasCommand.Present.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
BridgeSession.InvokeResult.ok(null)
}
ClawdisCanvasCommand.Hide.rawValue -> BridgeSession.InvokeResult.ok(null)
ClawdisCanvasCommand.Navigate.rawValue -> {
val url = CanvasController.parseNavigateUrl(paramsJson)
canvas.navigate(url)
BridgeSession.InvokeResult.ok(null)
}
ClawdisCanvasCommand.Eval.rawValue -> {
val js =
CanvasController.parseEvalJs(paramsJson)
?: return BridgeSession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: javaScript required",
)
val result =
try {
canvas.eval(js)
} catch (err: Throwable) {
return BridgeSession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable",
)
}
BridgeSession.InvokeResult.ok("""{"result":${result.toJsonString()}}""")
}
ClawdisCanvasCommand.Snapshot.rawValue -> {
val snapshotParams = CanvasController.parseSnapshotParams(paramsJson)
val base64 =
try {
canvas.snapshotBase64(
format = snapshotParams.format,
quality = snapshotParams.quality,
maxWidth = snapshotParams.maxWidth,
)
} catch (err: Throwable) {
return BridgeSession.InvokeResult.error(
code = "NODE_BACKGROUND_UNAVAILABLE",
message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable",
)
}
BridgeSession.InvokeResult.ok("""{"format":"${snapshotParams.format.rawValue}","base64":"$base64"}""")
}
ClawdisCanvasA2UICommand.Reset.rawValue -> {
val a2uiUrl = resolveA2uiHostUrl()
?: return BridgeSession.InvokeResult.error(
code = "A2UI_HOST_NOT_CONFIGURED",
message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host",
)
val ready = ensureA2uiReady(a2uiUrl)
if (!ready) {
return BridgeSession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",
message = "A2UI host not reachable",
)
}
val res = canvas.eval(a2uiResetJS)
BridgeSession.InvokeResult.ok(res)
}
ClawdisCanvasA2UICommand.Push.rawValue, ClawdisCanvasA2UICommand.PushJSONL.rawValue -> {
val messages =
try {
decodeA2uiMessages(command, paramsJson)
} catch (err: Throwable) {
return BridgeSession.InvokeResult.error(code = "INVALID_REQUEST", message = err.message ?: "invalid A2UI payload")
}
val a2uiUrl = resolveA2uiHostUrl()
?: return BridgeSession.InvokeResult.error(
code = "A2UI_HOST_NOT_CONFIGURED",
message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host",
)
val ready = ensureA2uiReady(a2uiUrl)
if (!ready) {
return BridgeSession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",
message = "A2UI host not reachable",
)
}
val js = a2uiApplyMessagesJS(messages)
val res = canvas.eval(js)
BridgeSession.InvokeResult.ok(res)
}
ClawdisCameraCommand.Snap.rawValue -> {
showCameraHud(message = "Taking photo…", kind = CameraHudKind.Photo)
triggerCameraFlash()
val res =
try {
camera.snap(paramsJson)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message = message, kind = CameraHudKind.Error, autoHideMs = 2200)
return BridgeSession.InvokeResult.error(code = code, message = message)
}
showCameraHud(message = "Photo captured", kind = CameraHudKind.Success, autoHideMs = 1600)
BridgeSession.InvokeResult.ok(res.payloadJson)
}
ClawdisCameraCommand.Clip.rawValue -> {
val includeAudio = paramsJson?.contains("\"includeAudio\":true") != false
if (includeAudio) externalAudioCaptureActive.value = true
try {
showCameraHud(message = "Recording…", kind = CameraHudKind.Recording)
val res =
try {
camera.clip(paramsJson)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
showCameraHud(message = message, kind = CameraHudKind.Error, autoHideMs = 2400)
return BridgeSession.InvokeResult.error(code = code, message = message)
}
showCameraHud(message = "Clip captured", kind = CameraHudKind.Success, autoHideMs = 1800)
BridgeSession.InvokeResult.ok(res.payloadJson)
} finally {
if (includeAudio) externalAudioCaptureActive.value = false
}
}
ClawdisScreenCommand.Record.rawValue -> {
val res =
try {
screenRecorder.record(paramsJson)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
return BridgeSession.InvokeResult.error(code = code, message = message)
}
BridgeSession.InvokeResult.ok(res.payloadJson)
}
else ->
BridgeSession.InvokeResult.error(
code = "INVALID_REQUEST",
message = "INVALID_REQUEST: unknown command",
)
}
}
private fun triggerCameraFlash() {
// Token is used as a pulse trigger; value doesn't matter as long as it changes.
_cameraFlashToken.value = SystemClock.elapsedRealtimeNanos()
}
private fun showCameraHud(message: String, kind: CameraHudKind, autoHideMs: Long? = null) {
val token = cameraHudSeq.incrementAndGet()
_cameraHud.value = CameraHudState(token = token, kind = kind, message = message)
if (autoHideMs != null && autoHideMs > 0) {
scope.launch {
delay(autoHideMs)
if (_cameraHud.value?.token == token) _cameraHud.value = null
}
}
}
private fun invokeErrorFromThrowable(err: Throwable): Pair<String, String> {
val raw = (err.message ?: "").trim()
if (raw.isEmpty()) return "UNAVAILABLE" to "UNAVAILABLE: camera error"
val idx = raw.indexOf(':')
if (idx <= 0) return "UNAVAILABLE" to raw
val code = raw.substring(0, idx).trim().ifEmpty { "UNAVAILABLE" }
val message = raw.substring(idx + 1).trim().ifEmpty { raw }
// Preserve full string for callers/logging, but keep the returned message human-friendly.
return code to "$code: $message"
}
private fun resolveA2uiHostUrl(): String? {
val raw = session.currentCanvasHostUrl()?.trim().orEmpty()
if (raw.isBlank()) return null
val base = raw.trimEnd('/')
return "${base}/__clawdis__/a2ui/"
}
private suspend fun ensureA2uiReady(a2uiUrl: String): Boolean {
try {
val already = canvas.eval(a2uiReadyCheckJS)
if (already == "true") return true
} catch (_: Throwable) {
// ignore
}
canvas.navigate(a2uiUrl)
repeat(50) {
try {
val ready = canvas.eval(a2uiReadyCheckJS)
if (ready == "true") return true
} catch (_: Throwable) {
// ignore
}
delay(120)
}
return false
}
private fun decodeA2uiMessages(command: String, paramsJson: String?): String {
val raw = paramsJson?.trim().orEmpty()
if (raw.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: paramsJSON required")
val obj =
json.parseToJsonElement(raw) as? JsonObject
?: throw IllegalArgumentException("INVALID_REQUEST: expected object params")
val jsonlField = (obj["jsonl"] as? JsonPrimitive)?.content?.trim().orEmpty()
val hasMessagesArray = obj["messages"] is JsonArray
if (command == ClawdisCanvasA2UICommand.PushJSONL.rawValue || (!hasMessagesArray && jsonlField.isNotBlank())) {
val jsonl = jsonlField
if (jsonl.isBlank()) throw IllegalArgumentException("INVALID_REQUEST: jsonl required")
val messages =
jsonl
.lineSequence()
.map { it.trim() }
.filter { it.isNotBlank() }
.mapIndexed { idx, line ->
val el = json.parseToJsonElement(line)
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI JSONL line ${idx + 1}: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}
.toList()
return JsonArray(messages).toString()
}
val arr = obj["messages"] as? JsonArray ?: throw IllegalArgumentException("INVALID_REQUEST: messages[] required")
val out =
arr.mapIndexed { idx, el ->
val msg =
el as? JsonObject
?: throw IllegalArgumentException("A2UI messages[${idx}]: expected a JSON object")
validateA2uiV0_8(msg, idx + 1)
msg
}
return JsonArray(out).toString()
}
private fun validateA2uiV0_8(msg: JsonObject, lineNumber: Int) {
if (msg.containsKey("createSurface")) {
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: looks like A2UI v0.9 (`createSurface`). Canvas supports v0.8 messages only.",
)
}
val allowed = setOf("beginRendering", "surfaceUpdate", "dataModelUpdate", "deleteSurface")
val matched = msg.keys.filter { allowed.contains(it) }
if (matched.size != 1) {
val found = msg.keys.sorted().joinToString(", ")
throw IllegalArgumentException(
"A2UI JSONL line $lineNumber: expected exactly one of ${allowed.sorted().joinToString(", ")}; found: $found",
)
}
}
}
private data class Quad<A, B, C, D>(val first: A, val second: B, val third: C, val fourth: D)
private const val a2uiReadyCheckJS: String =
"""
(() => {
try {
return !!globalThis.clawdisA2UI && typeof globalThis.clawdisA2UI.applyMessages === 'function';
} catch (_) {
return false;
}
})()
"""
private const val a2uiResetJS: String =
"""
(() => {
try {
if (!globalThis.clawdisA2UI) return { ok: false, error: "missing clawdisA2UI" };
return globalThis.clawdisA2UI.reset();
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
"""
private fun a2uiApplyMessagesJS(messagesJson: String): String {
return """
(() => {
try {
if (!globalThis.clawdisA2UI) return { ok: false, error: "missing clawdisA2UI" };
const messages = $messagesJson;
return globalThis.clawdisA2UI.applyMessages(messages);
} catch (e) {
return { ok: false, error: String(e?.message ?? e) };
}
})()
""".trimIndent()
}
private fun String.toJsonString(): String {
val escaped =
this.replace("\\", "\\\\")
.replace("\"", "\\\"")
.replace("\n", "\\n")
.replace("\r", "\\r")
return "\"$escaped\""
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}

View File

@@ -0,0 +1,132 @@
package com.steipete.clawdis.node
import android.content.pm.PackageManager
import android.content.Intent
import android.Manifest
import android.net.Uri
import android.provider.Settings
import androidx.appcompat.app.AlertDialog
import androidx.activity.ComponentActivity
import androidx.activity.result.ActivityResultLauncher
import androidx.activity.result.contract.ActivityResultContracts
import androidx.core.content.ContextCompat
import androidx.core.app.ActivityCompat
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlin.coroutines.resume
class PermissionRequester(private val activity: ComponentActivity) {
private val mutex = Mutex()
private var pending: CompletableDeferred<Map<String, Boolean>>? = null
private val launcher: ActivityResultLauncher<Array<String>> =
activity.registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { result ->
val p = pending
pending = null
p?.complete(result)
}
suspend fun requestIfMissing(
permissions: List<String>,
timeoutMs: Long = 20_000,
): Map<String, Boolean> =
mutex.withLock {
val missing =
permissions.filter { perm ->
ContextCompat.checkSelfPermission(activity, perm) != PackageManager.PERMISSION_GRANTED
}
if (missing.isEmpty()) {
return permissions.associateWith { true }
}
val needsRationale =
missing.any { ActivityCompat.shouldShowRequestPermissionRationale(activity, it) }
if (needsRationale) {
val proceed = showRationaleDialog(missing)
if (!proceed) {
return permissions.associateWith { perm ->
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
}
}
}
val deferred = CompletableDeferred<Map<String, Boolean>>()
pending = deferred
withContext(Dispatchers.Main) {
launcher.launch(missing.toTypedArray())
}
val result =
withContext(Dispatchers.Default) {
kotlinx.coroutines.withTimeout(timeoutMs) { deferred.await() }
}
// Merge: if something was already granted, treat it as granted even if launcher omitted it.
val merged =
permissions.associateWith { perm ->
val nowGranted =
ContextCompat.checkSelfPermission(activity, perm) == PackageManager.PERMISSION_GRANTED
result[perm] == true || nowGranted
}
val denied =
merged.filterValues { !it }.keys.filter {
!ActivityCompat.shouldShowRequestPermissionRationale(activity, it)
}
if (denied.isNotEmpty()) {
showSettingsDialog(denied)
}
return merged
}
private suspend fun showRationaleDialog(permissions: List<String>): Boolean =
withContext(Dispatchers.Main) {
suspendCancellableCoroutine { cont ->
AlertDialog.Builder(activity)
.setTitle("Permission required")
.setMessage(buildRationaleMessage(permissions))
.setPositiveButton("Continue") { _, _ -> cont.resume(true) }
.setNegativeButton("Not now") { _, _ -> cont.resume(false) }
.setOnCancelListener { cont.resume(false) }
.show()
}
}
private fun showSettingsDialog(permissions: List<String>) {
AlertDialog.Builder(activity)
.setTitle("Enable permission in Settings")
.setMessage(buildSettingsMessage(permissions))
.setPositiveButton("Open Settings") { _, _ ->
val intent =
Intent(
Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
Uri.fromParts("package", activity.packageName, null),
)
activity.startActivity(intent)
}
.setNegativeButton("Cancel", null)
.show()
}
private fun buildRationaleMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "Clawdis needs ${labels.joinToString(", ")} to capture camera media."
}
private fun buildSettingsMessage(permissions: List<String>): String {
val labels = permissions.map { permissionLabel(it) }
return "Please enable ${labels.joinToString(", ")} in Android Settings to continue."
}
private fun permissionLabel(permission: String): String =
when (permission) {
Manifest.permission.CAMERA -> "Camera"
Manifest.permission.RECORD_AUDIO -> "Microphone"
else -> permission
}
}

View File

@@ -0,0 +1,65 @@
package com.steipete.clawdis.node
import android.app.Activity
import android.content.Context
import android.content.Intent
import android.media.projection.MediaProjectionManager
import androidx.activity.ComponentActivity
import androidx.activity.result.ActivityResultLauncher
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AlertDialog
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlin.coroutines.resume
class ScreenCaptureRequester(private val activity: ComponentActivity) {
data class CaptureResult(val resultCode: Int, val data: Intent)
private val mutex = Mutex()
private var pending: CompletableDeferred<CaptureResult?>? = null
private val launcher: ActivityResultLauncher<Intent> =
activity.registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { result ->
val p = pending
pending = null
val data = result.data
if (result.resultCode == Activity.RESULT_OK && data != null) {
p?.complete(CaptureResult(result.resultCode, data))
} else {
p?.complete(null)
}
}
suspend fun requestCapture(timeoutMs: Long = 20_000): CaptureResult? =
mutex.withLock {
val proceed = showRationaleDialog()
if (!proceed) return null
val mgr = activity.getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager
val intent = mgr.createScreenCaptureIntent()
val deferred = CompletableDeferred<CaptureResult?>()
pending = deferred
withContext(Dispatchers.Main) { launcher.launch(intent) }
withContext(Dispatchers.Default) { withTimeout(timeoutMs) { deferred.await() } }
}
private suspend fun showRationaleDialog(): Boolean =
withContext(Dispatchers.Main) {
suspendCancellableCoroutine { cont ->
AlertDialog.Builder(activity)
.setTitle("Screen recording required")
.setMessage("Clawdis needs to record the screen for this command.")
.setPositiveButton("Continue") { _, _ -> cont.resume(true) }
.setNegativeButton("Not now") { _, _ -> cont.resume(false) }
.setOnCancelListener { cont.resume(false) }
.show()
}
}
}

View File

@@ -0,0 +1,192 @@
@file:Suppress("DEPRECATION")
package com.steipete.clawdis.node
import android.content.Context
import androidx.core.content.edit
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonPrimitive
import java.util.UUID
class SecurePrefs(context: Context) {
companion object {
val defaultWakeWords: List<String> = listOf("clawd", "claude")
private const val displayNameKey = "node.displayName"
private const val voiceWakeModeKey = "voiceWake.mode"
}
private val json = Json { ignoreUnknownKeys = true }
private val masterKey =
MasterKey.Builder(context)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
private val prefs =
EncryptedSharedPreferences.create(
context,
"clawdis.node.secure",
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM,
)
private val _instanceId = MutableStateFlow(loadOrCreateInstanceId())
val instanceId: StateFlow<String> = _instanceId
private val _displayName =
MutableStateFlow(loadOrMigrateDisplayName(context = context))
val displayName: StateFlow<String> = _displayName
private val _cameraEnabled = MutableStateFlow(prefs.getBoolean("camera.enabled", true))
val cameraEnabled: StateFlow<Boolean> = _cameraEnabled
private val _preventSleep = MutableStateFlow(prefs.getBoolean("screen.preventSleep", true))
val preventSleep: StateFlow<Boolean> = _preventSleep
private val _manualEnabled = MutableStateFlow(prefs.getBoolean("bridge.manual.enabled", false))
val manualEnabled: StateFlow<Boolean> = _manualEnabled
private val _manualHost = MutableStateFlow(prefs.getString("bridge.manual.host", "")!!)
val manualHost: StateFlow<String> = _manualHost
private val _manualPort = MutableStateFlow(prefs.getInt("bridge.manual.port", 18790))
val manualPort: StateFlow<Int> = _manualPort
private val _lastDiscoveredStableId =
MutableStateFlow(prefs.getString("bridge.lastDiscoveredStableId", "")!!)
val lastDiscoveredStableId: StateFlow<String> = _lastDiscoveredStableId
private val _canvasDebugStatusEnabled =
MutableStateFlow(prefs.getBoolean("canvas.debugStatusEnabled", false))
val canvasDebugStatusEnabled: StateFlow<Boolean> = _canvasDebugStatusEnabled
private val _wakeWords = MutableStateFlow(loadWakeWords())
val wakeWords: StateFlow<List<String>> = _wakeWords
private val _voiceWakeMode = MutableStateFlow(loadVoiceWakeMode())
val voiceWakeMode: StateFlow<VoiceWakeMode> = _voiceWakeMode
fun setLastDiscoveredStableId(value: String) {
val trimmed = value.trim()
prefs.edit { putString("bridge.lastDiscoveredStableId", trimmed) }
_lastDiscoveredStableId.value = trimmed
}
fun setDisplayName(value: String) {
val trimmed = value.trim()
prefs.edit { putString(displayNameKey, trimmed) }
_displayName.value = trimmed
}
fun setCameraEnabled(value: Boolean) {
prefs.edit { putBoolean("camera.enabled", value) }
_cameraEnabled.value = value
}
fun setPreventSleep(value: Boolean) {
prefs.edit { putBoolean("screen.preventSleep", value) }
_preventSleep.value = value
}
fun setManualEnabled(value: Boolean) {
prefs.edit { putBoolean("bridge.manual.enabled", value) }
_manualEnabled.value = value
}
fun setManualHost(value: String) {
val trimmed = value.trim()
prefs.edit { putString("bridge.manual.host", trimmed) }
_manualHost.value = trimmed
}
fun setManualPort(value: Int) {
prefs.edit { putInt("bridge.manual.port", value) }
_manualPort.value = value
}
fun setCanvasDebugStatusEnabled(value: Boolean) {
prefs.edit { putBoolean("canvas.debugStatusEnabled", value) }
_canvasDebugStatusEnabled.value = value
}
fun loadBridgeToken(): String? {
val key = "bridge.token.${_instanceId.value}"
return prefs.getString(key, null)
}
fun saveBridgeToken(token: String) {
val key = "bridge.token.${_instanceId.value}"
prefs.edit { putString(key, token.trim()) }
}
private fun loadOrCreateInstanceId(): String {
val existing = prefs.getString("node.instanceId", null)?.trim()
if (!existing.isNullOrBlank()) return existing
val fresh = UUID.randomUUID().toString()
prefs.edit { putString("node.instanceId", fresh) }
return fresh
}
private fun loadOrMigrateDisplayName(context: Context): String {
val existing = prefs.getString(displayNameKey, null)?.trim().orEmpty()
if (existing.isNotEmpty() && existing != "Android Node") return existing
val candidate = DeviceNames.bestDefaultNodeName(context).trim()
val resolved = candidate.ifEmpty { "Android Node" }
prefs.edit { putString(displayNameKey, resolved) }
return resolved
}
fun setWakeWords(words: List<String>) {
val sanitized = WakeWords.sanitize(words, defaultWakeWords)
val encoded =
JsonArray(sanitized.map { JsonPrimitive(it) }).toString()
prefs.edit { putString("voiceWake.triggerWords", encoded) }
_wakeWords.value = sanitized
}
fun setVoiceWakeMode(mode: VoiceWakeMode) {
prefs.edit { putString(voiceWakeModeKey, mode.rawValue) }
_voiceWakeMode.value = mode
}
private fun loadVoiceWakeMode(): VoiceWakeMode {
val raw = prefs.getString(voiceWakeModeKey, null)
val resolved = VoiceWakeMode.fromRawValue(raw)
// Default ON (foreground) when unset.
if (raw.isNullOrBlank()) {
prefs.edit { putString(voiceWakeModeKey, resolved.rawValue) }
}
return resolved
}
private fun loadWakeWords(): List<String> {
val raw = prefs.getString("voiceWake.triggerWords", null)?.trim()
if (raw.isNullOrEmpty()) return defaultWakeWords
return try {
val element = json.parseToJsonElement(raw)
val array = element as? JsonArray ?: return defaultWakeWords
val decoded =
array.mapNotNull { item ->
when (item) {
is JsonNull -> null
is JsonPrimitive -> item.content.trim().takeIf { it.isNotEmpty() }
else -> null
}
}
WakeWords.sanitize(decoded, defaultWakeWords)
} catch (_: Throwable) {
defaultWakeWords
}
}
}

View File

@@ -0,0 +1,15 @@
package com.steipete.clawdis.node
enum class VoiceWakeMode(val rawValue: String) {
Off("off"),
Foreground("foreground"),
Always("always"),
;
companion object {
fun fromRawValue(raw: String?): VoiceWakeMode {
return entries.firstOrNull { it.rawValue == raw?.trim()?.lowercase() } ?: Foreground
}
}
}

View File

@@ -0,0 +1,17 @@
package com.steipete.clawdis.node
object WakeWords {
const val maxWords: Int = 32
const val maxWordLength: Int = 64
fun parseCommaSeparated(input: String): List<String> {
return input.split(",").map { it.trim() }.filter { it.isNotEmpty() }
}
fun sanitize(words: List<String>, defaults: List<String>): List<String> {
val cleaned =
words.map { it.trim() }.filter { it.isNotEmpty() }.take(maxWords).map { it.take(maxWordLength) }
return cleaned.ifEmpty { defaults }
}
}

View File

@@ -0,0 +1,35 @@
package com.steipete.clawdis.node.bridge
object BonjourEscapes {
fun decode(input: String): String {
if (input.isEmpty()) return input
val bytes = mutableListOf<Byte>()
var i = 0
while (i < input.length) {
if (input[i] == '\\' && i + 3 < input.length) {
val d0 = input[i + 1]
val d1 = input[i + 2]
val d2 = input[i + 3]
if (d0.isDigit() && d1.isDigit() && d2.isDigit()) {
val value =
((d0.code - '0'.code) * 100) + ((d1.code - '0'.code) * 10) + (d2.code - '0'.code)
if (value in 0..255) {
bytes.add(value.toByte())
i += 4
continue
}
}
}
val codePoint = Character.codePointAt(input, i)
val charBytes = String(Character.toChars(codePoint)).toByteArray(Charsets.UTF_8)
for (b in charBytes) {
bytes.add(b)
}
i += Character.charCount(codePoint)
}
return String(bytes.toByteArray(), Charsets.UTF_8)
}
}

View File

@@ -0,0 +1,465 @@
package com.steipete.clawdis.node.bridge
import android.content.Context
import android.net.ConnectivityManager
import android.net.DnsResolver
import android.net.NetworkCapabilities
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.os.CancellationSignal
import android.util.Log
import java.io.IOException
import java.net.InetSocketAddress
import java.nio.ByteBuffer
import java.nio.charset.CodingErrorAction
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.Executor
import java.util.concurrent.Executors
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.suspendCancellableCoroutine
import org.xbill.DNS.AAAARecord
import org.xbill.DNS.ARecord
import org.xbill.DNS.DClass
import org.xbill.DNS.ExtendedResolver
import org.xbill.DNS.Message
import org.xbill.DNS.Name
import org.xbill.DNS.PTRRecord
import org.xbill.DNS.Record
import org.xbill.DNS.Rcode
import org.xbill.DNS.Resolver
import org.xbill.DNS.SRVRecord
import org.xbill.DNS.Section
import org.xbill.DNS.SimpleResolver
import org.xbill.DNS.TextParseException
import org.xbill.DNS.TXTRecord
import org.xbill.DNS.Type
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
@Suppress("DEPRECATION")
class BridgeDiscovery(
context: Context,
private val scope: CoroutineScope,
) {
private val nsd = context.getSystemService(NsdManager::class.java)
private val connectivity = context.getSystemService(ConnectivityManager::class.java)
private val dns = DnsResolver.getInstance()
private val serviceType = "_clawdis-bridge._tcp."
private val wideAreaDomain = "clawdis.internal."
private val logTag = "Clawdis/BridgeDiscovery"
private val localById = ConcurrentHashMap<String, BridgeEndpoint>()
private val unicastById = ConcurrentHashMap<String, BridgeEndpoint>()
private val _bridges = MutableStateFlow<List<BridgeEndpoint>>(emptyList())
val bridges: StateFlow<List<BridgeEndpoint>> = _bridges.asStateFlow()
private val _statusText = MutableStateFlow("Searching…")
val statusText: StateFlow<String> = _statusText.asStateFlow()
private var unicastJob: Job? = null
private val dnsExecutor: Executor = Executors.newCachedThreadPool()
@Volatile private var lastWideAreaRcode: Int? = null
@Volatile private var lastWideAreaCount: Int = 0
private val discoveryListener =
object : NsdManager.DiscoveryListener {
override fun onStartDiscoveryFailed(serviceType: String, errorCode: Int) {}
override fun onStopDiscoveryFailed(serviceType: String, errorCode: Int) {}
override fun onDiscoveryStarted(serviceType: String) {}
override fun onDiscoveryStopped(serviceType: String) {}
override fun onServiceFound(serviceInfo: NsdServiceInfo) {
if (serviceInfo.serviceType != this@BridgeDiscovery.serviceType) return
resolve(serviceInfo)
}
override fun onServiceLost(serviceInfo: NsdServiceInfo) {
val serviceName = BonjourEscapes.decode(serviceInfo.serviceName)
val id = stableId(serviceName, "local.")
localById.remove(id)
publish()
}
}
init {
startLocalDiscovery()
startUnicastDiscovery(wideAreaDomain)
}
private fun startLocalDiscovery() {
try {
nsd.discoverServices(serviceType, NsdManager.PROTOCOL_DNS_SD, discoveryListener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun stopLocalDiscovery() {
try {
nsd.stopServiceDiscovery(discoveryListener)
} catch (_: Throwable) {
// ignore (best-effort)
}
}
private fun startUnicastDiscovery(domain: String) {
unicastJob =
scope.launch(Dispatchers.IO) {
while (true) {
try {
refreshUnicast(domain)
} catch (_: Throwable) {
// ignore (best-effort)
}
delay(5000)
}
}
}
private fun resolve(serviceInfo: NsdServiceInfo) {
nsd.resolveService(
serviceInfo,
object : NsdManager.ResolveListener {
override fun onResolveFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {}
override fun onServiceResolved(resolved: NsdServiceInfo) {
val host = resolved.host?.hostAddress ?: return
val port = resolved.port
if (port <= 0) return
val rawServiceName = resolved.serviceName
val serviceName = BonjourEscapes.decode(rawServiceName)
val displayName = BonjourEscapes.decode(txt(resolved, "displayName") ?: serviceName)
val id = stableId(serviceName, "local.")
localById[id] = BridgeEndpoint(stableId = id, name = displayName, host = host, port = port)
publish()
}
},
)
}
private fun publish() {
_bridges.value =
(localById.values + unicastById.values).sortedBy { it.name.lowercase() }
_statusText.value = buildStatusText()
}
private fun buildStatusText(): String {
val localCount = localById.size
val wideRcode = lastWideAreaRcode
val wideCount = lastWideAreaCount
val wide =
when (wideRcode) {
null -> "Wide: ?"
Rcode.NOERROR -> "Wide: $wideCount"
Rcode.NXDOMAIN -> "Wide: NXDOMAIN"
else -> "Wide: ${Rcode.string(wideRcode)}"
}
return when {
localCount == 0 && wideRcode == null -> "Searching for bridges…"
localCount == 0 -> "$wide"
else -> "Local: $localCount$wide"
}
}
private fun stableId(serviceName: String, domain: String): String {
return "${serviceType}|${domain}|${normalizeName(serviceName)}"
}
private fun normalizeName(raw: String): String {
return raw.trim().split(Regex("\\s+")).joinToString(" ")
}
private fun txt(info: NsdServiceInfo, key: String): String? {
val bytes = info.attributes[key] ?: return null
return try {
String(bytes, Charsets.UTF_8).trim().ifEmpty { null }
} catch (_: Throwable) {
null
}
}
private suspend fun refreshUnicast(domain: String) {
val ptrName = "${serviceType}${domain}"
val ptrMsg = lookupUnicastMessage(ptrName, Type.PTR) ?: return
val ptrRecords = records(ptrMsg, Section.ANSWER).mapNotNull { it as? PTRRecord }
val next = LinkedHashMap<String, BridgeEndpoint>()
for (ptr in ptrRecords) {
val instanceFqdn = ptr.target.toString()
val srv =
recordByName(ptrMsg, instanceFqdn, Type.SRV) as? SRVRecord
?: run {
val msg = lookupUnicastMessage(instanceFqdn, Type.SRV) ?: return@run null
recordByName(msg, instanceFqdn, Type.SRV) as? SRVRecord
}
?: continue
val port = srv.port
if (port <= 0) continue
val targetFqdn = srv.target.toString()
val host =
resolveHostFromMessage(ptrMsg, targetFqdn)
?: resolveHostFromMessage(lookupUnicastMessage(instanceFqdn, Type.SRV), targetFqdn)
?: resolveHostUnicast(targetFqdn)
?: continue
val txtFromPtr =
recordsByName(ptrMsg, Section.ADDITIONAL)[keyName(instanceFqdn)]
.orEmpty()
.mapNotNull { it as? TXTRecord }
val txt =
if (txtFromPtr.isNotEmpty()) {
txtFromPtr
} else {
val msg = lookupUnicastMessage(instanceFqdn, Type.TXT)
records(msg, Section.ANSWER).mapNotNull { it as? TXTRecord }
}
val instanceName = BonjourEscapes.decode(decodeInstanceName(instanceFqdn, domain))
val displayName = BonjourEscapes.decode(txtValue(txt, "displayName") ?: instanceName)
val id = stableId(instanceName, domain)
next[id] = BridgeEndpoint(stableId = id, name = displayName, host = host, port = port)
}
unicastById.clear()
unicastById.putAll(next)
lastWideAreaRcode = ptrMsg.header.rcode
lastWideAreaCount = next.size
publish()
if (next.isEmpty()) {
Log.d(
logTag,
"wide-area discovery: 0 results for $ptrName (rcode=${Rcode.string(ptrMsg.header.rcode)})",
)
}
}
private fun decodeInstanceName(instanceFqdn: String, domain: String): String {
val suffix = "${serviceType}${domain}"
val withoutSuffix =
if (instanceFqdn.endsWith(suffix)) {
instanceFqdn.removeSuffix(suffix)
} else {
instanceFqdn.substringBefore(serviceType)
}
return normalizeName(stripTrailingDot(withoutSuffix))
}
private fun stripTrailingDot(raw: String): String {
return raw.removeSuffix(".")
}
private suspend fun lookupUnicastMessage(name: String, type: Int): Message? {
val query =
try {
Message.newQuery(
org.xbill.DNS.Record.newRecord(
Name.fromString(name),
type,
DClass.IN,
),
)
} catch (_: TextParseException) {
return null
}
val system = queryViaSystemDns(query)
if (records(system, Section.ANSWER).any { it.type == type }) return system
val direct = createDirectResolver() ?: return system
return try {
val msg = direct.send(query)
if (records(msg, Section.ANSWER).any { it.type == type }) msg else system
} catch (_: Throwable) {
system
}
}
private suspend fun queryViaSystemDns(query: Message): Message? {
val network = preferredDnsNetwork()
val bytes =
try {
rawQuery(network, query.toWire())
} catch (_: Throwable) {
return null
}
return try {
Message(bytes)
} catch (_: IOException) {
null
}
}
private fun records(msg: Message?, section: Int): List<Record> {
return msg?.getSectionArray(section)?.toList() ?: emptyList()
}
private fun keyName(raw: String): String {
return raw.trim().lowercase()
}
private fun recordsByName(msg: Message, section: Int): Map<String, List<Record>> {
val next = LinkedHashMap<String, MutableList<Record>>()
for (r in records(msg, section)) {
val name = r.name?.toString() ?: continue
next.getOrPut(keyName(name)) { mutableListOf() }.add(r)
}
return next
}
private fun recordByName(msg: Message, fqdn: String, type: Int): Record? {
val key = keyName(fqdn)
val byNameAnswer = recordsByName(msg, Section.ANSWER)
val fromAnswer = byNameAnswer[key].orEmpty().firstOrNull { it.type == type }
if (fromAnswer != null) return fromAnswer
val byNameAdditional = recordsByName(msg, Section.ADDITIONAL)
return byNameAdditional[key].orEmpty().firstOrNull { it.type == type }
}
private fun resolveHostFromMessage(msg: Message?, hostname: String): String? {
val m = msg ?: return null
val key = keyName(hostname)
val additional = recordsByName(m, Section.ADDITIONAL)[key].orEmpty()
val a = additional.mapNotNull { it as? ARecord }.mapNotNull { it.address?.hostAddress }
val aaaa = additional.mapNotNull { it as? AAAARecord }.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
private fun preferredDnsNetwork(): android.net.Network? {
val cm = connectivity ?: return null
// Prefer VPN (Tailscale) when present; otherwise use the active network.
cm.allNetworks.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let { return it }
return cm.activeNetwork
}
private fun createDirectResolver(): Resolver? {
val cm = connectivity ?: return null
val candidateNetworks =
buildList {
cm.allNetworks
.firstOrNull { n ->
val caps = cm.getNetworkCapabilities(n) ?: return@firstOrNull false
caps.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
}?.let(::add)
cm.activeNetwork?.let(::add)
}.distinct()
val servers =
candidateNetworks
.asSequence()
.flatMap { n ->
cm.getLinkProperties(n)?.dnsServers?.asSequence() ?: emptySequence()
}
.distinctBy { it.hostAddress ?: it.toString() }
.toList()
if (servers.isEmpty()) return null
return try {
val resolvers =
servers.mapNotNull { addr ->
try {
SimpleResolver().apply {
setAddress(InetSocketAddress(addr, 53))
setTimeout(3)
}
} catch (_: Throwable) {
null
}
}
if (resolvers.isEmpty()) return null
ExtendedResolver(resolvers.toTypedArray()).apply { setTimeout(3) }
} catch (_: Throwable) {
null
}
}
private suspend fun rawQuery(network: android.net.Network?, wireQuery: ByteArray): ByteArray =
suspendCancellableCoroutine { cont ->
val signal = CancellationSignal()
cont.invokeOnCancellation { signal.cancel() }
dns.rawQuery(
network,
wireQuery,
DnsResolver.FLAG_EMPTY,
dnsExecutor,
signal,
object : DnsResolver.Callback<ByteArray> {
override fun onAnswer(answer: ByteArray, rcode: Int) {
cont.resume(answer)
}
override fun onError(error: DnsResolver.DnsException) {
cont.resumeWithException(error)
}
},
)
}
private fun txtValue(records: List<TXTRecord>, key: String): String? {
val prefix = "$key="
for (r in records) {
val strings: List<String> =
try {
r.strings.mapNotNull { it as? String }
} catch (_: Throwable) {
emptyList()
}
for (s in strings) {
val trimmed = decodeDnsTxtString(s).trim()
if (trimmed.startsWith(prefix)) {
return trimmed.removePrefix(prefix).trim().ifEmpty { null }
}
}
}
return null
}
private fun decodeDnsTxtString(raw: String): String {
// dnsjava treats TXT as opaque bytes and decodes as ISO-8859-1 to preserve bytes.
// Our TXT payload is UTF-8 (written by the gateway), so re-decode when possible.
val bytes = raw.toByteArray(Charsets.ISO_8859_1)
val decoder =
Charsets.UTF_8
.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
return try {
decoder.decode(ByteBuffer.wrap(bytes)).toString()
} catch (_: Throwable) {
raw
}
}
private suspend fun resolveHostUnicast(hostname: String): String? {
val a =
records(lookupUnicastMessage(hostname, Type.A), Section.ANSWER)
.mapNotNull { it as? ARecord }
.mapNotNull { it.address?.hostAddress }
val aaaa =
records(lookupUnicastMessage(hostname, Type.AAAA), Section.ANSWER)
.mapNotNull { it as? AAAARecord }
.mapNotNull { it.address?.hostAddress }
return a.firstOrNull() ?: aaaa.firstOrNull()
}
}

View File

@@ -0,0 +1,19 @@
package com.steipete.clawdis.node.bridge
data class BridgeEndpoint(
val stableId: String,
val name: String,
val host: String,
val port: Int,
) {
companion object {
fun manual(host: String, port: Int): BridgeEndpoint =
BridgeEndpoint(
stableId = "manual|$host|$port",
name = "$host:$port",
host = host,
port = port,
)
}
}

View File

@@ -0,0 +1,131 @@
package com.steipete.clawdis.node.bridge
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.buildJsonObject
import java.io.BufferedReader
import java.io.BufferedWriter
import java.io.InputStreamReader
import java.io.OutputStreamWriter
import java.net.InetSocketAddress
import java.net.Socket
class BridgePairingClient {
private val json = Json { ignoreUnknownKeys = true }
data class Hello(
val nodeId: String,
val displayName: String?,
val token: String?,
val platform: String?,
val version: String?,
val deviceFamily: String?,
val modelIdentifier: String?,
val caps: List<String>?,
val commands: List<String>?,
)
data class PairResult(val ok: Boolean, val token: String?, val error: String? = null)
suspend fun pairAndHello(endpoint: BridgeEndpoint, hello: Hello): PairResult =
withContext(Dispatchers.IO) {
val socket = Socket()
socket.tcpNoDelay = true
socket.connect(InetSocketAddress(endpoint.host, endpoint.port), 8_000)
socket.soTimeout = 60_000
val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8))
val writer = BufferedWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8))
fun send(line: String) {
writer.write(line)
writer.write("\n")
writer.flush()
}
fun sendJson(obj: JsonObject) = send(obj.toString())
try {
sendJson(
buildJsonObject {
put("type", JsonPrimitive("hello"))
put("nodeId", JsonPrimitive(hello.nodeId))
hello.displayName?.let { put("displayName", JsonPrimitive(it)) }
hello.token?.let { put("token", JsonPrimitive(it)) }
hello.platform?.let { put("platform", JsonPrimitive(it)) }
hello.version?.let { put("version", JsonPrimitive(it)) }
hello.deviceFamily?.let { put("deviceFamily", JsonPrimitive(it)) }
hello.modelIdentifier?.let { put("modelIdentifier", JsonPrimitive(it)) }
hello.caps?.let { put("caps", JsonArray(it.map(::JsonPrimitive))) }
hello.commands?.let { put("commands", JsonArray(it.map(::JsonPrimitive))) }
},
)
val firstObj = json.parseToJsonElement(reader.readLine()).asObjectOrNull()
?: return@withContext PairResult(ok = false, token = null, error = "unexpected bridge response")
when (firstObj["type"].asStringOrNull()) {
"hello-ok" -> PairResult(ok = true, token = hello.token)
"error" -> {
val code = firstObj["code"].asStringOrNull() ?: "UNAVAILABLE"
val message = firstObj["message"].asStringOrNull() ?: "pairing required"
if (code != "NOT_PAIRED" && code != "UNAUTHORIZED") {
return@withContext PairResult(ok = false, token = null, error = "$code: $message")
}
sendJson(
buildJsonObject {
put("type", JsonPrimitive("pair-request"))
put("nodeId", JsonPrimitive(hello.nodeId))
hello.displayName?.let { put("displayName", JsonPrimitive(it)) }
hello.platform?.let { put("platform", JsonPrimitive(it)) }
hello.version?.let { put("version", JsonPrimitive(it)) }
hello.deviceFamily?.let { put("deviceFamily", JsonPrimitive(it)) }
hello.modelIdentifier?.let { put("modelIdentifier", JsonPrimitive(it)) }
hello.caps?.let { put("caps", JsonArray(it.map(::JsonPrimitive))) }
hello.commands?.let { put("commands", JsonArray(it.map(::JsonPrimitive))) }
},
)
while (true) {
val nextLine = reader.readLine() ?: break
val next = json.parseToJsonElement(nextLine).asObjectOrNull() ?: continue
when (next["type"].asStringOrNull()) {
"pair-ok" -> {
val token = next["token"].asStringOrNull()
return@withContext PairResult(ok = !token.isNullOrBlank(), token = token)
}
"error" -> {
val c = next["code"].asStringOrNull() ?: "UNAVAILABLE"
val m = next["message"].asStringOrNull() ?: "pairing failed"
return@withContext PairResult(ok = false, token = null, error = "$c: $m")
}
}
}
PairResult(ok = false, token = null, error = "pairing failed")
}
else -> PairResult(ok = false, token = null, error = "unexpected bridge response")
}
} finally {
try {
socket.close()
} catch (_: Throwable) {
// ignore
}
}
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}

View File

@@ -0,0 +1,317 @@
package com.steipete.clawdis.node.bridge
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.cancelAndJoin
import kotlinx.coroutines.delay
import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import java.io.BufferedReader
import java.io.BufferedWriter
import java.io.InputStreamReader
import java.io.OutputStreamWriter
import java.net.InetSocketAddress
import java.net.Socket
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
class BridgeSession(
private val scope: CoroutineScope,
private val onConnected: (serverName: String, remoteAddress: String?) -> Unit,
private val onDisconnected: (message: String) -> Unit,
private val onEvent: (event: String, payloadJson: String?) -> Unit,
private val onInvoke: suspend (InvokeRequest) -> InvokeResult,
) {
data class Hello(
val nodeId: String,
val displayName: String?,
val token: String?,
val platform: String?,
val version: String?,
val deviceFamily: String?,
val modelIdentifier: String?,
val caps: List<String>?,
val commands: List<String>?,
)
data class InvokeRequest(val id: String, val command: String, val paramsJson: String?)
data class InvokeResult(val ok: Boolean, val payloadJson: String?, val error: ErrorShape?) {
companion object {
fun ok(payloadJson: String?) = InvokeResult(ok = true, payloadJson = payloadJson, error = null)
fun error(code: String, message: String) =
InvokeResult(ok = false, payloadJson = null, error = ErrorShape(code = code, message = message))
}
}
data class ErrorShape(val code: String, val message: String)
private val json = Json { ignoreUnknownKeys = true }
private val writeLock = Mutex()
private val pending = ConcurrentHashMap<String, CompletableDeferred<RpcResponse>>()
@Volatile private var canvasHostUrl: String? = null
private var desired: Pair<BridgeEndpoint, Hello>? = null
private var job: Job? = null
fun connect(endpoint: BridgeEndpoint, hello: Hello) {
desired = endpoint to hello
if (job == null) {
job = scope.launch(Dispatchers.IO) { runLoop() }
}
}
fun disconnect() {
desired = null
scope.launch(Dispatchers.IO) {
job?.cancelAndJoin()
job = null
canvasHostUrl = null
onDisconnected("Offline")
}
}
fun currentCanvasHostUrl(): String? = canvasHostUrl
suspend fun sendEvent(event: String, payloadJson: String?) {
val conn = currentConnection ?: return
conn.sendJson(
buildJsonObject {
put("type", JsonPrimitive("event"))
put("event", JsonPrimitive(event))
if (payloadJson != null) put("payloadJSON", JsonPrimitive(payloadJson)) else put("payloadJSON", JsonNull)
},
)
}
suspend fun request(method: String, paramsJson: String?): String {
val conn = currentConnection ?: throw IllegalStateException("not connected")
val id = UUID.randomUUID().toString()
val deferred = CompletableDeferred<RpcResponse>()
pending[id] = deferred
conn.sendJson(
buildJsonObject {
put("type", JsonPrimitive("req"))
put("id", JsonPrimitive(id))
put("method", JsonPrimitive(method))
if (paramsJson != null) put("paramsJSON", JsonPrimitive(paramsJson)) else put("paramsJSON", JsonNull)
},
)
val res = deferred.await()
if (res.ok) return res.payloadJson ?: ""
val err = res.error
throw IllegalStateException("${err?.code ?: "UNAVAILABLE"}: ${err?.message ?: "request failed"}")
}
private data class RpcResponse(val id: String, val ok: Boolean, val payloadJson: String?, val error: ErrorShape?)
private class Connection(private val socket: Socket, private val reader: BufferedReader, private val writer: BufferedWriter, private val writeLock: Mutex) {
val remoteAddress: String? =
socket.inetAddress?.hostAddress?.takeIf { it.isNotBlank() }?.let { "${it}:${socket.port}" }
suspend fun sendJson(obj: JsonObject) {
writeLock.withLock {
writer.write(obj.toString())
writer.write("\n")
writer.flush()
}
}
fun closeQuietly() {
try {
socket.close()
} catch (_: Throwable) {
// ignore
}
}
}
@Volatile private var currentConnection: Connection? = null
private suspend fun runLoop() {
var attempt = 0
while (scope.isActive) {
val target = desired
if (target == null) {
currentConnection?.closeQuietly()
currentConnection = null
delay(250)
continue
}
val (endpoint, hello) = target
try {
onDisconnected(if (attempt == 0) "Connecting…" else "Reconnecting…")
connectOnce(endpoint, hello)
attempt = 0
} catch (err: Throwable) {
attempt += 1
onDisconnected("Bridge error: ${err.message ?: err::class.java.simpleName}")
val sleepMs = minOf(8_000L, (350.0 * Math.pow(1.7, attempt.toDouble())).toLong())
delay(sleepMs)
}
}
}
private fun invokeErrorFromThrowable(err: Throwable): InvokeResult {
val msg = err.message?.trim().takeIf { !it.isNullOrEmpty() } ?: err::class.java.simpleName
val parts = msg.split(":", limit = 2)
if (parts.size == 2) {
val code = parts[0].trim()
val rest = parts[1].trim()
if (code.isNotEmpty() && code.all { it.isUpperCase() || it == '_' }) {
return InvokeResult.error(code = code, message = rest.ifEmpty { msg })
}
}
return InvokeResult.error(code = "UNAVAILABLE", message = msg)
}
private suspend fun connectOnce(endpoint: BridgeEndpoint, hello: Hello) =
withContext(Dispatchers.IO) {
val socket = Socket()
socket.tcpNoDelay = true
socket.connect(InetSocketAddress(endpoint.host, endpoint.port), 8_000)
socket.soTimeout = 0
val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8))
val writer = BufferedWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8))
val conn = Connection(socket, reader, writer, writeLock)
currentConnection = conn
try {
conn.sendJson(
buildJsonObject {
put("type", JsonPrimitive("hello"))
put("nodeId", JsonPrimitive(hello.nodeId))
hello.displayName?.let { put("displayName", JsonPrimitive(it)) }
hello.token?.let { put("token", JsonPrimitive(it)) }
hello.platform?.let { put("platform", JsonPrimitive(it)) }
hello.version?.let { put("version", JsonPrimitive(it)) }
hello.deviceFamily?.let { put("deviceFamily", JsonPrimitive(it)) }
hello.modelIdentifier?.let { put("modelIdentifier", JsonPrimitive(it)) }
hello.caps?.let { put("caps", JsonArray(it.map(::JsonPrimitive))) }
hello.commands?.let { put("commands", JsonArray(it.map(::JsonPrimitive))) }
},
)
val firstLine = reader.readLine() ?: throw IllegalStateException("bridge closed connection")
val first = json.parseToJsonElement(firstLine).asObjectOrNull()
?: throw IllegalStateException("unexpected bridge response")
when (first["type"].asStringOrNull()) {
"hello-ok" -> {
val name = first["serverName"].asStringOrNull() ?: "Bridge"
canvasHostUrl = first["canvasHostUrl"].asStringOrNull()?.trim()?.ifEmpty { null }
onConnected(name, conn.remoteAddress)
}
"error" -> {
val code = first["code"].asStringOrNull() ?: "UNAVAILABLE"
val msg = first["message"].asStringOrNull() ?: "connect failed"
throw IllegalStateException("$code: $msg")
}
else -> throw IllegalStateException("unexpected bridge response")
}
while (scope.isActive) {
val line = reader.readLine() ?: break
val frame = json.parseToJsonElement(line).asObjectOrNull() ?: continue
when (frame["type"].asStringOrNull()) {
"event" -> {
val event = frame["event"].asStringOrNull() ?: return@withContext
val payload = frame["payloadJSON"].asStringOrNull()
onEvent(event, payload)
}
"ping" -> {
val id = frame["id"].asStringOrNull() ?: ""
conn.sendJson(buildJsonObject { put("type", JsonPrimitive("pong")); put("id", JsonPrimitive(id)) })
}
"res" -> {
val id = frame["id"].asStringOrNull() ?: continue
val ok = frame["ok"].asBooleanOrNull() ?: false
val payloadJson = frame["payloadJSON"].asStringOrNull()
val error =
frame["error"]?.let {
val obj = it.asObjectOrNull() ?: return@let null
val code = obj["code"].asStringOrNull() ?: "UNAVAILABLE"
val msg = obj["message"].asStringOrNull() ?: "request failed"
ErrorShape(code, msg)
}
pending.remove(id)?.complete(RpcResponse(id, ok, payloadJson, error))
}
"invoke" -> {
val id = frame["id"].asStringOrNull() ?: continue
val command = frame["command"].asStringOrNull() ?: ""
val params = frame["paramsJSON"].asStringOrNull()
val result =
try {
onInvoke(InvokeRequest(id, command, params))
} catch (err: Throwable) {
invokeErrorFromThrowable(err)
}
conn.sendJson(
buildJsonObject {
put("type", JsonPrimitive("invoke-res"))
put("id", JsonPrimitive(id))
put("ok", JsonPrimitive(result.ok))
if (result.payloadJson != null) put("payloadJSON", JsonPrimitive(result.payloadJson))
if (result.error != null) {
put(
"error",
buildJsonObject {
put("code", JsonPrimitive(result.error.code))
put("message", JsonPrimitive(result.error.message))
},
)
}
},
)
}
"invoke-res" -> {
// gateway->node only (ignore)
}
}
}
} finally {
currentConnection = null
for ((_, waiter) in pending) {
waiter.cancel()
}
pending.clear()
conn.closeQuietly()
}
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}
private fun JsonElement?.asBooleanOrNull(): Boolean? =
when (this) {
is JsonPrimitive -> {
val c = content.trim()
when {
c.equals("true", ignoreCase = true) -> true
c.equals("false", ignoreCase = true) -> false
else -> null
}
}
else -> null
}

View File

@@ -0,0 +1,509 @@
package com.steipete.clawdis.node.chat
import com.steipete.clawdis.node.bridge.BridgeSession
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
class ChatController(
private val scope: CoroutineScope,
private val session: BridgeSession,
private val json: Json,
) {
private val _sessionKey = MutableStateFlow("main")
val sessionKey: StateFlow<String> = _sessionKey.asStateFlow()
private val _sessionId = MutableStateFlow<String?>(null)
val sessionId: StateFlow<String?> = _sessionId.asStateFlow()
private val _messages = MutableStateFlow<List<ChatMessage>>(emptyList())
val messages: StateFlow<List<ChatMessage>> = _messages.asStateFlow()
private val _errorText = MutableStateFlow<String?>(null)
val errorText: StateFlow<String?> = _errorText.asStateFlow()
private val _healthOk = MutableStateFlow(false)
val healthOk: StateFlow<Boolean> = _healthOk.asStateFlow()
private val _thinkingLevel = MutableStateFlow("off")
val thinkingLevel: StateFlow<String> = _thinkingLevel.asStateFlow()
private val _pendingRunCount = MutableStateFlow(0)
val pendingRunCount: StateFlow<Int> = _pendingRunCount.asStateFlow()
private val _streamingAssistantText = MutableStateFlow<String?>(null)
val streamingAssistantText: StateFlow<String?> = _streamingAssistantText.asStateFlow()
private val pendingToolCallsById = ConcurrentHashMap<String, ChatPendingToolCall>()
private val _pendingToolCalls = MutableStateFlow<List<ChatPendingToolCall>>(emptyList())
val pendingToolCalls: StateFlow<List<ChatPendingToolCall>> = _pendingToolCalls.asStateFlow()
private val _sessions = MutableStateFlow<List<ChatSessionEntry>>(emptyList())
val sessions: StateFlow<List<ChatSessionEntry>> = _sessions.asStateFlow()
private val pendingRuns = mutableSetOf<String>()
private val pendingRunTimeoutJobs = ConcurrentHashMap<String, Job>()
private val pendingRunTimeoutMs = 120_000L
private var lastHealthPollAtMs: Long? = null
fun onDisconnected(message: String) {
_healthOk.value = false
// Not an error; keep connection status in the UI pill.
_errorText.value = null
clearPendingRuns()
pendingToolCallsById.clear()
publishPendingToolCalls()
_streamingAssistantText.value = null
_sessionId.value = null
}
fun load(sessionKey: String = "main") {
val key = sessionKey.trim().ifEmpty { "main" }
_sessionKey.value = key
scope.launch { bootstrap(forceHealth = true) }
}
fun refresh() {
scope.launch { bootstrap(forceHealth = true) }
}
fun refreshSessions(limit: Int? = null) {
scope.launch { fetchSessions(limit = limit) }
}
fun setThinkingLevel(thinkingLevel: String) {
val normalized = normalizeThinking(thinkingLevel)
if (normalized == _thinkingLevel.value) return
_thinkingLevel.value = normalized
}
fun switchSession(sessionKey: String) {
val key = sessionKey.trim()
if (key.isEmpty()) return
if (key == _sessionKey.value) return
_sessionKey.value = key
scope.launch { bootstrap(forceHealth = true) }
}
fun sendMessage(
message: String,
thinkingLevel: String,
attachments: List<OutgoingAttachment>,
) {
val trimmed = message.trim()
if (trimmed.isEmpty() && attachments.isEmpty()) return
if (!_healthOk.value) {
_errorText.value = "Gateway health not OK; cannot send"
return
}
val runId = UUID.randomUUID().toString()
val text = if (trimmed.isEmpty() && attachments.isNotEmpty()) "See attached." else trimmed
val sessionKey = _sessionKey.value
val thinking = normalizeThinking(thinkingLevel)
// Optimistic user message.
val userContent =
buildList {
add(ChatMessageContent(type = "text", text = text))
for (att in attachments) {
add(
ChatMessageContent(
type = att.type,
mimeType = att.mimeType,
fileName = att.fileName,
base64 = att.base64,
),
)
}
}
_messages.value =
_messages.value +
ChatMessage(
id = UUID.randomUUID().toString(),
role = "user",
content = userContent,
timestampMs = System.currentTimeMillis(),
)
armPendingRunTimeout(runId)
synchronized(pendingRuns) {
pendingRuns.add(runId)
_pendingRunCount.value = pendingRuns.size
}
_errorText.value = null
_streamingAssistantText.value = null
pendingToolCallsById.clear()
publishPendingToolCalls()
scope.launch {
try {
val params =
buildJsonObject {
put("sessionKey", JsonPrimitive(sessionKey))
put("message", JsonPrimitive(text))
put("thinking", JsonPrimitive(thinking))
put("timeoutMs", JsonPrimitive(30_000))
put("idempotencyKey", JsonPrimitive(runId))
if (attachments.isNotEmpty()) {
put(
"attachments",
JsonArray(
attachments.map { att ->
buildJsonObject {
put("type", JsonPrimitive(att.type))
put("mimeType", JsonPrimitive(att.mimeType))
put("fileName", JsonPrimitive(att.fileName))
put("content", JsonPrimitive(att.base64))
}
},
),
)
}
}
val res = session.request("chat.send", params.toString())
val actualRunId = parseRunId(res) ?: runId
if (actualRunId != runId) {
clearPendingRun(runId)
armPendingRunTimeout(actualRunId)
synchronized(pendingRuns) {
pendingRuns.add(actualRunId)
_pendingRunCount.value = pendingRuns.size
}
}
} catch (err: Throwable) {
clearPendingRun(runId)
_errorText.value = err.message
}
}
}
fun abort() {
val runIds =
synchronized(pendingRuns) {
pendingRuns.toList()
}
if (runIds.isEmpty()) return
scope.launch {
for (runId in runIds) {
try {
val params =
buildJsonObject {
put("sessionKey", JsonPrimitive(_sessionKey.value))
put("runId", JsonPrimitive(runId))
}
session.request("chat.abort", params.toString())
} catch (_: Throwable) {
// best-effort
}
}
}
}
fun handleBridgeEvent(event: String, payloadJson: String?) {
when (event) {
"tick" -> {
scope.launch { pollHealthIfNeeded(force = false) }
}
"health" -> {
// If we receive a health snapshot, the gateway is reachable.
_healthOk.value = true
}
"seqGap" -> {
_errorText.value = "Event stream interrupted; try refreshing."
clearPendingRuns()
}
"chat" -> {
if (payloadJson.isNullOrBlank()) return
handleChatEvent(payloadJson)
}
"agent" -> {
if (payloadJson.isNullOrBlank()) return
handleAgentEvent(payloadJson)
}
}
}
private suspend fun bootstrap(forceHealth: Boolean) {
_errorText.value = null
_healthOk.value = false
clearPendingRuns()
pendingToolCallsById.clear()
publishPendingToolCalls()
_streamingAssistantText.value = null
_sessionId.value = null
val key = _sessionKey.value
try {
try {
session.sendEvent("chat.subscribe", """{"sessionKey":"$key"}""")
} catch (_: Throwable) {
// best-effort
}
val historyJson = session.request("chat.history", """{"sessionKey":"$key"}""")
val history = parseHistory(historyJson, sessionKey = key)
_messages.value = history.messages
_sessionId.value = history.sessionId
history.thinkingLevel?.trim()?.takeIf { it.isNotEmpty() }?.let { _thinkingLevel.value = it }
pollHealthIfNeeded(force = forceHealth)
fetchSessions(limit = 50)
} catch (err: Throwable) {
_errorText.value = err.message
}
}
private suspend fun fetchSessions(limit: Int?) {
try {
val params =
buildJsonObject {
put("includeGlobal", JsonPrimitive(true))
put("includeUnknown", JsonPrimitive(false))
if (limit != null && limit > 0) put("limit", JsonPrimitive(limit))
}
val res = session.request("sessions.list", params.toString())
_sessions.value = parseSessions(res)
} catch (_: Throwable) {
// best-effort
}
}
private suspend fun pollHealthIfNeeded(force: Boolean) {
val now = System.currentTimeMillis()
val last = lastHealthPollAtMs
if (!force && last != null && now - last < 10_000) return
lastHealthPollAtMs = now
try {
session.request("health", null)
_healthOk.value = true
} catch (_: Throwable) {
_healthOk.value = false
}
}
private fun handleChatEvent(payloadJson: String) {
val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return
val sessionKey = payload["sessionKey"].asStringOrNull()?.trim()
if (!sessionKey.isNullOrEmpty() && sessionKey != _sessionKey.value) return
val runId = payload["runId"].asStringOrNull()
if (runId != null) {
val isPending =
synchronized(pendingRuns) {
pendingRuns.contains(runId)
}
if (!isPending) return
}
val state = payload["state"].asStringOrNull()
when (state) {
"final", "aborted", "error" -> {
if (state == "error") {
_errorText.value = payload["errorMessage"].asStringOrNull() ?: "Chat failed"
}
if (runId != null) clearPendingRun(runId) else clearPendingRuns()
pendingToolCallsById.clear()
publishPendingToolCalls()
_streamingAssistantText.value = null
scope.launch {
try {
val historyJson =
session.request("chat.history", """{"sessionKey":"${_sessionKey.value}"}""")
val history = parseHistory(historyJson, sessionKey = _sessionKey.value)
_messages.value = history.messages
_sessionId.value = history.sessionId
history.thinkingLevel?.trim()?.takeIf { it.isNotEmpty() }?.let { _thinkingLevel.value = it }
} catch (_: Throwable) {
// best-effort
}
}
}
}
}
private fun handleAgentEvent(payloadJson: String) {
val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return
val runId = payload["runId"].asStringOrNull()
val sessionId = _sessionId.value
if (sessionId != null && runId != sessionId) return
val stream = payload["stream"].asStringOrNull()
val data = payload["data"].asObjectOrNull()
when (stream) {
"assistant" -> {
val text = data?.get("text")?.asStringOrNull()
if (!text.isNullOrEmpty()) {
_streamingAssistantText.value = text
}
}
"tool" -> {
val phase = data?.get("phase")?.asStringOrNull()
val name = data?.get("name")?.asStringOrNull()
val toolCallId = data?.get("toolCallId")?.asStringOrNull()
if (phase.isNullOrEmpty() || name.isNullOrEmpty() || toolCallId.isNullOrEmpty()) return
val ts = payload["ts"].asLongOrNull() ?: System.currentTimeMillis()
if (phase == "start") {
pendingToolCallsById[toolCallId] =
ChatPendingToolCall(
toolCallId = toolCallId,
name = name,
startedAtMs = ts,
isError = null,
)
publishPendingToolCalls()
} else if (phase == "result") {
pendingToolCallsById.remove(toolCallId)
publishPendingToolCalls()
}
}
"error" -> {
_errorText.value = "Event stream interrupted; try refreshing."
clearPendingRuns()
pendingToolCallsById.clear()
publishPendingToolCalls()
_streamingAssistantText.value = null
}
}
}
private fun publishPendingToolCalls() {
_pendingToolCalls.value =
pendingToolCallsById.values.sortedBy { it.startedAtMs }
}
private fun armPendingRunTimeout(runId: String) {
pendingRunTimeoutJobs[runId]?.cancel()
pendingRunTimeoutJobs[runId] =
scope.launch {
delay(pendingRunTimeoutMs)
val stillPending =
synchronized(pendingRuns) {
pendingRuns.contains(runId)
}
if (!stillPending) return@launch
clearPendingRun(runId)
_errorText.value = "Timed out waiting for a reply; try again or refresh."
}
}
private fun clearPendingRun(runId: String) {
pendingRunTimeoutJobs.remove(runId)?.cancel()
synchronized(pendingRuns) {
pendingRuns.remove(runId)
_pendingRunCount.value = pendingRuns.size
}
}
private fun clearPendingRuns() {
for ((_, job) in pendingRunTimeoutJobs) {
job.cancel()
}
pendingRunTimeoutJobs.clear()
synchronized(pendingRuns) {
pendingRuns.clear()
_pendingRunCount.value = 0
}
}
private fun parseHistory(historyJson: String, sessionKey: String): ChatHistory {
val root = json.parseToJsonElement(historyJson).asObjectOrNull() ?: return ChatHistory(sessionKey, null, null, emptyList())
val sid = root["sessionId"].asStringOrNull()
val thinkingLevel = root["thinkingLevel"].asStringOrNull()
val array = root["messages"].asArrayOrNull() ?: JsonArray(emptyList())
val messages =
array.mapNotNull { item ->
val obj = item.asObjectOrNull() ?: return@mapNotNull null
val role = obj["role"].asStringOrNull() ?: return@mapNotNull null
val content = obj["content"].asArrayOrNull()?.mapNotNull(::parseMessageContent) ?: emptyList()
val ts = obj["timestamp"].asLongOrNull()
ChatMessage(
id = UUID.randomUUID().toString(),
role = role,
content = content,
timestampMs = ts,
)
}
return ChatHistory(sessionKey = sessionKey, sessionId = sid, thinkingLevel = thinkingLevel, messages = messages)
}
private fun parseMessageContent(el: JsonElement): ChatMessageContent? {
val obj = el.asObjectOrNull() ?: return null
val type = obj["type"].asStringOrNull() ?: "text"
return if (type == "text") {
ChatMessageContent(type = "text", text = obj["text"].asStringOrNull())
} else {
ChatMessageContent(
type = type,
mimeType = obj["mimeType"].asStringOrNull(),
fileName = obj["fileName"].asStringOrNull(),
base64 = obj["content"].asStringOrNull(),
)
}
}
private fun parseSessions(jsonString: String): List<ChatSessionEntry> {
val root = json.parseToJsonElement(jsonString).asObjectOrNull() ?: return emptyList()
val sessions = root["sessions"].asArrayOrNull() ?: return emptyList()
return sessions.mapNotNull { item ->
val obj = item.asObjectOrNull() ?: return@mapNotNull null
val key = obj["key"].asStringOrNull()?.trim().orEmpty()
if (key.isEmpty()) return@mapNotNull null
val updatedAt = obj["updatedAt"].asLongOrNull()
ChatSessionEntry(key = key, updatedAtMs = updatedAt)
}
}
private fun parseRunId(resJson: String): String? {
return try {
json.parseToJsonElement(resJson).asObjectOrNull()?.get("runId").asStringOrNull()
} catch (_: Throwable) {
null
}
}
private fun normalizeThinking(raw: String): String {
return when (raw.trim().lowercase()) {
"low" -> "low"
"medium" -> "medium"
"high" -> "high"
else -> "off"
}
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonElement?.asArrayOrNull(): JsonArray? = this as? JsonArray
private fun JsonElement?.asStringOrNull(): String? =
when (this) {
is JsonNull -> null
is JsonPrimitive -> content
else -> null
}
private fun JsonElement?.asLongOrNull(): Long? =
when (this) {
is JsonPrimitive -> content.toLongOrNull()
else -> null
}

View File

@@ -0,0 +1,42 @@
package com.steipete.clawdis.node.chat
data class ChatMessage(
val id: String,
val role: String,
val content: List<ChatMessageContent>,
val timestampMs: Long?,
)
data class ChatMessageContent(
val type: String = "text",
val text: String? = null,
val mimeType: String? = null,
val fileName: String? = null,
val base64: String? = null,
)
data class ChatPendingToolCall(
val toolCallId: String,
val name: String,
val startedAtMs: Long,
val isError: Boolean? = null,
)
data class ChatSessionEntry(
val key: String,
val updatedAtMs: Long?,
)
data class ChatHistory(
val sessionKey: String,
val sessionId: String?,
val thinkingLevel: String?,
val messages: List<ChatMessage>,
)
data class OutgoingAttachment(
val type: String,
val mimeType: String,
val fileName: String,
val base64: String,
)

View File

@@ -0,0 +1,258 @@
package com.steipete.clawdis.node.node
import android.Manifest
import android.content.Context
import android.annotation.SuppressLint
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.util.Base64
import android.content.pm.PackageManager
import androidx.lifecycle.LifecycleOwner
import androidx.camera.core.CameraSelector
import androidx.camera.core.ImageCapture
import androidx.camera.core.ImageCaptureException
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.video.FileOutputOptions
import androidx.camera.video.Recorder
import androidx.camera.video.Recording
import androidx.camera.video.VideoCapture
import androidx.camera.video.VideoRecordEvent
import androidx.core.content.ContextCompat
import androidx.core.content.ContextCompat.checkSelfPermission
import androidx.core.graphics.scale
import com.steipete.clawdis.node.PermissionRequester
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withTimeout
import kotlinx.coroutines.withContext
import java.io.ByteArrayOutputStream
import java.io.File
import java.util.concurrent.Executor
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
class CameraCaptureManager(private val context: Context) {
data class Payload(val payloadJson: String)
@Volatile private var lifecycleOwner: LifecycleOwner? = null
@Volatile private var permissionRequester: PermissionRequester? = null
fun attachLifecycleOwner(owner: LifecycleOwner) {
lifecycleOwner = owner
}
fun attachPermissionRequester(requester: PermissionRequester) {
permissionRequester = requester
}
private suspend fun ensureCameraPermission() {
val granted = checkSelfPermission(context, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester = permissionRequester
?: throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.CAMERA))
if (results[Manifest.permission.CAMERA] != true) {
throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission")
}
}
private suspend fun ensureMicPermission() {
val granted = checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
if (granted) return
val requester = permissionRequester
?: throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
val results = requester.requestIfMissing(listOf(Manifest.permission.RECORD_AUDIO))
if (results[Manifest.permission.RECORD_AUDIO] != true) {
throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
}
}
suspend fun snap(paramsJson: String?): Payload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val facing = parseFacing(paramsJson) ?: "front"
val quality = (parseQuality(paramsJson) ?: 0.9).coerceIn(0.1, 1.0)
val maxWidth = parseMaxWidth(paramsJson)
val provider = context.cameraProvider()
val capture = ImageCapture.Builder().build()
val selector =
if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA
provider.unbindAll()
provider.bindToLifecycle(owner, selector, capture)
val bytes = capture.takeJpegBytes(context.mainExecutor())
val decoded = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
?: throw IllegalStateException("UNAVAILABLE: failed to decode captured image")
val scaled =
if (maxWidth != null && maxWidth > 0 && decoded.width > maxWidth) {
val h =
(decoded.height.toDouble() * (maxWidth.toDouble() / decoded.width.toDouble()))
.toInt()
.coerceAtLeast(1)
decoded.scale(maxWidth, h)
} else {
decoded
}
val out = ByteArrayOutputStream()
val jpegQuality = (quality * 100.0).toInt().coerceIn(10, 100)
if (!scaled.compress(Bitmap.CompressFormat.JPEG, jpegQuality, out)) {
throw IllegalStateException("UNAVAILABLE: failed to encode JPEG")
}
val base64 = Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
Payload(
"""{"format":"jpg","base64":"$base64","width":${scaled.width},"height":${scaled.height}}""",
)
}
@SuppressLint("MissingPermission")
suspend fun clip(paramsJson: String?): Payload =
withContext(Dispatchers.Main) {
ensureCameraPermission()
val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready")
val facing = parseFacing(paramsJson) ?: "front"
val durationMs = (parseDurationMs(paramsJson) ?: 3_000).coerceIn(200, 60_000)
val includeAudio = parseIncludeAudio(paramsJson) ?: true
if (includeAudio) ensureMicPermission()
val provider = context.cameraProvider()
val recorder = Recorder.Builder().build()
val videoCapture = VideoCapture.withOutput(recorder)
val selector =
if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA
provider.unbindAll()
provider.bindToLifecycle(owner, selector, videoCapture)
val file = File.createTempFile("clawdis-clip-", ".mp4")
val outputOptions = FileOutputOptions.Builder(file).build()
val finalized = kotlinx.coroutines.CompletableDeferred<VideoRecordEvent.Finalize>()
val recording: Recording =
videoCapture.output
.prepareRecording(context, outputOptions)
.apply {
if (includeAudio) withAudioEnabled()
}
.start(context.mainExecutor()) { event ->
if (event is VideoRecordEvent.Finalize) {
finalized.complete(event)
}
}
try {
kotlinx.coroutines.delay(durationMs.toLong())
} finally {
recording.stop()
}
val finalizeEvent =
try {
withTimeout(10_000) { finalized.await() }
} catch (err: Throwable) {
file.delete()
throw IllegalStateException("UNAVAILABLE: camera clip finalize timed out")
}
if (finalizeEvent.hasError()) {
file.delete()
throw IllegalStateException("UNAVAILABLE: camera clip failed")
}
val bytes = file.readBytes()
file.delete()
val base64 = Base64.encodeToString(bytes, Base64.NO_WRAP)
Payload(
"""{"format":"mp4","base64":"$base64","durationMs":$durationMs,"hasAudio":${includeAudio}}""",
)
}
private fun parseFacing(paramsJson: String?): String? =
when {
paramsJson?.contains("\"front\"") == true -> "front"
paramsJson?.contains("\"back\"") == true -> "back"
else -> null
}
private fun parseQuality(paramsJson: String?): Double? =
parseNumber(paramsJson, key = "quality")?.toDoubleOrNull()
private fun parseMaxWidth(paramsJson: String?): Int? =
parseNumber(paramsJson, key = "maxWidth")?.toIntOrNull()
private fun parseDurationMs(paramsJson: String?): Int? =
parseNumber(paramsJson, key = "durationMs")?.toIntOrNull()
private fun parseIncludeAudio(paramsJson: String?): Boolean? {
val raw = paramsJson ?: return null
val key = "\"includeAudio\""
val idx = raw.indexOf(key)
if (idx < 0) return null
val colon = raw.indexOf(':', idx + key.length)
if (colon < 0) return null
val tail = raw.substring(colon + 1).trimStart()
return when {
tail.startsWith("true") -> true
tail.startsWith("false") -> false
else -> null
}
}
private fun parseNumber(paramsJson: String?, key: String): String? {
val raw = paramsJson ?: return null
val needle = "\"$key\""
val idx = raw.indexOf(needle)
if (idx < 0) return null
val colon = raw.indexOf(':', idx + needle.length)
if (colon < 0) return null
val tail = raw.substring(colon + 1).trimStart()
return tail.takeWhile { it.isDigit() || it == '.' }
}
private fun Context.mainExecutor(): Executor = ContextCompat.getMainExecutor(this)
}
private suspend fun Context.cameraProvider(): ProcessCameraProvider =
suspendCancellableCoroutine { cont ->
val future = ProcessCameraProvider.getInstance(this)
future.addListener(
{
try {
cont.resume(future.get())
} catch (e: Exception) {
cont.resumeWithException(e)
}
},
ContextCompat.getMainExecutor(this),
)
}
private suspend fun ImageCapture.takeJpegBytes(executor: Executor): ByteArray =
suspendCancellableCoroutine { cont ->
val file = File.createTempFile("clawdis-snap-", ".jpg")
val options = ImageCapture.OutputFileOptions.Builder(file).build()
takePicture(
options,
executor,
object : ImageCapture.OnImageSavedCallback {
override fun onError(exception: ImageCaptureException) {
cont.resumeWithException(exception)
}
override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
try {
val bytes = file.readBytes()
cont.resume(bytes)
} catch (e: Exception) {
cont.resumeWithException(e)
} finally {
file.delete()
}
}
},
)
}

View File

@@ -0,0 +1,256 @@
package com.steipete.clawdis.node.node
import android.graphics.Bitmap
import android.graphics.Canvas
import android.os.Looper
import android.webkit.WebView
import androidx.core.graphics.createBitmap
import androidx.core.graphics.scale
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.withContext
import java.io.ByteArrayOutputStream
import android.util.Base64
import org.json.JSONObject
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlin.coroutines.resume
class CanvasController {
enum class SnapshotFormat(val rawValue: String) {
Png("png"),
Jpeg("jpeg"),
}
@Volatile private var webView: WebView? = null
@Volatile private var url: String? = null
@Volatile private var debugStatusEnabled: Boolean = false
@Volatile private var debugStatusTitle: String? = null
@Volatile private var debugStatusSubtitle: String? = null
private val scaffoldAssetUrl = "file:///android_asset/CanvasScaffold/scaffold.html"
private fun clampJpegQuality(quality: Double?): Int {
val q = (quality ?: 0.82).coerceIn(0.1, 1.0)
return (q * 100.0).toInt().coerceIn(1, 100)
}
fun attach(webView: WebView) {
this.webView = webView
reload()
applyDebugStatus()
}
fun navigate(url: String) {
val trimmed = url.trim()
this.url = if (trimmed.isBlank() || trimmed == "/") null else trimmed
reload()
}
fun currentUrl(): String? = url
fun isDefaultCanvas(): Boolean = url == null
fun setDebugStatusEnabled(enabled: Boolean) {
debugStatusEnabled = enabled
applyDebugStatus()
}
fun setDebugStatus(title: String?, subtitle: String?) {
debugStatusTitle = title
debugStatusSubtitle = subtitle
applyDebugStatus()
}
fun onPageFinished() {
applyDebugStatus()
}
private inline fun withWebViewOnMain(crossinline block: (WebView) -> Unit) {
val wv = webView ?: return
if (Looper.myLooper() == Looper.getMainLooper()) {
block(wv)
} else {
wv.post { block(wv) }
}
}
private fun reload() {
val currentUrl = url
withWebViewOnMain { wv ->
if (currentUrl == null) {
wv.loadUrl(scaffoldAssetUrl)
} else {
wv.loadUrl(currentUrl)
}
}
}
private fun applyDebugStatus() {
val enabled = debugStatusEnabled
val title = debugStatusTitle
val subtitle = debugStatusSubtitle
withWebViewOnMain { wv ->
val titleJs = title?.let { JSONObject.quote(it) } ?: "null"
val subtitleJs = subtitle?.let { JSONObject.quote(it) } ?: "null"
val js = """
(() => {
try {
const api = globalThis.__clawdis;
if (!api) return;
if (typeof api.setDebugStatusEnabled === 'function') {
api.setDebugStatusEnabled(${if (enabled) "true" else "false"});
}
if (!${if (enabled) "true" else "false"}) return;
if (typeof api.setStatus === 'function') {
api.setStatus($titleJs, $subtitleJs);
}
} catch (_) {}
})();
""".trimIndent()
wv.evaluateJavascript(js, null)
}
}
suspend fun eval(javaScript: String): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
suspendCancellableCoroutine { cont ->
wv.evaluateJavascript(javaScript) { result ->
cont.resume(result ?: "")
}
}
}
suspend fun snapshotPngBase64(maxWidth: Int?): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
val bmp = wv.captureBitmap()
val scaled =
if (maxWidth != null && maxWidth > 0 && bmp.width > maxWidth) {
val h = (bmp.height.toDouble() * (maxWidth.toDouble() / bmp.width.toDouble())).toInt().coerceAtLeast(1)
bmp.scale(maxWidth, h)
} else {
bmp
}
val out = ByteArrayOutputStream()
scaled.compress(Bitmap.CompressFormat.PNG, 100, out)
Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
}
suspend fun snapshotBase64(format: SnapshotFormat, quality: Double?, maxWidth: Int?): String =
withContext(Dispatchers.Main) {
val wv = webView ?: throw IllegalStateException("no webview")
val bmp = wv.captureBitmap()
val scaled =
if (maxWidth != null && maxWidth > 0 && bmp.width > maxWidth) {
val h = (bmp.height.toDouble() * (maxWidth.toDouble() / bmp.width.toDouble())).toInt().coerceAtLeast(1)
bmp.scale(maxWidth, h)
} else {
bmp
}
val out = ByteArrayOutputStream()
val (compressFormat, compressQuality) =
when (format) {
SnapshotFormat.Png -> Bitmap.CompressFormat.PNG to 100
SnapshotFormat.Jpeg -> Bitmap.CompressFormat.JPEG to clampJpegQuality(quality)
}
scaled.compress(compressFormat, compressQuality, out)
Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
}
private suspend fun WebView.captureBitmap(): Bitmap =
suspendCancellableCoroutine { cont ->
val width = width.coerceAtLeast(1)
val height = height.coerceAtLeast(1)
val bitmap = createBitmap(width, height, Bitmap.Config.ARGB_8888)
// WebView isn't supported by PixelCopy.request(...) directly; draw() is the most reliable
// cross-version snapshot for this lightweight "canvas" use-case.
draw(Canvas(bitmap))
cont.resume(bitmap)
}
companion object {
data class SnapshotParams(val format: SnapshotFormat, val quality: Double?, val maxWidth: Int?)
fun parseNavigateUrl(paramsJson: String?): String {
val obj = parseParamsObject(paramsJson) ?: return ""
return obj.string("url").trim()
}
fun parseEvalJs(paramsJson: String?): String? {
val obj = parseParamsObject(paramsJson) ?: return null
val js = obj.string("javaScript").trim()
return js.takeIf { it.isNotBlank() }
}
fun parseSnapshotMaxWidth(paramsJson: String?): Int? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("maxWidth")) return null
val width = obj.int("maxWidth") ?: 0
return width.takeIf { it > 0 }
}
fun parseSnapshotFormat(paramsJson: String?): SnapshotFormat {
val obj = parseParamsObject(paramsJson) ?: return SnapshotFormat.Jpeg
val raw = obj.string("format").trim().lowercase()
return when (raw) {
"png" -> SnapshotFormat.Png
"jpeg", "jpg" -> SnapshotFormat.Jpeg
"" -> SnapshotFormat.Jpeg
else -> SnapshotFormat.Jpeg
}
}
fun parseSnapshotQuality(paramsJson: String?): Double? {
val obj = parseParamsObject(paramsJson) ?: return null
if (!obj.containsKey("quality")) return null
val q = obj.double("quality") ?: Double.NaN
if (!q.isFinite()) return null
return q.coerceIn(0.1, 1.0)
}
fun parseSnapshotParams(paramsJson: String?): SnapshotParams {
return SnapshotParams(
format = parseSnapshotFormat(paramsJson),
quality = parseSnapshotQuality(paramsJson),
maxWidth = parseSnapshotMaxWidth(paramsJson),
)
}
private val json = Json { ignoreUnknownKeys = true }
private fun parseParamsObject(paramsJson: String?): JsonObject? {
val raw = paramsJson?.trim().orEmpty()
if (raw.isEmpty()) return null
return try {
json.parseToJsonElement(raw).asObjectOrNull()
} catch (_: Throwable) {
null
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
private fun JsonObject.string(key: String): String {
val prim = this[key] as? JsonPrimitive ?: return ""
val raw = prim.content
return raw.takeIf { it != "null" }.orEmpty()
}
private fun JsonObject.int(key: String): Int? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toIntOrNull()
}
private fun JsonObject.double(key: String): Double? {
val prim = this[key] as? JsonPrimitive ?: return null
return prim.content.toDoubleOrNull()
}
}
}

View File

@@ -0,0 +1,196 @@
package com.steipete.clawdis.node.node
import android.content.Context
import android.hardware.display.DisplayManager
import android.media.MediaRecorder
import android.media.projection.MediaProjectionManager
import android.util.Base64
import com.steipete.clawdis.node.ScreenCaptureRequester
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.withContext
import java.io.File
import kotlin.math.roundToInt
class ScreenRecordManager(private val context: Context) {
data class Payload(val payloadJson: String)
@Volatile private var screenCaptureRequester: ScreenCaptureRequester? = null
@Volatile private var permissionRequester: com.steipete.clawdis.node.PermissionRequester? = null
fun attachScreenCaptureRequester(requester: ScreenCaptureRequester) {
screenCaptureRequester = requester
}
fun attachPermissionRequester(requester: com.steipete.clawdis.node.PermissionRequester) {
permissionRequester = requester
}
suspend fun record(paramsJson: String?): Payload =
withContext(Dispatchers.Default) {
val requester =
screenCaptureRequester
?: throw IllegalStateException(
"SCREEN_PERMISSION_REQUIRED: grant Screen Recording permission",
)
val durationMs = (parseDurationMs(paramsJson) ?: 10_000).coerceIn(250, 60_000)
val fps = (parseFps(paramsJson) ?: 10.0).coerceIn(1.0, 60.0)
val fpsInt = fps.roundToInt().coerceIn(1, 60)
val screenIndex = parseScreenIndex(paramsJson)
val includeAudio = parseIncludeAudio(paramsJson) ?: true
val format = parseString(paramsJson, key = "format")
if (format != null && format.lowercase() != "mp4") {
throw IllegalArgumentException("INVALID_REQUEST: screen format must be mp4")
}
if (screenIndex != null && screenIndex != 0) {
throw IllegalArgumentException("INVALID_REQUEST: screenIndex must be 0 on Android")
}
val capture = requester.requestCapture()
?: throw IllegalStateException(
"SCREEN_PERMISSION_REQUIRED: grant Screen Recording permission",
)
val mgr =
context.getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager
val projection = mgr.getMediaProjection(capture.resultCode, capture.data)
?: throw IllegalStateException("UNAVAILABLE: screen capture unavailable")
val metrics = context.resources.displayMetrics
val width = metrics.widthPixels
val height = metrics.heightPixels
val densityDpi = metrics.densityDpi
val file = File.createTempFile("clawdis-screen-", ".mp4")
if (includeAudio) ensureMicPermission()
val recorder = MediaRecorder()
var virtualDisplay: android.hardware.display.VirtualDisplay? = null
try {
if (includeAudio) {
recorder.setAudioSource(MediaRecorder.AudioSource.MIC)
}
recorder.setVideoSource(MediaRecorder.VideoSource.SURFACE)
recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
recorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264)
if (includeAudio) {
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
recorder.setAudioChannels(1)
recorder.setAudioSamplingRate(44_100)
recorder.setAudioEncodingBitRate(96_000)
}
recorder.setVideoSize(width, height)
recorder.setVideoFrameRate(fpsInt)
recorder.setVideoEncodingBitRate(estimateBitrate(width, height, fpsInt))
recorder.setOutputFile(file.absolutePath)
recorder.prepare()
val surface = recorder.surface
virtualDisplay =
projection.createVirtualDisplay(
"clawdis-screen",
width,
height,
densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
surface,
null,
null,
)
recorder.start()
delay(durationMs.toLong())
} finally {
try {
recorder.stop()
} catch (_: Throwable) {
// ignore
}
recorder.reset()
recorder.release()
virtualDisplay?.release()
projection.stop()
}
val bytes = withContext(Dispatchers.IO) { file.readBytes() }
file.delete()
val base64 = Base64.encodeToString(bytes, Base64.NO_WRAP)
Payload(
"""{"format":"mp4","base64":"$base64","durationMs":$durationMs,"fps":$fpsInt,"screenIndex":0,"hasAudio":$includeAudio}""",
)
}
private suspend fun ensureMicPermission() {
val granted =
androidx.core.content.ContextCompat.checkSelfPermission(
context,
android.Manifest.permission.RECORD_AUDIO,
) == android.content.pm.PackageManager.PERMISSION_GRANTED
if (granted) return
val requester =
permissionRequester
?: throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
val results = requester.requestIfMissing(listOf(android.Manifest.permission.RECORD_AUDIO))
if (results[android.Manifest.permission.RECORD_AUDIO] != true) {
throw IllegalStateException("MIC_PERMISSION_REQUIRED: grant Microphone permission")
}
}
private fun parseDurationMs(paramsJson: String?): Int? =
parseNumber(paramsJson, key = "durationMs")?.toIntOrNull()
private fun parseFps(paramsJson: String?): Double? =
parseNumber(paramsJson, key = "fps")?.toDoubleOrNull()
private fun parseScreenIndex(paramsJson: String?): Int? =
parseNumber(paramsJson, key = "screenIndex")?.toIntOrNull()
private fun parseIncludeAudio(paramsJson: String?): Boolean? {
val raw = paramsJson ?: return null
val key = "\"includeAudio\""
val idx = raw.indexOf(key)
if (idx < 0) return null
val colon = raw.indexOf(':', idx + key.length)
if (colon < 0) return null
val tail = raw.substring(colon + 1).trimStart()
return when {
tail.startsWith("true") -> true
tail.startsWith("false") -> false
else -> null
}
}
private fun parseNumber(paramsJson: String?, key: String): String? {
val raw = paramsJson ?: return null
val needle = "\"$key\""
val idx = raw.indexOf(needle)
if (idx < 0) return null
val colon = raw.indexOf(':', idx + needle.length)
if (colon < 0) return null
val tail = raw.substring(colon + 1).trimStart()
return tail.takeWhile { it.isDigit() || it == '.' || it == '-' }
}
private fun parseString(paramsJson: String?, key: String): String? {
val raw = paramsJson ?: return null
val needle = "\"$key\""
val idx = raw.indexOf(needle)
if (idx < 0) return null
val colon = raw.indexOf(':', idx + needle.length)
if (colon < 0) return null
val tail = raw.substring(colon + 1).trimStart()
if (!tail.startsWith('\"')) return null
val rest = tail.drop(1)
val end = rest.indexOf('\"')
if (end < 0) return null
return rest.substring(0, end)
}
private fun estimateBitrate(width: Int, height: Int, fps: Int): Int {
val pixels = width.toLong() * height.toLong()
val raw = (pixels * fps.toLong() * 2L).toInt()
return raw.coerceIn(1_000_000, 12_000_000)
}
}

View File

@@ -0,0 +1,66 @@
package com.steipete.clawdis.node.protocol
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
object ClawdisCanvasA2UIAction {
fun extractActionName(userAction: JsonObject): String? {
val name =
(userAction["name"] as? JsonPrimitive)
?.content
?.trim()
.orEmpty()
if (name.isNotEmpty()) return name
val action =
(userAction["action"] as? JsonPrimitive)
?.content
?.trim()
.orEmpty()
return action.ifEmpty { null }
}
fun sanitizeTagValue(value: String): String {
val trimmed = value.trim().ifEmpty { "-" }
val normalized = trimmed.replace(" ", "_")
val out = StringBuilder(normalized.length)
for (c in normalized) {
val ok =
c.isLetterOrDigit() ||
c == '_' ||
c == '-' ||
c == '.' ||
c == ':'
out.append(if (ok) c else '_')
}
return out.toString()
}
fun formatAgentMessage(
actionName: String,
sessionKey: String,
surfaceId: String,
sourceComponentId: String,
host: String,
instanceId: String,
contextJson: String?,
): String {
val ctxSuffix = contextJson?.takeIf { it.isNotBlank() }?.let { " ctx=$it" }.orEmpty()
return listOf(
"CANVAS_A2UI",
"action=${sanitizeTagValue(actionName)}",
"session=${sanitizeTagValue(sessionKey)}",
"surface=${sanitizeTagValue(surfaceId)}",
"component=${sanitizeTagValue(sourceComponentId)}",
"host=${sanitizeTagValue(host)}",
"instance=${sanitizeTagValue(instanceId)}$ctxSuffix",
"default=update_canvas",
).joinToString(separator = " ")
}
fun jsDispatchA2UIActionStatus(actionId: String, ok: Boolean, error: String?): String {
val err = (error ?: "").replace("\\", "\\\\").replace("\"", "\\\"")
val okLiteral = if (ok) "true" else "false"
val idEscaped = actionId.replace("\\", "\\\\").replace("\"", "\\\"")
return "window.dispatchEvent(new CustomEvent('clawdis:a2ui-action-status', { detail: { id: \"${idEscaped}\", ok: ${okLiteral}, error: \"${err}\" } }));"
}
}

View File

@@ -0,0 +1,51 @@
package com.steipete.clawdis.node.protocol
enum class ClawdisCapability(val rawValue: String) {
Canvas("canvas"),
Camera("camera"),
Screen("screen"),
VoiceWake("voiceWake"),
}
enum class ClawdisCanvasCommand(val rawValue: String) {
Present("canvas.present"),
Hide("canvas.hide"),
Navigate("canvas.navigate"),
Eval("canvas.eval"),
Snapshot("canvas.snapshot"),
;
companion object {
const val NamespacePrefix: String = "canvas."
}
}
enum class ClawdisCanvasA2UICommand(val rawValue: String) {
Push("canvas.a2ui.push"),
PushJSONL("canvas.a2ui.pushJSONL"),
Reset("canvas.a2ui.reset"),
;
companion object {
const val NamespacePrefix: String = "canvas.a2ui."
}
}
enum class ClawdisCameraCommand(val rawValue: String) {
Snap("camera.snap"),
Clip("camera.clip"),
;
companion object {
const val NamespacePrefix: String = "camera."
}
}
enum class ClawdisScreenCommand(val rawValue: String) {
Record("screen.record"),
;
companion object {
const val NamespacePrefix: String = "screen."
}
}

View File

@@ -0,0 +1,123 @@
package com.steipete.clawdis.node.ui
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.slideInVertically
import androidx.compose.animation.slideOutVertically
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.statusBarsPadding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Error
import androidx.compose.material.icons.filled.FiberManualRecord
import androidx.compose.material.icons.filled.PhotoCamera
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableFloatStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.CameraHudKind
import com.steipete.clawdis.node.CameraHudState
import kotlinx.coroutines.delay
@Composable
fun CameraHudOverlay(
hud: CameraHudState?,
flashToken: Long,
modifier: Modifier = Modifier,
) {
Box(modifier = modifier.fillMaxSize()) {
CameraFlash(token = flashToken)
AnimatedVisibility(
visible = hud != null,
enter = slideInVertically(initialOffsetY = { -it / 2 }) + fadeIn(),
exit = slideOutVertically(targetOffsetY = { -it / 2 }) + fadeOut(),
modifier = Modifier.align(Alignment.TopStart).statusBarsPadding().padding(start = 12.dp, top = 58.dp),
) {
if (hud != null) {
Toast(hud = hud)
}
}
}
}
@Composable
private fun CameraFlash(token: Long) {
var alpha by remember { mutableFloatStateOf(0f) }
LaunchedEffect(token) {
if (token == 0L) return@LaunchedEffect
alpha = 0.85f
delay(110)
alpha = 0f
}
Box(
modifier =
Modifier
.fillMaxSize()
.alpha(alpha)
.background(Color.White),
)
}
@Composable
private fun Toast(hud: CameraHudState) {
Surface(
shape = RoundedCornerShape(14.dp),
color = MaterialTheme.colorScheme.surface.copy(alpha = 0.85f),
tonalElevation = 2.dp,
shadowElevation = 8.dp,
) {
Row(
modifier = Modifier.padding(vertical = 10.dp, horizontal = 12.dp),
verticalAlignment = Alignment.CenterVertically,
) {
when (hud.kind) {
CameraHudKind.Photo -> {
Icon(Icons.Default.PhotoCamera, contentDescription = null)
Spacer(Modifier.size(10.dp))
CircularProgressIndicator(modifier = Modifier.size(14.dp), strokeWidth = 2.dp)
}
CameraHudKind.Recording -> {
Icon(Icons.Default.FiberManualRecord, contentDescription = null, tint = Color.Red)
}
CameraHudKind.Success -> {
Icon(Icons.Default.CheckCircle, contentDescription = null)
}
CameraHudKind.Error -> {
Icon(Icons.Default.Error, contentDescription = null)
}
}
Spacer(Modifier.size(10.dp))
Text(
text = hud.message,
style = MaterialTheme.typography.bodyMedium,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
}
}
}

View File

@@ -0,0 +1,10 @@
package com.steipete.clawdis.node.ui
import androidx.compose.runtime.Composable
import com.steipete.clawdis.node.MainViewModel
import com.steipete.clawdis.node.ui.chat.ChatSheetContent
@Composable
fun ChatSheet(viewModel: MainViewModel) {
ChatSheetContent(viewModel = viewModel)
}

View File

@@ -0,0 +1,32 @@
package com.steipete.clawdis.node.ui
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.dynamicDarkColorScheme
import androidx.compose.material3.dynamicLightColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
@Composable
fun ClawdisTheme(content: @Composable () -> Unit) {
val context = LocalContext.current
val isDark = isSystemInDarkTheme()
val colorScheme = if (isDark) dynamicDarkColorScheme(context) else dynamicLightColorScheme(context)
MaterialTheme(colorScheme = colorScheme, content = content)
}
@Composable
fun overlayContainerColor(): Color {
val scheme = MaterialTheme.colorScheme
val isDark = isSystemInDarkTheme()
val base = if (isDark) scheme.surfaceContainerLow else scheme.surfaceContainerHigh
// Light mode: background stays dark (canvas), so clamp overlays away from pure-white glare.
return if (isDark) base else base.copy(alpha = 0.88f)
}
@Composable
fun overlayIconColor(): Color {
return MaterialTheme.colorScheme.onSurfaceVariant
}

View File

@@ -0,0 +1,240 @@
package com.steipete.clawdis.node.ui
import android.annotation.SuppressLint
import android.Manifest
import android.content.pm.PackageManager
import android.graphics.Color
import android.util.Log
import android.view.View
import android.webkit.JavascriptInterface
import android.webkit.WebView
import android.webkit.WebSettings
import android.webkit.WebResourceError
import android.webkit.WebResourceRequest
import android.webkit.WebResourceResponse
import android.webkit.WebViewClient
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.FilledTonalIconButton
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ChatBubble
import androidx.compose.material.icons.filled.Settings
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.compose.ui.window.Popup
import androidx.compose.ui.window.PopupProperties
import androidx.core.content.ContextCompat
import com.steipete.clawdis.node.MainViewModel
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun RootScreen(viewModel: MainViewModel) {
var sheet by remember { mutableStateOf<Sheet?>(null) }
val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = true)
val safeOverlayInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal)
val context = LocalContext.current
val serverName by viewModel.serverName.collectAsState()
val statusText by viewModel.statusText.collectAsState()
val cameraHud by viewModel.cameraHud.collectAsState()
val cameraFlashToken by viewModel.cameraFlashToken.collectAsState()
val bridgeState =
remember(serverName, statusText) {
when {
serverName != null -> BridgeState.Connected
statusText.contains("connecting", ignoreCase = true) ||
statusText.contains("reconnecting", ignoreCase = true) -> BridgeState.Connecting
statusText.contains("error", ignoreCase = true) -> BridgeState.Error
else -> BridgeState.Disconnected
}
}
val voiceEnabled =
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
Box(modifier = Modifier.fillMaxSize()) {
CanvasView(viewModel = viewModel, modifier = Modifier.fillMaxSize())
}
// Camera HUD (flash + toast) must be in a Popup to render above the WebView.
Popup(alignment = Alignment.Center, properties = PopupProperties(focusable = false)) {
CameraHudOverlay(hud = cameraHud, flashToken = cameraFlashToken, modifier = Modifier.fillMaxSize())
}
// Keep the overlay buttons above the WebView canvas (AndroidView), otherwise they may not receive touches.
Popup(alignment = Alignment.TopStart, properties = PopupProperties(focusable = false)) {
StatusPill(
bridge = bridgeState,
voiceEnabled = voiceEnabled,
onClick = { sheet = Sheet.Settings },
modifier = Modifier.windowInsetsPadding(safeOverlayInsets).padding(start = 12.dp, top = 12.dp),
)
}
Popup(alignment = Alignment.TopEnd, properties = PopupProperties(focusable = false)) {
Column(
modifier = Modifier.windowInsetsPadding(safeOverlayInsets).padding(end = 12.dp, top = 12.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
horizontalAlignment = Alignment.End,
) {
OverlayIconButton(
onClick = { sheet = Sheet.Chat },
icon = { Icon(Icons.Default.ChatBubble, contentDescription = "Chat") },
)
OverlayIconButton(
onClick = { sheet = Sheet.Settings },
icon = { Icon(Icons.Default.Settings, contentDescription = "Settings") },
)
}
}
val currentSheet = sheet
if (currentSheet != null) {
ModalBottomSheet(
onDismissRequest = { sheet = null },
sheetState = sheetState,
) {
when (currentSheet) {
Sheet.Chat -> ChatSheet(viewModel = viewModel)
Sheet.Settings -> SettingsSheet(viewModel = viewModel)
}
}
}
}
private enum class Sheet {
Chat,
Settings,
}
@Composable
private fun OverlayIconButton(
onClick: () -> Unit,
icon: @Composable () -> Unit,
) {
FilledTonalIconButton(
onClick = onClick,
modifier = Modifier.size(44.dp),
colors =
IconButtonDefaults.filledTonalIconButtonColors(
containerColor = overlayContainerColor(),
contentColor = overlayIconColor(),
),
) {
icon()
}
}
@SuppressLint("SetJavaScriptEnabled")
@Composable
private fun CanvasView(viewModel: MainViewModel, modifier: Modifier = Modifier) {
val context = LocalContext.current
val isDebuggable = (context.applicationInfo.flags and android.content.pm.ApplicationInfo.FLAG_DEBUGGABLE) != 0
AndroidView(
modifier = modifier,
factory = {
WebView(context).apply {
settings.javaScriptEnabled = true
// Some embedded web UIs (incl. the "background website") use localStorage/sessionStorage.
settings.domStorageEnabled = true
settings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE
webViewClient =
object : WebViewClient() {
override fun onReceivedError(
view: WebView,
request: WebResourceRequest,
error: WebResourceError,
) {
if (!isDebuggable) return
if (!request.isForMainFrame) return
Log.e("ClawdisWebView", "onReceivedError: ${error.errorCode} ${error.description} ${request.url}")
}
override fun onReceivedHttpError(
view: WebView,
request: WebResourceRequest,
errorResponse: WebResourceResponse,
) {
if (!isDebuggable) return
if (!request.isForMainFrame) return
Log.e(
"ClawdisWebView",
"onReceivedHttpError: ${errorResponse.statusCode} ${errorResponse.reasonPhrase} ${request.url}",
)
}
override fun onPageFinished(view: WebView, url: String?) {
viewModel.canvas.onPageFinished()
}
}
setBackgroundColor(Color.BLACK)
setLayerType(View.LAYER_TYPE_HARDWARE, null)
val a2uiBridge =
CanvasA2UIActionBridge { payload ->
viewModel.handleCanvasA2UIActionFromWebView(payload)
}
addJavascriptInterface(a2uiBridge, CanvasA2UIActionBridge.interfaceName)
addJavascriptInterface(
CanvasA2UIActionLegacyBridge(a2uiBridge),
CanvasA2UIActionLegacyBridge.interfaceName,
)
viewModel.canvas.attach(this)
}
},
)
}
private class CanvasA2UIActionBridge(private val onMessage: (String) -> Unit) {
@JavascriptInterface
fun postMessage(payload: String?) {
val msg = payload?.trim().orEmpty()
if (msg.isEmpty()) return
onMessage(msg)
}
companion object {
const val interfaceName: String = "clawdisCanvasA2UIAction"
}
}
private class CanvasA2UIActionLegacyBridge(private val bridge: CanvasA2UIActionBridge) {
@JavascriptInterface
fun canvasAction(payload: String?) {
bridge.postMessage(payload)
}
@JavascriptInterface
fun postMessage(payload: String?) {
bridge.postMessage(payload)
}
companion object {
const val interfaceName: String = "Android"
}
}

View File

@@ -0,0 +1,417 @@
package com.steipete.clawdis.node.ui
import android.Manifest
import android.content.pm.PackageManager
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.WindowInsets
import androidx.compose.foundation.layout.WindowInsetsSides
import androidx.compose.foundation.layout.fillMaxHeight
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.imePadding
import androidx.compose.foundation.layout.only
import androidx.compose.foundation.layout.safeDrawing
import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ExpandLess
import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material3.Button
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.ListItem
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.RadioButton
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import com.steipete.clawdis.node.MainViewModel
import com.steipete.clawdis.node.NodeForegroundService
import com.steipete.clawdis.node.VoiceWakeMode
@Composable
fun SettingsSheet(viewModel: MainViewModel) {
val context = LocalContext.current
val instanceId by viewModel.instanceId.collectAsState()
val displayName by viewModel.displayName.collectAsState()
val cameraEnabled by viewModel.cameraEnabled.collectAsState()
val preventSleep by viewModel.preventSleep.collectAsState()
val wakeWords by viewModel.wakeWords.collectAsState()
val voiceWakeMode by viewModel.voiceWakeMode.collectAsState()
val voiceWakeStatusText by viewModel.voiceWakeStatusText.collectAsState()
val isConnected by viewModel.isConnected.collectAsState()
val manualEnabled by viewModel.manualEnabled.collectAsState()
val manualHost by viewModel.manualHost.collectAsState()
val manualPort by viewModel.manualPort.collectAsState()
val canvasDebugStatusEnabled by viewModel.canvasDebugStatusEnabled.collectAsState()
val statusText by viewModel.statusText.collectAsState()
val serverName by viewModel.serverName.collectAsState()
val remoteAddress by viewModel.remoteAddress.collectAsState()
val bridges by viewModel.bridges.collectAsState()
val discoveryStatusText by viewModel.discoveryStatusText.collectAsState()
val listState = rememberLazyListState()
val (wakeWordsText, setWakeWordsText) = remember { mutableStateOf("") }
val (advancedExpanded, setAdvancedExpanded) = remember { mutableStateOf(false) }
LaunchedEffect(wakeWords) { setWakeWordsText(wakeWords.joinToString(", ")) }
val permissionLauncher =
rememberLauncherForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { perms ->
val cameraOk = perms[Manifest.permission.CAMERA] == true
viewModel.setCameraEnabled(cameraOk)
}
val audioPermissionLauncher =
rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { _ ->
// Status text is handled by NodeRuntime.
}
fun setCameraEnabledChecked(checked: Boolean) {
if (!checked) {
viewModel.setCameraEnabled(false)
return
}
val cameraOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.CAMERA) ==
PackageManager.PERMISSION_GRANTED
if (cameraOk) {
viewModel.setCameraEnabled(true)
} else {
permissionLauncher.launch(arrayOf(Manifest.permission.CAMERA, Manifest.permission.RECORD_AUDIO))
}
}
val visibleBridges =
if (isConnected && remoteAddress != null) {
bridges.filterNot { "${it.host}:${it.port}" == remoteAddress }
} else {
bridges
}
val bridgeDiscoveryFooterText =
if (visibleBridges.isEmpty()) {
discoveryStatusText
} else if (isConnected) {
"Discovery active • ${visibleBridges.size} other bridge${if (visibleBridges.size == 1) "" else "s"} found"
} else {
"Discovery active • ${visibleBridges.size} bridge${if (visibleBridges.size == 1) "" else "s"} found"
}
LazyColumn(
state = listState,
modifier =
Modifier
.fillMaxWidth()
.fillMaxHeight()
.imePadding()
.windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Bottom)),
contentPadding = PaddingValues(16.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
// Order parity: Node → Bridge → Voice → Camera → Screen.
item { Text("Node", style = MaterialTheme.typography.titleSmall) }
item {
OutlinedTextField(
value = displayName,
onValueChange = viewModel::setDisplayName,
label = { Text("Name") },
modifier = Modifier.fillMaxWidth(),
)
}
item { Text("Instance ID: $instanceId", color = MaterialTheme.colorScheme.onSurfaceVariant) }
item { HorizontalDivider() }
// Bridge
item { Text("Bridge", style = MaterialTheme.typography.titleSmall) }
item { ListItem(headlineContent = { Text("Status") }, supportingContent = { Text(statusText) }) }
if (serverName != null) {
item { ListItem(headlineContent = { Text("Server") }, supportingContent = { Text(serverName!!) }) }
}
if (remoteAddress != null) {
item { ListItem(headlineContent = { Text("Address") }, supportingContent = { Text(remoteAddress!!) }) }
}
item {
// UI sanity: "Disconnect" only when we have an active remote.
if (isConnected && remoteAddress != null) {
Button(
onClick = {
viewModel.disconnect()
NodeForegroundService.stop(context)
},
) {
Text("Disconnect")
}
}
}
item { HorizontalDivider() }
if (!isConnected || visibleBridges.isNotEmpty()) {
item {
Text(
if (isConnected) "Other Bridges" else "Discovered Bridges",
style = MaterialTheme.typography.titleSmall,
)
}
if (!isConnected && visibleBridges.isEmpty()) {
item { Text("No bridges found yet.", color = MaterialTheme.colorScheme.onSurfaceVariant) }
} else {
items(items = visibleBridges, key = { it.stableId }) { bridge ->
ListItem(
headlineContent = { Text(bridge.name) },
supportingContent = { Text("${bridge.host}:${bridge.port}") },
trailingContent = {
Button(
onClick = {
NodeForegroundService.start(context)
viewModel.connect(bridge)
},
) {
Text("Connect")
}
},
)
}
}
item {
Text(
bridgeDiscoveryFooterText,
modifier = Modifier.fillMaxWidth(),
textAlign = TextAlign.Center,
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
item { HorizontalDivider() }
item {
ListItem(
headlineContent = { Text("Advanced") },
supportingContent = { Text("Manual bridge connection") },
trailingContent = {
Icon(
imageVector = if (advancedExpanded) Icons.Filled.ExpandLess else Icons.Filled.ExpandMore,
contentDescription = if (advancedExpanded) "Collapse" else "Expand",
)
},
modifier =
Modifier.clickable {
setAdvancedExpanded(!advancedExpanded)
},
)
}
item {
AnimatedVisibility(visible = advancedExpanded) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp), modifier = Modifier.fillMaxWidth()) {
ListItem(
headlineContent = { Text("Use Manual Bridge") },
supportingContent = { Text("Use this when discovery is blocked.") },
trailingContent = { Switch(checked = manualEnabled, onCheckedChange = viewModel::setManualEnabled) },
)
OutlinedTextField(
value = manualHost,
onValueChange = viewModel::setManualHost,
label = { Text("Host") },
modifier = Modifier.fillMaxWidth(),
enabled = manualEnabled,
)
OutlinedTextField(
value = manualPort.toString(),
onValueChange = { v -> viewModel.setManualPort(v.toIntOrNull() ?: 0) },
label = { Text("Port") },
modifier = Modifier.fillMaxWidth(),
enabled = manualEnabled,
)
val hostOk = manualHost.trim().isNotEmpty()
val portOk = manualPort in 1..65535
Button(
onClick = {
NodeForegroundService.start(context)
viewModel.connectManual()
},
enabled = manualEnabled && hostOk && portOk,
) {
Text("Connect (Manual)")
}
}
}
}
item { HorizontalDivider() }
// Voice
item { Text("Voice", style = MaterialTheme.typography.titleSmall) }
item {
val enabled = voiceWakeMode != VoiceWakeMode.Off
ListItem(
headlineContent = { Text("Voice Wake") },
supportingContent = { Text(voiceWakeStatusText) },
trailingContent = {
Switch(
checked = enabled,
onCheckedChange = { on ->
if (on) {
val micOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
viewModel.setVoiceWakeMode(VoiceWakeMode.Foreground)
} else {
viewModel.setVoiceWakeMode(VoiceWakeMode.Off)
}
},
)
},
)
}
item {
AnimatedVisibility(visible = voiceWakeMode != VoiceWakeMode.Off) {
Column(verticalArrangement = Arrangement.spacedBy(6.dp), modifier = Modifier.fillMaxWidth()) {
ListItem(
headlineContent = { Text("Foreground Only") },
supportingContent = { Text("Listens only while Clawdis is open.") },
trailingContent = {
RadioButton(
selected = voiceWakeMode == VoiceWakeMode.Foreground,
onClick = {
val micOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
viewModel.setVoiceWakeMode(VoiceWakeMode.Foreground)
},
)
},
)
ListItem(
headlineContent = { Text("Always") },
supportingContent = { Text("Keeps listening in the background (shows a persistent notification).") },
trailingContent = {
RadioButton(
selected = voiceWakeMode == VoiceWakeMode.Always,
onClick = {
val micOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
viewModel.setVoiceWakeMode(VoiceWakeMode.Always)
},
)
},
)
}
}
}
item {
OutlinedTextField(
value = wakeWordsText,
onValueChange = setWakeWordsText,
label = { Text("Wake Words (comma-separated)") },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
)
}
item {
Row(horizontalArrangement = Arrangement.spacedBy(12.dp)) {
Button(
onClick = {
val parsed = com.steipete.clawdis.node.WakeWords.parseCommaSeparated(wakeWordsText)
viewModel.setWakeWords(parsed)
},
enabled = isConnected,
) {
Text("Save + Sync")
}
Button(onClick = viewModel::resetWakeWordsDefaults) { Text("Reset defaults") }
}
}
item {
Text(
if (isConnected) {
"Any node can edit wake words. Changes sync via the gateway bridge."
} else {
"Connect to a gateway to sync wake words globally."
},
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
item { HorizontalDivider() }
// Camera
item { Text("Camera", style = MaterialTheme.typography.titleSmall) }
item {
ListItem(
headlineContent = { Text("Allow Camera") },
supportingContent = { Text("Allows the bridge to request photos or short video clips (foreground only).") },
trailingContent = { Switch(checked = cameraEnabled, onCheckedChange = ::setCameraEnabledChecked) },
)
}
item {
Text(
"Tip: grant Microphone permission for video clips with audio.",
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
item { HorizontalDivider() }
// Screen
item { Text("Screen", style = MaterialTheme.typography.titleSmall) }
item {
ListItem(
headlineContent = { Text("Prevent Sleep") },
supportingContent = { Text("Keeps the screen awake while Clawdis is open.") },
trailingContent = { Switch(checked = preventSleep, onCheckedChange = viewModel::setPreventSleep) },
)
}
item { HorizontalDivider() }
// Debug
item { Text("Debug", style = MaterialTheme.typography.titleSmall) }
item {
ListItem(
headlineContent = { Text("Debug Canvas Status") },
supportingContent = { Text("Show status text in the canvas when debug is enabled.") },
trailingContent = {
Switch(
checked = canvasDebugStatusEnabled,
onCheckedChange = viewModel::setCanvasDebugStatusEnabled,
)
},
)
}
item { Spacer(modifier = Modifier.height(20.dp)) }
}
}

View File

@@ -0,0 +1,87 @@
package com.steipete.clawdis.node.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.MicOff
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.VerticalDivider
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.unit.dp
@Composable
fun StatusPill(
bridge: BridgeState,
voiceEnabled: Boolean,
onClick: () -> Unit,
modifier: Modifier = Modifier,
) {
Surface(
onClick = onClick,
modifier = modifier,
shape = RoundedCornerShape(14.dp),
color = overlayContainerColor(),
tonalElevation = 3.dp,
shadowElevation = 0.dp,
) {
Row(
modifier = Modifier.padding(horizontal = 12.dp, vertical = 8.dp),
horizontalArrangement = Arrangement.spacedBy(10.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Row(horizontalArrangement = Arrangement.spacedBy(8.dp), verticalAlignment = Alignment.CenterVertically) {
Surface(
modifier = Modifier.size(9.dp),
shape = CircleShape,
color = bridge.color,
) {}
Text(
text = bridge.title,
style = MaterialTheme.typography.labelLarge,
)
}
VerticalDivider(
modifier = Modifier.height(14.dp).alpha(0.35f),
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Icon(
imageVector = if (voiceEnabled) Icons.Default.Mic else Icons.Default.MicOff,
contentDescription = if (voiceEnabled) "Voice enabled" else "Voice disabled",
tint =
if (voiceEnabled) {
overlayIconColor()
} else {
MaterialTheme.colorScheme.onSurfaceVariant
},
modifier = Modifier.size(18.dp),
)
Spacer(modifier = Modifier.width(2.dp))
}
}
}
enum class BridgeState(val title: String, val color: Color) {
Connected("Connected", Color(0xFF2ECC71)),
Connecting("Connecting…", Color(0xFFF1C40F)),
Error("Error", Color(0xFFE74C3C)),
Disconnected("Offline", Color(0xFF9E9E9E)),
}

View File

@@ -0,0 +1,254 @@
package com.steipete.clawdis.node.ui.chat
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.horizontalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ArrowUpward
import androidx.compose.material.icons.filled.AttachFile
import androidx.compose.material.icons.filled.FolderOpen
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.FilledTonalButton
import androidx.compose.material3.FilledTonalIconButton
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.unit.dp
@Composable
fun ChatComposer(
sessionKey: String,
healthOk: Boolean,
thinkingLevel: String,
pendingRunCount: Int,
errorText: String?,
attachments: List<PendingImageAttachment>,
onPickImages: () -> Unit,
onRemoveAttachment: (id: String) -> Unit,
onSetThinkingLevel: (level: String) -> Unit,
onShowSessions: () -> Unit,
onRefresh: () -> Unit,
onAbort: () -> Unit,
onSend: (text: String) -> Unit,
) {
var input by rememberSaveable { mutableStateOf("") }
var showThinkingMenu by remember { mutableStateOf(false) }
val canSend = pendingRunCount == 0 && (input.trim().isNotEmpty() || attachments.isNotEmpty()) && healthOk
Surface(
shape = MaterialTheme.shapes.large,
color = MaterialTheme.colorScheme.surfaceContainer,
tonalElevation = 0.dp,
shadowElevation = 0.dp,
) {
Column(modifier = Modifier.padding(10.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box {
FilledTonalButton(
onClick = { showThinkingMenu = true },
contentPadding = ButtonDefaults.ContentPadding,
) {
Text("Thinking: ${thinkingLabel(thinkingLevel)}")
}
DropdownMenu(expanded = showThinkingMenu, onDismissRequest = { showThinkingMenu = false }) {
ThinkingMenuItem("off", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false }
ThinkingMenuItem("low", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false }
ThinkingMenuItem("medium", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false }
ThinkingMenuItem("high", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false }
}
}
Spacer(modifier = Modifier.weight(1f))
FilledTonalIconButton(onClick = onShowSessions, modifier = Modifier.size(42.dp)) {
Icon(Icons.Default.FolderOpen, contentDescription = "Sessions")
}
FilledTonalIconButton(onClick = onRefresh, modifier = Modifier.size(42.dp)) {
Icon(Icons.Default.Refresh, contentDescription = "Refresh")
}
FilledTonalIconButton(onClick = onPickImages, modifier = Modifier.size(42.dp)) {
Icon(Icons.Default.AttachFile, contentDescription = "Add image")
}
}
if (attachments.isNotEmpty()) {
AttachmentsStrip(attachments = attachments, onRemoveAttachment = onRemoveAttachment)
}
OutlinedTextField(
value = input,
onValueChange = { input = it },
modifier = Modifier.fillMaxWidth(),
placeholder = { Text("Message Clawd…") },
minLines = 2,
maxLines = 6,
)
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
ConnectionPill(sessionKey = sessionKey, healthOk = healthOk)
Spacer(modifier = Modifier.weight(1f))
if (pendingRunCount > 0) {
FilledTonalIconButton(
onClick = onAbort,
colors =
IconButtonDefaults.filledTonalIconButtonColors(
containerColor = Color(0x33E74C3C),
contentColor = Color(0xFFE74C3C),
),
) {
Icon(Icons.Default.Stop, contentDescription = "Abort")
}
} else {
FilledTonalIconButton(onClick = {
val text = input
input = ""
onSend(text)
}, enabled = canSend) {
Icon(Icons.Default.ArrowUpward, contentDescription = "Send")
}
}
}
if (!errorText.isNullOrBlank()) {
Text(
text = errorText,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.error,
maxLines = 2,
)
}
}
}
}
@Composable
private fun ConnectionPill(sessionKey: String, healthOk: Boolean) {
Surface(
shape = RoundedCornerShape(999.dp),
color = MaterialTheme.colorScheme.surfaceContainerHighest,
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 6.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Surface(
modifier = Modifier.size(7.dp),
shape = androidx.compose.foundation.shape.CircleShape,
color = if (healthOk) Color(0xFF2ECC71) else Color(0xFFF39C12),
) {}
Text(sessionKey, style = MaterialTheme.typography.labelSmall)
Text(
if (healthOk) "Connected" else "Connecting…",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
@Composable
private fun ThinkingMenuItem(
value: String,
current: String,
onSet: (String) -> Unit,
onDismiss: () -> Unit,
) {
DropdownMenuItem(
text = { Text(thinkingLabel(value)) },
onClick = {
onSet(value)
onDismiss()
},
trailingIcon = {
if (value == current.trim().lowercase()) {
Text("")
} else {
Spacer(modifier = Modifier.width(10.dp))
}
},
)
}
private fun thinkingLabel(raw: String): String {
return when (raw.trim().lowercase()) {
"low" -> "Low"
"medium" -> "Medium"
"high" -> "High"
else -> "Off"
}
}
@Composable
private fun AttachmentsStrip(
attachments: List<PendingImageAttachment>,
onRemoveAttachment: (id: String) -> Unit,
) {
Row(
modifier = Modifier.fillMaxWidth().horizontalScroll(rememberScrollState()),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
for (att in attachments) {
AttachmentChip(
fileName = att.fileName,
onRemove = { onRemoveAttachment(att.id) },
)
}
}
}
@Composable
private fun AttachmentChip(fileName: String, onRemove: () -> Unit) {
Surface(
shape = RoundedCornerShape(999.dp),
color = MaterialTheme.colorScheme.primary.copy(alpha = 0.10f),
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(text = fileName, style = MaterialTheme.typography.bodySmall, maxLines = 1)
FilledTonalIconButton(
onClick = onRemove,
modifier = Modifier.size(30.dp),
) {
Text("×")
}
}
}
}

View File

@@ -0,0 +1,214 @@
package com.steipete.clawdis.node.ui.chat
import android.graphics.BitmapFactory
import android.util.Base64
import androidx.compose.foundation.Image
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.text.AnnotatedString
import androidx.compose.ui.text.SpanStyle
import androidx.compose.ui.text.buildAnnotatedString
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontStyle
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.withStyle
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@Composable
fun ChatMarkdown(text: String) {
val blocks = remember(text) { splitMarkdown(text) }
val inlineCodeBg = MaterialTheme.colorScheme.surfaceContainerLow
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
for (b in blocks) {
when (b) {
is ChatMarkdownBlock.Text -> {
val trimmed = b.text.trimEnd()
if (trimmed.isEmpty()) continue
Text(
text = parseInlineMarkdown(trimmed, inlineCodeBg = inlineCodeBg),
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
)
}
is ChatMarkdownBlock.Code -> {
SelectionContainer(modifier = Modifier.fillMaxWidth()) {
ChatCodeBlock(code = b.code, language = b.language)
}
}
is ChatMarkdownBlock.InlineImage -> {
InlineBase64Image(base64 = b.base64, mimeType = b.mimeType)
}
}
}
}
}
private sealed interface ChatMarkdownBlock {
data class Text(val text: String) : ChatMarkdownBlock
data class Code(val code: String, val language: String?) : ChatMarkdownBlock
data class InlineImage(val mimeType: String?, val base64: String) : ChatMarkdownBlock
}
private fun splitMarkdown(raw: String): List<ChatMarkdownBlock> {
if (raw.isEmpty()) return emptyList()
val out = ArrayList<ChatMarkdownBlock>()
var idx = 0
while (idx < raw.length) {
val fenceStart = raw.indexOf("```", startIndex = idx)
if (fenceStart < 0) {
out.addAll(splitInlineImages(raw.substring(idx)))
break
}
if (fenceStart > idx) {
out.addAll(splitInlineImages(raw.substring(idx, fenceStart)))
}
val langLineStart = fenceStart + 3
val langLineEnd = raw.indexOf('\n', startIndex = langLineStart).let { if (it < 0) raw.length else it }
val language = raw.substring(langLineStart, langLineEnd).trim().ifEmpty { null }
val codeStart = if (langLineEnd < raw.length && raw[langLineEnd] == '\n') langLineEnd + 1 else langLineEnd
val fenceEnd = raw.indexOf("```", startIndex = codeStart)
if (fenceEnd < 0) {
out.addAll(splitInlineImages(raw.substring(fenceStart)))
break
}
val code = raw.substring(codeStart, fenceEnd)
out.add(ChatMarkdownBlock.Code(code = code, language = language))
idx = fenceEnd + 3
}
return out
}
private fun splitInlineImages(text: String): List<ChatMarkdownBlock> {
if (text.isEmpty()) return emptyList()
val regex = Regex("data:image/([a-zA-Z0-9+.-]+);base64,([A-Za-z0-9+/=\\n\\r]+)")
val out = ArrayList<ChatMarkdownBlock>()
var idx = 0
while (idx < text.length) {
val m = regex.find(text, startIndex = idx) ?: break
val start = m.range.first
val end = m.range.last + 1
if (start > idx) out.add(ChatMarkdownBlock.Text(text.substring(idx, start)))
val mime = "image/" + (m.groupValues.getOrNull(1)?.trim()?.ifEmpty { "png" } ?: "png")
val b64 = m.groupValues.getOrNull(2)?.replace("\n", "")?.replace("\r", "")?.trim().orEmpty()
if (b64.isNotEmpty()) {
out.add(ChatMarkdownBlock.InlineImage(mimeType = mime, base64 = b64))
}
idx = end
}
if (idx < text.length) out.add(ChatMarkdownBlock.Text(text.substring(idx)))
return out
}
private fun parseInlineMarkdown(text: String, inlineCodeBg: androidx.compose.ui.graphics.Color): AnnotatedString {
if (text.isEmpty()) return AnnotatedString("")
val out = buildAnnotatedString {
var i = 0
while (i < text.length) {
if (text.startsWith("**", startIndex = i)) {
val end = text.indexOf("**", startIndex = i + 2)
if (end > i + 2) {
withStyle(SpanStyle(fontWeight = FontWeight.SemiBold)) {
append(text.substring(i + 2, end))
}
i = end + 2
continue
}
}
if (text[i] == '`') {
val end = text.indexOf('`', startIndex = i + 1)
if (end > i + 1) {
withStyle(
SpanStyle(
fontFamily = FontFamily.Monospace,
background = inlineCodeBg,
),
) {
append(text.substring(i + 1, end))
}
i = end + 1
continue
}
}
if (text[i] == '*' && (i + 1 < text.length && text[i + 1] != '*')) {
val end = text.indexOf('*', startIndex = i + 1)
if (end > i + 1) {
withStyle(SpanStyle(fontStyle = FontStyle.Italic)) {
append(text.substring(i + 1, end))
}
i = end + 1
continue
}
}
append(text[i])
i += 1
}
}
return out
}
@Composable
private fun InlineBase64Image(base64: String, mimeType: String?) {
var image by remember(base64) { mutableStateOf<androidx.compose.ui.graphics.ImageBitmap?>(null) }
var failed by remember(base64) { mutableStateOf(false) }
LaunchedEffect(base64) {
failed = false
image =
withContext(Dispatchers.Default) {
try {
val bytes = Base64.decode(base64, Base64.DEFAULT)
val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size) ?: return@withContext null
bitmap.asImageBitmap()
} catch (_: Throwable) {
null
}
}
if (image == null) failed = true
}
if (image != null) {
Image(
bitmap = image!!,
contentDescription = mimeType ?: "image",
contentScale = ContentScale.Fit,
modifier = Modifier.fillMaxWidth(),
)
} else if (failed) {
Text(
text = "Image unavailable",
modifier = Modifier.padding(vertical = 2.dp),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}

View File

@@ -0,0 +1,111 @@
package com.steipete.clawdis.node.ui.chat
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ArrowCircleDown
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.chat.ChatMessage
import com.steipete.clawdis.node.chat.ChatPendingToolCall
@Composable
fun ChatMessageListCard(
messages: List<ChatMessage>,
pendingRunCount: Int,
pendingToolCalls: List<ChatPendingToolCall>,
streamingAssistantText: String?,
modifier: Modifier = Modifier,
) {
val listState = rememberLazyListState()
LaunchedEffect(messages.size, pendingRunCount, pendingToolCalls.size, streamingAssistantText) {
val total =
messages.size +
(if (pendingRunCount > 0) 1 else 0) +
(if (pendingToolCalls.isNotEmpty()) 1 else 0) +
(if (!streamingAssistantText.isNullOrBlank()) 1 else 0)
if (total <= 0) return@LaunchedEffect
listState.animateScrollToItem(index = total - 1)
}
Card(
modifier = modifier.fillMaxWidth(),
shape = MaterialTheme.shapes.large,
colors =
CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceContainer,
),
elevation = CardDefaults.cardElevation(defaultElevation = 0.dp),
) {
Box(modifier = Modifier.fillMaxSize()) {
LazyColumn(
modifier = Modifier.fillMaxSize(),
state = listState,
verticalArrangement = Arrangement.spacedBy(14.dp),
contentPadding = androidx.compose.foundation.layout.PaddingValues(top = 12.dp, bottom = 12.dp, start = 12.dp, end = 12.dp),
) {
items(count = messages.size, key = { idx -> messages[idx].id }) { idx ->
ChatMessageBubble(message = messages[idx])
}
if (pendingRunCount > 0) {
item(key = "typing") {
ChatTypingIndicatorBubble()
}
}
if (pendingToolCalls.isNotEmpty()) {
item(key = "tools") {
ChatPendingToolsBubble(toolCalls = pendingToolCalls)
}
}
val stream = streamingAssistantText?.trim()
if (!stream.isNullOrEmpty()) {
item(key = "stream") {
ChatStreamingAssistantBubble(text = stream)
}
}
}
if (messages.isEmpty() && pendingRunCount == 0 && pendingToolCalls.isEmpty() && streamingAssistantText.isNullOrBlank()) {
EmptyChatHint(modifier = Modifier.align(Alignment.Center))
}
}
}
}
@Composable
private fun EmptyChatHint(modifier: Modifier = Modifier) {
Row(
modifier = modifier.alpha(0.7f),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Icon(
imageVector = Icons.Default.ArrowCircleDown,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
Text(
text = "Message Clawd…",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}

View File

@@ -0,0 +1,218 @@
package com.steipete.clawdis.node.ui.chat
import android.graphics.BitmapFactory
import android.util.Base64
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.compose.foundation.Image
import com.steipete.clawdis.node.chat.ChatMessage
import com.steipete.clawdis.node.chat.ChatMessageContent
import com.steipete.clawdis.node.chat.ChatPendingToolCall
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@Composable
fun ChatMessageBubble(message: ChatMessage) {
val isUser = message.role.lowercase() == "user"
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = if (isUser) Arrangement.End else Arrangement.Start,
) {
Surface(
shape = RoundedCornerShape(16.dp),
tonalElevation = 0.dp,
shadowElevation = 0.dp,
color = Color.Transparent,
modifier = Modifier.fillMaxWidth(0.92f),
) {
Box(
modifier =
Modifier
.background(bubbleBackground(isUser))
.padding(horizontal = 12.dp, vertical = 10.dp),
) {
ChatMessageBody(content = message.content)
}
}
}
}
@Composable
private fun ChatMessageBody(content: List<ChatMessageContent>) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
for (part in content) {
when (part.type) {
"text" -> {
val text = part.text ?: continue
ChatMarkdown(text = text)
}
else -> {
val b64 = part.base64 ?: continue
ChatBase64Image(base64 = b64, mimeType = part.mimeType)
}
}
}
}
}
@Composable
fun ChatTypingIndicatorBubble() {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) {
Surface(
shape = RoundedCornerShape(16.dp),
color = MaterialTheme.colorScheme.surfaceContainer,
) {
Row(
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
DotPulse()
Text("Thinking…", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
}
}
@Composable
fun ChatPendingToolsBubble(toolCalls: List<ChatPendingToolCall>) {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) {
Surface(
shape = RoundedCornerShape(16.dp),
color = MaterialTheme.colorScheme.surfaceContainer,
) {
Column(modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text("Tools", style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurface)
for (t in toolCalls.take(6)) {
Text("· ${t.name}", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
if (toolCalls.size > 6) {
Text("… +${toolCalls.size - 6} more", style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
}
}
}
@Composable
fun ChatStreamingAssistantBubble(text: String) {
Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) {
Surface(
shape = RoundedCornerShape(16.dp),
color = MaterialTheme.colorScheme.surfaceContainer,
) {
Box(modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp)) {
ChatMarkdown(text = text)
}
}
}
}
@Composable
private fun bubbleBackground(isUser: Boolean): Brush {
return if (isUser) {
Brush.linearGradient(
colors = listOf(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.primary.copy(alpha = 0.78f)),
)
} else {
Brush.linearGradient(
colors = listOf(MaterialTheme.colorScheme.surfaceContainer, MaterialTheme.colorScheme.surfaceContainerHigh),
)
}
}
@Composable
private fun ChatBase64Image(base64: String, mimeType: String?) {
var image by remember(base64) { mutableStateOf<androidx.compose.ui.graphics.ImageBitmap?>(null) }
var failed by remember(base64) { mutableStateOf(false) }
LaunchedEffect(base64) {
failed = false
image =
withContext(Dispatchers.Default) {
try {
val bytes = Base64.decode(base64, Base64.DEFAULT)
val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size) ?: return@withContext null
bitmap.asImageBitmap()
} catch (_: Throwable) {
null
}
}
if (image == null) failed = true
}
if (image != null) {
Image(
bitmap = image!!,
contentDescription = mimeType ?: "attachment",
contentScale = ContentScale.Fit,
modifier = Modifier.fillMaxWidth(),
)
} else if (failed) {
Text("Unsupported attachment", style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
@Composable
private fun DotPulse() {
Row(horizontalArrangement = Arrangement.spacedBy(5.dp), verticalAlignment = Alignment.CenterVertically) {
PulseDot(alpha = 0.38f)
PulseDot(alpha = 0.62f)
PulseDot(alpha = 0.90f)
}
}
@Composable
private fun PulseDot(alpha: Float) {
Surface(
modifier = Modifier.size(6.dp).alpha(alpha),
shape = CircleShape,
color = MaterialTheme.colorScheme.onSurfaceVariant,
) {}
}
@Composable
fun ChatCodeBlock(code: String, language: String?) {
Surface(
shape = RoundedCornerShape(12.dp),
color = MaterialTheme.colorScheme.surfaceContainerLowest,
modifier = Modifier.fillMaxWidth(),
) {
Text(
text = code.trimEnd(),
modifier = Modifier.padding(10.dp),
fontFamily = FontFamily.Monospace,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurface,
)
}
}

View File

@@ -0,0 +1,93 @@
package com.steipete.clawdis.node.ui.chat
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.FilledTonalIconButton
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.chat.ChatSessionEntry
@Composable
fun ChatSessionsDialog(
currentSessionKey: String,
sessions: List<ChatSessionEntry>,
onDismiss: () -> Unit,
onRefresh: () -> Unit,
onSelect: (sessionKey: String) -> Unit,
) {
AlertDialog(
onDismissRequest = onDismiss,
confirmButton = {},
title = {
Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.fillMaxWidth()) {
Text("Sessions", style = MaterialTheme.typography.titleMedium)
Spacer(modifier = Modifier.weight(1f))
FilledTonalIconButton(onClick = onRefresh) {
Icon(Icons.Default.Refresh, contentDescription = "Refresh")
}
}
},
text = {
if (sessions.isEmpty()) {
Text("No sessions", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
} else {
LazyColumn(verticalArrangement = Arrangement.spacedBy(8.dp)) {
items(sessions, key = { it.key }) { entry ->
SessionRow(
entry = entry,
isCurrent = entry.key == currentSessionKey,
onClick = { onSelect(entry.key) },
)
}
}
}
},
)
}
@Composable
private fun SessionRow(
entry: ChatSessionEntry,
isCurrent: Boolean,
onClick: () -> Unit,
) {
Surface(
onClick = onClick,
shape = MaterialTheme.shapes.medium,
color =
if (isCurrent) {
MaterialTheme.colorScheme.primary.copy(alpha = 0.14f)
} else {
MaterialTheme.colorScheme.surfaceContainer
},
modifier = Modifier.fillMaxWidth(),
) {
Row(
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp),
) {
Text(entry.key, style = MaterialTheme.typography.bodyMedium)
Spacer(modifier = Modifier.weight(1f))
if (isCurrent) {
Text("Current", style = MaterialTheme.typography.labelSmall, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
}
}

View File

@@ -0,0 +1,157 @@
package com.steipete.clawdis.node.ui.chat
import android.content.ContentResolver
import android.net.Uri
import android.util.Base64
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateListOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.MainViewModel
import com.steipete.clawdis.node.chat.OutgoingAttachment
import java.io.ByteArrayOutputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
@Composable
fun ChatSheetContent(viewModel: MainViewModel) {
val messages by viewModel.chatMessages.collectAsState()
val errorText by viewModel.chatError.collectAsState()
val pendingRunCount by viewModel.pendingRunCount.collectAsState()
val healthOk by viewModel.chatHealthOk.collectAsState()
val sessionKey by viewModel.chatSessionKey.collectAsState()
val thinkingLevel by viewModel.chatThinkingLevel.collectAsState()
val streamingAssistantText by viewModel.chatStreamingAssistantText.collectAsState()
val pendingToolCalls by viewModel.chatPendingToolCalls.collectAsState()
val sessions by viewModel.chatSessions.collectAsState()
var showSessions by remember { mutableStateOf(false) }
LaunchedEffect(Unit) {
viewModel.loadChat("main")
}
val context = LocalContext.current
val resolver = context.contentResolver
val scope = rememberCoroutineScope()
val attachments = remember { mutableStateListOf<PendingImageAttachment>() }
val pickImages =
rememberLauncherForActivityResult(ActivityResultContracts.GetMultipleContents()) { uris ->
if (uris.isNullOrEmpty()) return@rememberLauncherForActivityResult
scope.launch(Dispatchers.IO) {
val next =
uris.take(8).mapNotNull { uri ->
try {
loadImageAttachment(resolver, uri)
} catch (_: Throwable) {
null
}
}
withContext(Dispatchers.Main) {
attachments.addAll(next)
}
}
}
Column(
modifier =
Modifier
.fillMaxSize()
.padding(horizontal = 12.dp, vertical = 12.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
ChatMessageListCard(
messages = messages,
pendingRunCount = pendingRunCount,
pendingToolCalls = pendingToolCalls,
streamingAssistantText = streamingAssistantText,
modifier = Modifier.weight(1f, fill = true),
)
ChatComposer(
sessionKey = sessionKey,
healthOk = healthOk,
thinkingLevel = thinkingLevel,
pendingRunCount = pendingRunCount,
errorText = errorText,
attachments = attachments,
onPickImages = { pickImages.launch("image/*") },
onRemoveAttachment = { id -> attachments.removeAll { it.id == id } },
onSetThinkingLevel = { level -> viewModel.setChatThinkingLevel(level) },
onShowSessions = { showSessions = true },
onRefresh = { viewModel.refreshChat() },
onAbort = { viewModel.abortChat() },
onSend = { text ->
val outgoing =
attachments.map { att ->
OutgoingAttachment(
type = "image",
mimeType = att.mimeType,
fileName = att.fileName,
base64 = att.base64,
)
}
viewModel.sendChat(message = text, thinking = thinkingLevel, attachments = outgoing)
attachments.clear()
},
)
}
if (showSessions) {
ChatSessionsDialog(
currentSessionKey = sessionKey,
sessions = sessions,
onDismiss = { showSessions = false },
onRefresh = { viewModel.refreshChatSessions(limit = 50) },
onSelect = { key ->
viewModel.switchChatSession(key)
showSessions = false
},
)
}
}
data class PendingImageAttachment(
val id: String,
val fileName: String,
val mimeType: String,
val base64: String,
)
private suspend fun loadImageAttachment(resolver: ContentResolver, uri: Uri): PendingImageAttachment {
val mimeType = resolver.getType(uri) ?: "image/*"
val fileName = (uri.lastPathSegment ?: "image").substringAfterLast('/')
val bytes =
withContext(Dispatchers.IO) {
resolver.openInputStream(uri)?.use { input ->
val out = ByteArrayOutputStream()
input.copyTo(out)
out.toByteArray()
} ?: ByteArray(0)
}
if (bytes.isEmpty()) throw IllegalStateException("empty attachment")
val base64 = Base64.encodeToString(bytes, Base64.NO_WRAP)
return PendingImageAttachment(
id = uri.toString() + "#" + System.currentTimeMillis().toString(),
fileName = fileName,
mimeType = mimeType,
base64 = base64,
)
}

View File

@@ -0,0 +1,40 @@
package com.steipete.clawdis.node.voice
object VoiceWakeCommandExtractor {
fun extractCommand(text: String, triggerWords: List<String>): String? {
val raw = text.trim()
if (raw.isEmpty()) return null
val triggers =
triggerWords
.map { it.trim().lowercase() }
.filter { it.isNotEmpty() }
.distinct()
if (triggers.isEmpty()) return null
val alternation = triggers.joinToString("|") { Regex.escape(it) }
// Match: "<anything> <trigger><punct/space> <command>"
val regex = Regex("(?i)(?:^|\\s)($alternation)\\b[\\s\\p{Punct}]*([\\s\\S]+)$")
val match = regex.find(raw) ?: return null
val extracted = match.groupValues.getOrNull(2)?.trim().orEmpty()
if (extracted.isEmpty()) return null
val cleaned = extracted.trimStart { it.isWhitespace() || it.isPunctuation() }.trim()
if (cleaned.isEmpty()) return null
return cleaned
}
}
private fun Char.isPunctuation(): Boolean {
return when (Character.getType(this)) {
Character.CONNECTOR_PUNCTUATION.toInt(),
Character.DASH_PUNCTUATION.toInt(),
Character.START_PUNCTUATION.toInt(),
Character.END_PUNCTUATION.toInt(),
Character.INITIAL_QUOTE_PUNCTUATION.toInt(),
Character.FINAL_QUOTE_PUNCTUATION.toInt(),
Character.OTHER_PUNCTUATION.toInt(),
-> true
else -> false
}
}

View File

@@ -0,0 +1,173 @@
package com.steipete.clawdis.node.voice
import android.content.Context
import android.content.Intent
import android.os.Bundle
import android.os.Handler
import android.os.Looper
import android.speech.RecognitionListener
import android.speech.RecognizerIntent
import android.speech.SpeechRecognizer
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
class VoiceWakeManager(
private val context: Context,
private val scope: CoroutineScope,
private val onCommand: suspend (String) -> Unit,
) {
private val mainHandler = Handler(Looper.getMainLooper())
private val _isListening = MutableStateFlow(false)
val isListening: StateFlow<Boolean> = _isListening
private val _statusText = MutableStateFlow("Off")
val statusText: StateFlow<String> = _statusText
var triggerWords: List<String> = emptyList()
private set
private var recognizer: SpeechRecognizer? = null
private var restartJob: Job? = null
private var lastDispatched: String? = null
private var stopRequested = false
fun setTriggerWords(words: List<String>) {
triggerWords = words
}
fun start() {
mainHandler.post {
if (_isListening.value) return@post
stopRequested = false
if (!SpeechRecognizer.isRecognitionAvailable(context)) {
_isListening.value = false
_statusText.value = "Speech recognizer unavailable"
return@post
}
try {
recognizer?.destroy()
recognizer = SpeechRecognizer.createSpeechRecognizer(context).also { it.setRecognitionListener(listener) }
startListeningInternal()
} catch (err: Throwable) {
_isListening.value = false
_statusText.value = "Start failed: ${err.message ?: err::class.simpleName}"
}
}
}
fun stop(statusText: String = "Off") {
stopRequested = true
restartJob?.cancel()
restartJob = null
mainHandler.post {
_isListening.value = false
_statusText.value = statusText
recognizer?.cancel()
recognizer?.destroy()
recognizer = null
}
}
private fun startListeningInternal() {
val r = recognizer ?: return
val intent =
Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH).apply {
putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM)
putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true)
putExtra(RecognizerIntent.EXTRA_MAX_RESULTS, 3)
putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, context.packageName)
}
_statusText.value = "Listening"
_isListening.value = true
r.startListening(intent)
}
private fun scheduleRestart(delayMs: Long = 350) {
if (stopRequested) return
restartJob?.cancel()
restartJob =
scope.launch {
delay(delayMs)
mainHandler.post {
if (stopRequested) return@post
try {
recognizer?.cancel()
startListeningInternal()
} catch (_: Throwable) {
// Will be picked up by onError and retry again.
}
}
}
}
private fun handleTranscription(text: String) {
val command = VoiceWakeCommandExtractor.extractCommand(text, triggerWords) ?: return
if (command == lastDispatched) return
lastDispatched = command
scope.launch { onCommand(command) }
_statusText.value = "Triggered"
scheduleRestart(delayMs = 650)
}
private val listener =
object : RecognitionListener {
override fun onReadyForSpeech(params: Bundle?) {
_statusText.value = "Listening"
}
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(rmsdB: Float) {}
override fun onBufferReceived(buffer: ByteArray?) {}
override fun onEndOfSpeech() {
scheduleRestart()
}
override fun onError(error: Int) {
if (stopRequested) return
_isListening.value = false
if (error == SpeechRecognizer.ERROR_INSUFFICIENT_PERMISSIONS) {
_statusText.value = "Microphone permission required"
return
}
_statusText.value =
when (error) {
SpeechRecognizer.ERROR_AUDIO -> "Audio error"
SpeechRecognizer.ERROR_CLIENT -> "Client error"
SpeechRecognizer.ERROR_NETWORK -> "Network error"
SpeechRecognizer.ERROR_NETWORK_TIMEOUT -> "Network timeout"
SpeechRecognizer.ERROR_NO_MATCH -> "Listening"
SpeechRecognizer.ERROR_RECOGNIZER_BUSY -> "Recognizer busy"
SpeechRecognizer.ERROR_SERVER -> "Server error"
SpeechRecognizer.ERROR_SPEECH_TIMEOUT -> "Listening"
else -> "Speech error ($error)"
}
scheduleRestart(delayMs = 600)
}
override fun onResults(results: Bundle?) {
val list = results?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION).orEmpty()
list.firstOrNull()?.let(::handleTranscription)
scheduleRestart()
}
override fun onPartialResults(partialResults: Bundle?) {
val list = partialResults?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION).orEmpty()
list.firstOrNull()?.let(::handleTranscription)
}
override fun onEvent(eventType: Int, params: Bundle?) {}
}
}

View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@color/ic_launcher_background" />
<foreground android:drawable="@mipmap/ic_launcher_foreground" />
<monochrome android:drawable="@mipmap/ic_launcher_foreground" />
</adaptive-icon>

View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@color/ic_launcher_background" />
<foreground android:drawable="@mipmap/ic_launcher_foreground" />
<monochrome android:drawable="@mipmap/ic_launcher_foreground" />
</adaptive-icon>

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

Some files were not shown because too many files have changed in this diff Show More