Compare commits
10 Commits
main
...
secrets/pr
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ca8a0dee4b | ||
|
|
43748ea2ef | ||
|
|
ed402b6944 | ||
|
|
6eb674d6d0 | ||
|
|
16b7d4bbed | ||
|
|
209287d5e0 | ||
|
|
bb052deec6 | ||
|
|
06e51505bc | ||
|
|
2b7206470a | ||
|
|
7103ba3758 |
2
.gitignore
vendored
2
.gitignore
vendored
@@ -99,5 +99,3 @@ package-lock.json
|
||||
|
||||
# Local iOS signing overrides
|
||||
apps/ios/LocalSigning.xcconfig
|
||||
# Generated protocol schema (produced via pnpm protocol:gen)
|
||||
dist/protocol.schema.json
|
||||
|
||||
17
CHANGELOG.md
17
CHANGELOG.md
@@ -8,7 +8,6 @@ Docs: https://docs.openclaw.ai
|
||||
|
||||
- Channels/Config: unify channel preview streaming config handling with a shared resolver and canonical migration path.
|
||||
- Discord/Allowlist: canonicalize resolved Discord allowlist names to IDs and split resolution flow for clearer fail-closed behavior.
|
||||
- Memory/FTS: add Korean stop-word filtering and particle-aware keyword extraction (including mixed Korean/English stems) for query expansion in FTS-only search mode. (#18899) Thanks @ruypang.
|
||||
- iOS/Talk: prefetch TTS segments and suppress expected speech-cancellation errors for smoother talk playback. (#22833) Thanks @ngutman.
|
||||
|
||||
### Breaking
|
||||
@@ -17,22 +16,7 @@ Docs: https://docs.openclaw.ai
|
||||
|
||||
### Fixes
|
||||
|
||||
- Slack/Slash commands: preserve the Bolt app receiver when registering external select options handlers so monitor startup does not crash on runtimes that require bound `app.options` calls. (#23209) Thanks @0xgaia.
|
||||
- Agents/Ollama: preserve unsafe integer tool-call arguments as exact strings during NDJSON parsing, preventing large numeric IDs from being rounded before tool execution. (#23170) Thanks @BestJoester.
|
||||
- Cron/Gateway: keep `cron.list` and `cron.status` responsive during startup catch-up by avoiding a long-held cron lock while missed jobs execute. (#23106) Thanks @jayleekr.
|
||||
- Gateway/Config reload: compare array-valued config paths structurally during diffing so unchanged `memory.qmd.paths` and `memory.qmd.scope.rules` no longer trigger false restart-required reloads. (#23185) Thanks @rex05ai.
|
||||
- Cron/Scheduling: validate runtime cron expressions before schedule/stagger evaluation so malformed persisted jobs report a clear `invalid cron schedule: expr is required` error instead of crashing with `undefined.trim` failures and auto-disable churn. (#23223) Thanks @asimons81.
|
||||
- TUI/Input: enable multiline-paste burst coalescing on macOS Terminal.app and iTerm so pasted blocks no longer submit line-by-line as separate messages. (#18809) Thanks @fwends.
|
||||
- TUI/Status: request immediate renders after setting `sending`/`waiting` activity states so in-flight runs always show visible progress indicators instead of appearing idle until completion. (#21549) Thanks @13Guinness.
|
||||
- Agents/Fallbacks: treat JSON payloads with `type: "api_error"` + `"Internal server error"` as transient failover errors so Anthropic 500-style failures trigger model fallback. (#23193) Thanks @jarvis-lane.
|
||||
- Agents/Diagnostics: include resolved lifecycle error text in `embedded run agent end` warnings so UI/TUI “Connection error” runs expose actionable provider failure reasons in gateway logs. (#23054) Thanks @Raize.
|
||||
- Gateway/Pairing: treat operator.admin pairing tokens as satisfying operator.write requests so legacy devices stop looping through scope-upgrade prompts introduced in 2026.2.19. (#23125, #23006) Thanks @vignesh07.
|
||||
- Gateway/Pairing: treat `operator.admin` as satisfying other `operator.*` scope checks during device-auth verification so local CLI/TUI sessions stop entering pairing-required loops for pairing/approval-scoped commands. (#22062, #22193, #21191) Thanks @Botaccess, @jhartshorn, and @ctbritt.
|
||||
- Gateway/Pairing: preserve existing approved token scopes when processing repair pairings that omit `scopes`, preventing empty-scope token regressions on reconnecting clients. (#21906) Thanks @paki81.
|
||||
- Plugins/CLI: make `openclaw plugins enable` and plugin install/link flows update allowlists via shared plugin-enable policy so enabled plugins are not left disabled by allowlist mismatch. (#23190) Thanks @downwind7clawd-ctrl.
|
||||
- Memory/QMD: add optional `memory.qmd.mcporter` search routing so QMD `query/search/vsearch` can run through mcporter keep-alive flows (including multi-collection paths) to reduce cold starts, while keeping searches on agent-scoped QMD state for consistent recall. (#19617) Thanks @nicole-luxe and @vignesh07.
|
||||
- Chat/UI: strip inline reply/audio directive tags (`[[reply_to_current]]`, `[[reply_to:<id>]]`, `[[audio_as_voice]]`) from displayed chat history, live chat event output, and session preview snippets so control tags no longer leak into user-visible surfaces.
|
||||
- BlueBubbles/DM history: restore DM backfill context with account-scoped rolling history, bounded backfill retries, and safer history payload limits. (#20302) Thanks @Ryan-Haines.
|
||||
- Security/Config: block prototype-key traversal during config merge patch and legacy migration merge helpers (`__proto__`, `constructor`, `prototype`) to prevent prototype pollution during config mutation flows. (#22968) Thanks @Clawborn.
|
||||
- Security/Shell env: validate login-shell executable paths for shell-env fallback (`/etc/shells` + trusted prefixes) and block `SHELL` in dangerous env override policy paths so untrusted shell-path injection falls back safely to `/bin/sh`. Thanks @athuljayaram for reporting.
|
||||
- Security/Config: make parsed chat allowlist checks fail closed when `allowFrom` is empty, restoring expected DM/pairing gating.
|
||||
@@ -106,7 +90,6 @@ Docs: https://docs.openclaw.ai
|
||||
- CLI/Pairing: default `pairing list` and `pairing approve` to the sole available pairing channel when omitted, so TUI-only setups can recover from `pairing required` without guessing channel arguments. (#21527) Thanks @losts1.
|
||||
- TUI/Pairing: show explicit pairing-required recovery guidance after gateway disconnects that return `pairing required`, including approval steps to unblock quickstart TUI hatching on fresh installs. (#21841) Thanks @nicolinux.
|
||||
- TUI/Input: suppress duplicate backspace events arriving in the same input burst window so SSH sessions no longer delete two characters per backspace press in the composer. (#19318) Thanks @eheimer.
|
||||
- TUI/Models: scope `models.list` to the configured model allowlist (`agents.defaults.models`) so `/model` picker no longer floods with unrelated catalog entries by default. (#18816) Thanks @fwends.
|
||||
- TUI/Heartbeat: suppress heartbeat ACK/prompt noise in chat streaming when `showOk` is disabled, while still preserving non-ACK heartbeat alerts in final output. (#20228) Thanks @bhalliburton.
|
||||
- TUI/History: cap chat-log component growth and prune stale render nodes/references so large default history loads no longer overflow render recursion with `RangeError: Maximum call stack size exceeded`. (#18068) Thanks @JaniJegoroff.
|
||||
- Memory/QMD: diversify mixed-source search ranking when both session and memory collections are present so session transcript hits no longer crowd out durable memory-file matches in top results. (#19913) Thanks @alextempr.
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
curl -fsSL https://git.yalu.pro/evgeny/openclaw/raw/branch/main/install.sh | bash
|
||||
|
||||
|
||||
# 🦞 OpenClaw — Personal AI Assistant
|
||||
|
||||
<p align="center">
|
||||
|
||||
@@ -1,177 +0,0 @@
|
||||
import type { OpenClawConfig } from "openclaw/plugin-sdk";
|
||||
import { resolveBlueBubblesServerAccount } from "./account-resolve.js";
|
||||
import { blueBubblesFetchWithTimeout, buildBlueBubblesApiUrl } from "./types.js";
|
||||
|
||||
export type BlueBubblesHistoryEntry = {
|
||||
sender: string;
|
||||
body: string;
|
||||
timestamp?: number;
|
||||
messageId?: string;
|
||||
};
|
||||
|
||||
export type BlueBubblesHistoryFetchResult = {
|
||||
entries: BlueBubblesHistoryEntry[];
|
||||
/**
|
||||
* True when at least one API path returned a recognized response shape.
|
||||
* False means all attempts failed or returned unusable data.
|
||||
*/
|
||||
resolved: boolean;
|
||||
};
|
||||
|
||||
export type BlueBubblesMessageData = {
|
||||
guid?: string;
|
||||
text?: string;
|
||||
handle_id?: string;
|
||||
is_from_me?: boolean;
|
||||
date_created?: number;
|
||||
date_delivered?: number;
|
||||
associated_message_guid?: string;
|
||||
sender?: {
|
||||
address?: string;
|
||||
display_name?: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type BlueBubblesChatOpts = {
|
||||
serverUrl?: string;
|
||||
password?: string;
|
||||
accountId?: string;
|
||||
timeoutMs?: number;
|
||||
cfg?: OpenClawConfig;
|
||||
};
|
||||
|
||||
function resolveAccount(params: BlueBubblesChatOpts) {
|
||||
return resolveBlueBubblesServerAccount(params);
|
||||
}
|
||||
|
||||
const MAX_HISTORY_FETCH_LIMIT = 100;
|
||||
const HISTORY_SCAN_MULTIPLIER = 8;
|
||||
const MAX_HISTORY_SCAN_MESSAGES = 500;
|
||||
const MAX_HISTORY_BODY_CHARS = 2_000;
|
||||
|
||||
function clampHistoryLimit(limit: number): number {
|
||||
if (!Number.isFinite(limit)) {
|
||||
return 0;
|
||||
}
|
||||
const normalized = Math.floor(limit);
|
||||
if (normalized <= 0) {
|
||||
return 0;
|
||||
}
|
||||
return Math.min(normalized, MAX_HISTORY_FETCH_LIMIT);
|
||||
}
|
||||
|
||||
function truncateHistoryBody(text: string): string {
|
||||
if (text.length <= MAX_HISTORY_BODY_CHARS) {
|
||||
return text;
|
||||
}
|
||||
return `${text.slice(0, MAX_HISTORY_BODY_CHARS).trimEnd()}...`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch message history from BlueBubbles API for a specific chat.
|
||||
* This provides the initial backfill for both group chats and DMs.
|
||||
*/
|
||||
export async function fetchBlueBubblesHistory(
|
||||
chatIdentifier: string,
|
||||
limit: number,
|
||||
opts: BlueBubblesChatOpts = {},
|
||||
): Promise<BlueBubblesHistoryFetchResult> {
|
||||
const effectiveLimit = clampHistoryLimit(limit);
|
||||
if (!chatIdentifier.trim() || effectiveLimit <= 0) {
|
||||
return { entries: [], resolved: true };
|
||||
}
|
||||
|
||||
let baseUrl: string;
|
||||
let password: string;
|
||||
try {
|
||||
({ baseUrl, password } = resolveAccount(opts));
|
||||
} catch {
|
||||
return { entries: [], resolved: false };
|
||||
}
|
||||
|
||||
// Try different common API patterns for fetching messages
|
||||
const possiblePaths = [
|
||||
`/api/v1/chat/${encodeURIComponent(chatIdentifier)}/messages?limit=${effectiveLimit}&sort=DESC`,
|
||||
`/api/v1/messages?chatGuid=${encodeURIComponent(chatIdentifier)}&limit=${effectiveLimit}`,
|
||||
`/api/v1/chat/${encodeURIComponent(chatIdentifier)}/message?limit=${effectiveLimit}`,
|
||||
];
|
||||
|
||||
for (const path of possiblePaths) {
|
||||
try {
|
||||
const url = buildBlueBubblesApiUrl({ baseUrl, path, password });
|
||||
const res = await blueBubblesFetchWithTimeout(
|
||||
url,
|
||||
{ method: "GET" },
|
||||
opts.timeoutMs ?? 10000,
|
||||
);
|
||||
|
||||
if (!res.ok) {
|
||||
continue; // Try next path
|
||||
}
|
||||
|
||||
const data = await res.json().catch(() => null);
|
||||
if (!data) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Handle different response structures
|
||||
let messages: unknown[] = [];
|
||||
if (Array.isArray(data)) {
|
||||
messages = data;
|
||||
} else if (data.data && Array.isArray(data.data)) {
|
||||
messages = data.data;
|
||||
} else if (data.messages && Array.isArray(data.messages)) {
|
||||
messages = data.messages;
|
||||
} else {
|
||||
continue;
|
||||
}
|
||||
|
||||
const historyEntries: BlueBubblesHistoryEntry[] = [];
|
||||
|
||||
const maxScannedMessages = Math.min(
|
||||
Math.max(effectiveLimit * HISTORY_SCAN_MULTIPLIER, effectiveLimit),
|
||||
MAX_HISTORY_SCAN_MESSAGES,
|
||||
);
|
||||
for (let i = 0; i < messages.length && i < maxScannedMessages; i++) {
|
||||
const item = messages[i];
|
||||
const msg = item as BlueBubblesMessageData;
|
||||
|
||||
// Skip messages without text content
|
||||
const text = msg.text?.trim();
|
||||
if (!text) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const sender = msg.is_from_me
|
||||
? "me"
|
||||
: msg.sender?.display_name || msg.sender?.address || msg.handle_id || "Unknown";
|
||||
const timestamp = msg.date_created || msg.date_delivered;
|
||||
|
||||
historyEntries.push({
|
||||
sender,
|
||||
body: truncateHistoryBody(text),
|
||||
timestamp,
|
||||
messageId: msg.guid,
|
||||
});
|
||||
}
|
||||
|
||||
// Sort by timestamp (oldest first for context)
|
||||
historyEntries.sort((a, b) => {
|
||||
const aTime = a.timestamp || 0;
|
||||
const bTime = b.timestamp || 0;
|
||||
return aTime - bTime;
|
||||
});
|
||||
|
||||
return {
|
||||
entries: historyEntries.slice(0, effectiveLimit), // Ensure we don't exceed the requested limit
|
||||
resolved: true,
|
||||
};
|
||||
} catch (error) {
|
||||
// Continue to next path
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// If none of the API paths worked, return empty history
|
||||
return { entries: [], resolved: false };
|
||||
}
|
||||
@@ -1,21 +1,17 @@
|
||||
import type { OpenClawConfig } from "openclaw/plugin-sdk";
|
||||
import {
|
||||
createReplyPrefixOptions,
|
||||
evictOldHistoryKeys,
|
||||
logAckFailure,
|
||||
logInboundDrop,
|
||||
logTypingFailure,
|
||||
recordPendingHistoryEntryIfEnabled,
|
||||
resolveAckReaction,
|
||||
resolveDmGroupAccessDecision,
|
||||
resolveEffectiveAllowFromLists,
|
||||
resolveControlCommandGate,
|
||||
stripMarkdown,
|
||||
type HistoryEntry,
|
||||
} from "openclaw/plugin-sdk";
|
||||
import { downloadBlueBubblesAttachment } from "./attachments.js";
|
||||
import { markBlueBubblesChatRead, sendBlueBubblesTyping } from "./chat.js";
|
||||
import { fetchBlueBubblesHistory } from "./history.js";
|
||||
import { sendBlueBubblesMedia } from "./media-send.js";
|
||||
import {
|
||||
buildMessagePlaceholder,
|
||||
@@ -243,178 +239,6 @@ function resolveBlueBubblesAckReaction(params: {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* In-memory rolling history map keyed by account + chat identifier.
|
||||
* Populated from incoming messages during the session.
|
||||
* API backfill is attempted until one fetch resolves (or retries are exhausted).
|
||||
*/
|
||||
const chatHistories = new Map<string, HistoryEntry[]>();
|
||||
type HistoryBackfillState = {
|
||||
attempts: number;
|
||||
firstAttemptAt: number;
|
||||
nextAttemptAt: number;
|
||||
resolved: boolean;
|
||||
};
|
||||
|
||||
const historyBackfills = new Map<string, HistoryBackfillState>();
|
||||
const HISTORY_BACKFILL_BASE_DELAY_MS = 5_000;
|
||||
const HISTORY_BACKFILL_MAX_DELAY_MS = 2 * 60 * 1000;
|
||||
const HISTORY_BACKFILL_MAX_ATTEMPTS = 6;
|
||||
const HISTORY_BACKFILL_RETRY_WINDOW_MS = 30 * 60 * 1000;
|
||||
const MAX_STORED_HISTORY_ENTRY_CHARS = 2_000;
|
||||
const MAX_INBOUND_HISTORY_ENTRY_CHARS = 1_200;
|
||||
const MAX_INBOUND_HISTORY_TOTAL_CHARS = 12_000;
|
||||
|
||||
function buildAccountScopedHistoryKey(accountId: string, historyIdentifier: string): string {
|
||||
return `${accountId}\u0000${historyIdentifier}`;
|
||||
}
|
||||
|
||||
function historyDedupKey(entry: HistoryEntry): string {
|
||||
const messageId = entry.messageId?.trim();
|
||||
if (messageId) {
|
||||
return `id:${messageId}`;
|
||||
}
|
||||
return `fallback:${entry.sender}\u0000${entry.body}\u0000${entry.timestamp ?? ""}`;
|
||||
}
|
||||
|
||||
function truncateHistoryBody(body: string, maxChars: number): string {
|
||||
const trimmed = body.trim();
|
||||
if (!trimmed) {
|
||||
return "";
|
||||
}
|
||||
if (trimmed.length <= maxChars) {
|
||||
return trimmed;
|
||||
}
|
||||
return `${trimmed.slice(0, maxChars).trimEnd()}...`;
|
||||
}
|
||||
|
||||
function mergeHistoryEntries(params: {
|
||||
apiEntries: HistoryEntry[];
|
||||
currentEntries: HistoryEntry[];
|
||||
limit: number;
|
||||
}): HistoryEntry[] {
|
||||
if (params.limit <= 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const merged: HistoryEntry[] = [];
|
||||
const seen = new Set<string>();
|
||||
const appendUnique = (entry: HistoryEntry) => {
|
||||
const key = historyDedupKey(entry);
|
||||
if (seen.has(key)) {
|
||||
return;
|
||||
}
|
||||
seen.add(key);
|
||||
merged.push(entry);
|
||||
};
|
||||
|
||||
for (const entry of params.apiEntries) {
|
||||
appendUnique(entry);
|
||||
}
|
||||
for (const entry of params.currentEntries) {
|
||||
appendUnique(entry);
|
||||
}
|
||||
|
||||
if (merged.length <= params.limit) {
|
||||
return merged;
|
||||
}
|
||||
return merged.slice(merged.length - params.limit);
|
||||
}
|
||||
|
||||
function pruneHistoryBackfillState(): void {
|
||||
for (const key of historyBackfills.keys()) {
|
||||
if (!chatHistories.has(key)) {
|
||||
historyBackfills.delete(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function markHistoryBackfillResolved(historyKey: string): void {
|
||||
const state = historyBackfills.get(historyKey);
|
||||
if (state) {
|
||||
state.resolved = true;
|
||||
historyBackfills.set(historyKey, state);
|
||||
return;
|
||||
}
|
||||
historyBackfills.set(historyKey, {
|
||||
attempts: 0,
|
||||
firstAttemptAt: Date.now(),
|
||||
nextAttemptAt: Number.POSITIVE_INFINITY,
|
||||
resolved: true,
|
||||
});
|
||||
}
|
||||
|
||||
function planHistoryBackfillAttempt(historyKey: string, now: number): HistoryBackfillState | null {
|
||||
const existing = historyBackfills.get(historyKey);
|
||||
if (existing?.resolved) {
|
||||
return null;
|
||||
}
|
||||
if (existing && now - existing.firstAttemptAt > HISTORY_BACKFILL_RETRY_WINDOW_MS) {
|
||||
markHistoryBackfillResolved(historyKey);
|
||||
return null;
|
||||
}
|
||||
if (existing && existing.attempts >= HISTORY_BACKFILL_MAX_ATTEMPTS) {
|
||||
markHistoryBackfillResolved(historyKey);
|
||||
return null;
|
||||
}
|
||||
if (existing && now < existing.nextAttemptAt) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const attempts = (existing?.attempts ?? 0) + 1;
|
||||
const firstAttemptAt = existing?.firstAttemptAt ?? now;
|
||||
const backoffDelay = Math.min(
|
||||
HISTORY_BACKFILL_BASE_DELAY_MS * 2 ** (attempts - 1),
|
||||
HISTORY_BACKFILL_MAX_DELAY_MS,
|
||||
);
|
||||
const state: HistoryBackfillState = {
|
||||
attempts,
|
||||
firstAttemptAt,
|
||||
nextAttemptAt: now + backoffDelay,
|
||||
resolved: false,
|
||||
};
|
||||
historyBackfills.set(historyKey, state);
|
||||
return state;
|
||||
}
|
||||
|
||||
function buildInboundHistorySnapshot(params: {
|
||||
entries: HistoryEntry[];
|
||||
limit: number;
|
||||
}): Array<{ sender: string; body: string; timestamp?: number }> | undefined {
|
||||
if (params.limit <= 0 || params.entries.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
const recent = params.entries.slice(-params.limit);
|
||||
const selected: Array<{ sender: string; body: string; timestamp?: number }> = [];
|
||||
let remainingChars = MAX_INBOUND_HISTORY_TOTAL_CHARS;
|
||||
|
||||
for (let i = recent.length - 1; i >= 0; i--) {
|
||||
const entry = recent[i];
|
||||
const body = truncateHistoryBody(entry.body, MAX_INBOUND_HISTORY_ENTRY_CHARS);
|
||||
if (!body) {
|
||||
continue;
|
||||
}
|
||||
if (selected.length > 0 && body.length > remainingChars) {
|
||||
break;
|
||||
}
|
||||
selected.push({
|
||||
sender: entry.sender,
|
||||
body,
|
||||
timestamp: entry.timestamp,
|
||||
});
|
||||
remainingChars -= body.length;
|
||||
if (remainingChars <= 0) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (selected.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
selected.reverse();
|
||||
return selected;
|
||||
}
|
||||
|
||||
export async function processMessage(
|
||||
message: NormalizedWebhookMessage,
|
||||
target: WebhookTarget,
|
||||
@@ -984,118 +808,9 @@ export async function processMessage(
|
||||
.trim();
|
||||
};
|
||||
|
||||
// History: in-memory rolling map with bounded API backfill retries
|
||||
const historyLimit = isGroup
|
||||
? (account.config.historyLimit ?? 0)
|
||||
: (account.config.dmHistoryLimit ?? 0);
|
||||
|
||||
const historyIdentifier =
|
||||
chatGuid ||
|
||||
chatIdentifier ||
|
||||
(chatId ? String(chatId) : null) ||
|
||||
(isGroup ? null : message.senderId) ||
|
||||
"";
|
||||
const historyKey = historyIdentifier
|
||||
? buildAccountScopedHistoryKey(account.accountId, historyIdentifier)
|
||||
: "";
|
||||
|
||||
// Record the current message into rolling history
|
||||
if (historyKey && historyLimit > 0) {
|
||||
const nowMs = Date.now();
|
||||
const senderLabel = message.fromMe ? "me" : message.senderName || message.senderId;
|
||||
const normalizedHistoryBody = truncateHistoryBody(text, MAX_STORED_HISTORY_ENTRY_CHARS);
|
||||
const currentEntries = recordPendingHistoryEntryIfEnabled({
|
||||
historyMap: chatHistories,
|
||||
limit: historyLimit,
|
||||
historyKey,
|
||||
entry: normalizedHistoryBody
|
||||
? {
|
||||
sender: senderLabel,
|
||||
body: normalizedHistoryBody,
|
||||
timestamp: message.timestamp ?? nowMs,
|
||||
messageId: message.messageId ?? undefined,
|
||||
}
|
||||
: null,
|
||||
});
|
||||
pruneHistoryBackfillState();
|
||||
|
||||
const backfillAttempt = planHistoryBackfillAttempt(historyKey, nowMs);
|
||||
if (backfillAttempt) {
|
||||
try {
|
||||
const backfillResult = await fetchBlueBubblesHistory(historyIdentifier, historyLimit, {
|
||||
cfg: config,
|
||||
accountId: account.accountId,
|
||||
});
|
||||
if (backfillResult.resolved) {
|
||||
markHistoryBackfillResolved(historyKey);
|
||||
}
|
||||
if (backfillResult.entries.length > 0) {
|
||||
const apiEntries: HistoryEntry[] = [];
|
||||
for (const entry of backfillResult.entries) {
|
||||
const body = truncateHistoryBody(entry.body, MAX_STORED_HISTORY_ENTRY_CHARS);
|
||||
if (!body) {
|
||||
continue;
|
||||
}
|
||||
apiEntries.push({
|
||||
sender: entry.sender,
|
||||
body,
|
||||
timestamp: entry.timestamp,
|
||||
messageId: entry.messageId,
|
||||
});
|
||||
}
|
||||
const merged = mergeHistoryEntries({
|
||||
apiEntries,
|
||||
currentEntries:
|
||||
currentEntries.length > 0 ? currentEntries : (chatHistories.get(historyKey) ?? []),
|
||||
limit: historyLimit,
|
||||
});
|
||||
if (chatHistories.has(historyKey)) {
|
||||
chatHistories.delete(historyKey);
|
||||
}
|
||||
chatHistories.set(historyKey, merged);
|
||||
evictOldHistoryKeys(chatHistories);
|
||||
logVerbose(
|
||||
core,
|
||||
runtime,
|
||||
`backfilled ${backfillResult.entries.length} history messages for ${isGroup ? "group" : "DM"}: ${historyIdentifier}`,
|
||||
);
|
||||
} else if (!backfillResult.resolved) {
|
||||
const remainingAttempts = HISTORY_BACKFILL_MAX_ATTEMPTS - backfillAttempt.attempts;
|
||||
const nextBackoffMs = Math.max(backfillAttempt.nextAttemptAt - nowMs, 0);
|
||||
logVerbose(
|
||||
core,
|
||||
runtime,
|
||||
`history backfill unresolved for ${historyIdentifier}; retries left=${Math.max(remainingAttempts, 0)} next_in_ms=${nextBackoffMs}`,
|
||||
);
|
||||
}
|
||||
} catch (err) {
|
||||
const remainingAttempts = HISTORY_BACKFILL_MAX_ATTEMPTS - backfillAttempt.attempts;
|
||||
const nextBackoffMs = Math.max(backfillAttempt.nextAttemptAt - nowMs, 0);
|
||||
logVerbose(
|
||||
core,
|
||||
runtime,
|
||||
`history backfill failed for ${historyIdentifier}: ${String(err)} (retries left=${Math.max(remainingAttempts, 0)} next_in_ms=${nextBackoffMs})`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Build inbound history from the in-memory map
|
||||
let inboundHistory: Array<{ sender: string; body: string; timestamp?: number }> | undefined;
|
||||
if (historyKey && historyLimit > 0) {
|
||||
const entries = chatHistories.get(historyKey);
|
||||
if (entries && entries.length > 0) {
|
||||
inboundHistory = buildInboundHistorySnapshot({
|
||||
entries,
|
||||
limit: historyLimit,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const ctxPayload = core.channel.reply.finalizeInboundContext({
|
||||
Body: body,
|
||||
BodyForAgent: rawBody,
|
||||
InboundHistory: inboundHistory,
|
||||
RawBody: rawBody,
|
||||
CommandBody: rawBody,
|
||||
BodyForCommands: rawBody,
|
||||
|
||||
@@ -4,7 +4,6 @@ import type { OpenClawConfig, PluginRuntime } from "openclaw/plugin-sdk";
|
||||
import { removeAckReactionAfterReply, shouldAckReaction } from "openclaw/plugin-sdk";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { ResolvedBlueBubblesAccount } from "./accounts.js";
|
||||
import { fetchBlueBubblesHistory } from "./history.js";
|
||||
import {
|
||||
handleBlueBubblesWebhookRequest,
|
||||
registerBlueBubblesWebhookTarget,
|
||||
@@ -39,10 +38,6 @@ vi.mock("./reactions.js", async () => {
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("./history.js", () => ({
|
||||
fetchBlueBubblesHistory: vi.fn().mockResolvedValue({ entries: [], resolved: true }),
|
||||
}));
|
||||
|
||||
// Mock runtime
|
||||
const mockEnqueueSystemEvent = vi.fn();
|
||||
const mockBuildPairingReply = vi.fn(() => "Pairing code: TESTCODE");
|
||||
@@ -91,7 +86,6 @@ const mockChunkByNewline = vi.fn((text: string) => (text ? [text] : []));
|
||||
const mockChunkTextWithMode = vi.fn((text: string) => (text ? [text] : []));
|
||||
const mockChunkMarkdownTextWithMode = vi.fn((text: string) => (text ? [text] : []));
|
||||
const mockResolveChunkMode = vi.fn(() => "length");
|
||||
const mockFetchBlueBubblesHistory = vi.mocked(fetchBlueBubblesHistory);
|
||||
|
||||
function createMockRuntime(): PluginRuntime {
|
||||
return {
|
||||
@@ -361,7 +355,6 @@ describe("BlueBubbles webhook monitor", () => {
|
||||
vi.clearAllMocks();
|
||||
// Reset short ID state between tests for predictable behavior
|
||||
_resetBlueBubblesShortIdState();
|
||||
mockFetchBlueBubblesHistory.mockResolvedValue({ entries: [], resolved: true });
|
||||
mockReadAllowFromStore.mockResolvedValue([]);
|
||||
mockUpsertPairingRequest.mockResolvedValue({ code: "TESTCODE", created: true });
|
||||
mockResolveRequireMention.mockReturnValue(false);
|
||||
@@ -2998,279 +2991,6 @@ describe("BlueBubbles webhook monitor", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("history backfill", () => {
|
||||
it("scopes in-memory history by account to avoid cross-account leakage", async () => {
|
||||
mockFetchBlueBubblesHistory.mockImplementation(async (_chatIdentifier, _limit, opts) => {
|
||||
if (opts?.accountId === "acc-a") {
|
||||
return {
|
||||
resolved: true,
|
||||
entries: [
|
||||
{ sender: "A", body: "a-history", messageId: "a-history-1", timestamp: 1000 },
|
||||
],
|
||||
};
|
||||
}
|
||||
if (opts?.accountId === "acc-b") {
|
||||
return {
|
||||
resolved: true,
|
||||
entries: [
|
||||
{ sender: "B", body: "b-history", messageId: "b-history-1", timestamp: 1000 },
|
||||
],
|
||||
};
|
||||
}
|
||||
return { resolved: true, entries: [] };
|
||||
});
|
||||
|
||||
const accountA: ResolvedBlueBubblesAccount = {
|
||||
...createMockAccount({ dmHistoryLimit: 3, password: "password-a" }),
|
||||
accountId: "acc-a",
|
||||
};
|
||||
const accountB: ResolvedBlueBubblesAccount = {
|
||||
...createMockAccount({ dmHistoryLimit: 3, password: "password-b" }),
|
||||
accountId: "acc-b",
|
||||
};
|
||||
const config: OpenClawConfig = {};
|
||||
const core = createMockRuntime();
|
||||
setBlueBubblesRuntime(core);
|
||||
|
||||
const unregisterA = registerBlueBubblesWebhookTarget({
|
||||
account: accountA,
|
||||
config,
|
||||
runtime: { log: vi.fn(), error: vi.fn() },
|
||||
core,
|
||||
path: "/bluebubbles-webhook",
|
||||
});
|
||||
const unregisterB = registerBlueBubblesWebhookTarget({
|
||||
account: accountB,
|
||||
config,
|
||||
runtime: { log: vi.fn(), error: vi.fn() },
|
||||
core,
|
||||
path: "/bluebubbles-webhook",
|
||||
});
|
||||
unregister = () => {
|
||||
unregisterA();
|
||||
unregisterB();
|
||||
};
|
||||
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook?password=password-a", {
|
||||
type: "new-message",
|
||||
data: {
|
||||
text: "message for account a",
|
||||
handle: { address: "+15551234567" },
|
||||
isGroup: false,
|
||||
isFromMe: false,
|
||||
guid: "a-msg-1",
|
||||
chatGuid: "iMessage;-;+15551234567",
|
||||
date: Date.now(),
|
||||
},
|
||||
}),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook?password=password-b", {
|
||||
type: "new-message",
|
||||
data: {
|
||||
text: "message for account b",
|
||||
handle: { address: "+15551234567" },
|
||||
isGroup: false,
|
||||
isFromMe: false,
|
||||
guid: "b-msg-1",
|
||||
chatGuid: "iMessage;-;+15551234567",
|
||||
date: Date.now(),
|
||||
},
|
||||
}),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
|
||||
expect(mockDispatchReplyWithBufferedBlockDispatcher).toHaveBeenCalledTimes(2);
|
||||
const firstCall = mockDispatchReplyWithBufferedBlockDispatcher.mock.calls[0]?.[0];
|
||||
const secondCall = mockDispatchReplyWithBufferedBlockDispatcher.mock.calls[1]?.[0];
|
||||
const firstHistory = (firstCall?.ctx.InboundHistory ?? []) as Array<{ body: string }>;
|
||||
const secondHistory = (secondCall?.ctx.InboundHistory ?? []) as Array<{ body: string }>;
|
||||
expect(firstHistory.map((entry) => entry.body)).toContain("a-history");
|
||||
expect(secondHistory.map((entry) => entry.body)).toContain("b-history");
|
||||
expect(secondHistory.map((entry) => entry.body)).not.toContain("a-history");
|
||||
});
|
||||
|
||||
it("dedupes and caps merged history to dmHistoryLimit", async () => {
|
||||
mockFetchBlueBubblesHistory.mockResolvedValueOnce({
|
||||
resolved: true,
|
||||
entries: [
|
||||
{ sender: "Friend", body: "older context", messageId: "hist-1", timestamp: 1000 },
|
||||
{ sender: "Friend", body: "current text", messageId: "msg-1", timestamp: 2000 },
|
||||
],
|
||||
});
|
||||
|
||||
const account = createMockAccount({ dmHistoryLimit: 2 });
|
||||
const config: OpenClawConfig = {};
|
||||
const core = createMockRuntime();
|
||||
setBlueBubblesRuntime(core);
|
||||
|
||||
unregister = registerBlueBubblesWebhookTarget({
|
||||
account,
|
||||
config,
|
||||
runtime: { log: vi.fn(), error: vi.fn() },
|
||||
core,
|
||||
path: "/bluebubbles-webhook",
|
||||
});
|
||||
|
||||
const req = createMockRequest("POST", "/bluebubbles-webhook", {
|
||||
type: "new-message",
|
||||
data: {
|
||||
text: "current text",
|
||||
handle: { address: "+15551234567" },
|
||||
isGroup: false,
|
||||
isFromMe: false,
|
||||
guid: "msg-1",
|
||||
chatGuid: "iMessage;-;+15550002002",
|
||||
date: Date.now(),
|
||||
},
|
||||
});
|
||||
const res = createMockResponse();
|
||||
|
||||
await handleBlueBubblesWebhookRequest(req, res);
|
||||
await flushAsync();
|
||||
|
||||
const callArgs = getFirstDispatchCall();
|
||||
const inboundHistory = (callArgs.ctx.InboundHistory ?? []) as Array<{ body: string }>;
|
||||
expect(inboundHistory).toHaveLength(2);
|
||||
expect(inboundHistory.map((entry) => entry.body)).toEqual(["older context", "current text"]);
|
||||
expect(inboundHistory.filter((entry) => entry.body === "current text")).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("uses exponential backoff for unresolved backfill and stops after resolve", async () => {
|
||||
mockFetchBlueBubblesHistory
|
||||
.mockResolvedValueOnce({ resolved: false, entries: [] })
|
||||
.mockResolvedValueOnce({
|
||||
resolved: true,
|
||||
entries: [
|
||||
{ sender: "Friend", body: "older context", messageId: "hist-1", timestamp: 1000 },
|
||||
],
|
||||
});
|
||||
|
||||
const account = createMockAccount({ dmHistoryLimit: 4 });
|
||||
const config: OpenClawConfig = {};
|
||||
const core = createMockRuntime();
|
||||
setBlueBubblesRuntime(core);
|
||||
|
||||
unregister = registerBlueBubblesWebhookTarget({
|
||||
account,
|
||||
config,
|
||||
runtime: { log: vi.fn(), error: vi.fn() },
|
||||
core,
|
||||
path: "/bluebubbles-webhook",
|
||||
});
|
||||
|
||||
const mkPayload = (guid: string, text: string, now: number) => ({
|
||||
type: "new-message",
|
||||
data: {
|
||||
text,
|
||||
handle: { address: "+15551234567" },
|
||||
isGroup: false,
|
||||
isFromMe: false,
|
||||
guid,
|
||||
chatGuid: "iMessage;-;+15550003003",
|
||||
date: now,
|
||||
},
|
||||
});
|
||||
|
||||
let now = 1_700_000_000_000;
|
||||
const nowSpy = vi.spyOn(Date, "now").mockImplementation(() => now);
|
||||
try {
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook", mkPayload("msg-1", "first text", now)),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
expect(mockFetchBlueBubblesHistory).toHaveBeenCalledTimes(1);
|
||||
|
||||
now += 1_000;
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook", mkPayload("msg-2", "second text", now)),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
expect(mockFetchBlueBubblesHistory).toHaveBeenCalledTimes(1);
|
||||
|
||||
now += 6_000;
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook", mkPayload("msg-3", "third text", now)),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
expect(mockFetchBlueBubblesHistory).toHaveBeenCalledTimes(2);
|
||||
|
||||
const thirdCall = mockDispatchReplyWithBufferedBlockDispatcher.mock.calls[2]?.[0];
|
||||
const thirdHistory = (thirdCall?.ctx.InboundHistory ?? []) as Array<{ body: string }>;
|
||||
expect(thirdHistory.map((entry) => entry.body)).toContain("older context");
|
||||
expect(thirdHistory.map((entry) => entry.body)).toContain("third text");
|
||||
|
||||
now += 10_000;
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook", mkPayload("msg-4", "fourth text", now)),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
expect(mockFetchBlueBubblesHistory).toHaveBeenCalledTimes(2);
|
||||
} finally {
|
||||
nowSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("caps inbound history payload size to reduce prompt-bomb risk", async () => {
|
||||
const huge = "x".repeat(8_000);
|
||||
mockFetchBlueBubblesHistory.mockResolvedValueOnce({
|
||||
resolved: true,
|
||||
entries: Array.from({ length: 20 }, (_, idx) => ({
|
||||
sender: `Friend ${idx}`,
|
||||
body: `${huge} ${idx}`,
|
||||
messageId: `hist-${idx}`,
|
||||
timestamp: idx + 1,
|
||||
})),
|
||||
});
|
||||
|
||||
const account = createMockAccount({ dmHistoryLimit: 20 });
|
||||
const config: OpenClawConfig = {};
|
||||
const core = createMockRuntime();
|
||||
setBlueBubblesRuntime(core);
|
||||
|
||||
unregister = registerBlueBubblesWebhookTarget({
|
||||
account,
|
||||
config,
|
||||
runtime: { log: vi.fn(), error: vi.fn() },
|
||||
core,
|
||||
path: "/bluebubbles-webhook",
|
||||
});
|
||||
|
||||
await handleBlueBubblesWebhookRequest(
|
||||
createMockRequest("POST", "/bluebubbles-webhook", {
|
||||
type: "new-message",
|
||||
data: {
|
||||
text: "latest text",
|
||||
handle: { address: "+15551234567" },
|
||||
isGroup: false,
|
||||
isFromMe: false,
|
||||
guid: "msg-bomb-1",
|
||||
chatGuid: "iMessage;-;+15550004004",
|
||||
date: Date.now(),
|
||||
},
|
||||
}),
|
||||
createMockResponse(),
|
||||
);
|
||||
await flushAsync();
|
||||
|
||||
const callArgs = getFirstDispatchCall();
|
||||
const inboundHistory = (callArgs.ctx.InboundHistory ?? []) as Array<{ body: string }>;
|
||||
const totalChars = inboundHistory.reduce((sum, entry) => sum + entry.body.length, 0);
|
||||
expect(inboundHistory.length).toBeLessThan(20);
|
||||
expect(totalChars).toBeLessThanOrEqual(12_000);
|
||||
expect(inboundHistory.every((entry) => entry.body.length <= 1_203)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("fromMe messages", () => {
|
||||
it("ignores messages from self (fromMe=true)", async () => {
|
||||
const account = createMockAccount();
|
||||
|
||||
58
install.sh
58
install.sh
@@ -1,58 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
INSTALL_DIR="/opt/openclaw"
|
||||
REPO="https://git.yalu.pro/evgeny/openclaw.git"
|
||||
|
||||
# ====== PORT SETTINGS ======
|
||||
OPENCLAW_PORT="${OPENCLAW_PORT:-18789}"
|
||||
OPENCLAW_HOST="${OPENCLAW_HOST:-0.0.0.0}"
|
||||
|
||||
echo "Installing OpenClaw..."
|
||||
echo "Gateway will run on: ${OPENCLAW_HOST}:${OPENCLAW_PORT}"
|
||||
|
||||
# ====== Node 22 ======
|
||||
if ! command -v node >/dev/null || [[ $(node -v | cut -d. -f1 | tr -d v) -lt 22 ]]; then
|
||||
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
||||
apt install -y nodejs
|
||||
fi
|
||||
|
||||
# ====== pnpm ======
|
||||
if ! command -v pnpm >/dev/null; then
|
||||
npm install -g pnpm
|
||||
fi
|
||||
|
||||
# ====== clone or update ======
|
||||
if [ -d "$INSTALL_DIR" ]; then
|
||||
git -C "$INSTALL_DIR" pull
|
||||
else
|
||||
git clone "$REPO" "$INSTALL_DIR"
|
||||
fi
|
||||
|
||||
cd "$INSTALL_DIR"
|
||||
|
||||
pnpm install
|
||||
pnpm build
|
||||
|
||||
echo "Running onboarding..."
|
||||
npx openclaw onboard
|
||||
|
||||
echo "Starting OpenClaw..."
|
||||
|
||||
# передаём порт в runtime
|
||||
export OPENCLAW_PORT
|
||||
export OPENCLAW_HOST
|
||||
|
||||
npx openclaw start &
|
||||
|
||||
sleep 5
|
||||
|
||||
SERVER_IP=$(hostname -I | awk '{print $1}')
|
||||
|
||||
echo ""
|
||||
echo "====================================="
|
||||
echo " OpenClaw is running!"
|
||||
echo ""
|
||||
echo " Local: http://localhost:${OPENCLAW_PORT}"
|
||||
echo " Server: http://${SERVER_IP}:${OPENCLAW_PORT}"
|
||||
echo "====================================="
|
||||
71
src/agents/auth-profiles.runtime-snapshot-save.test.ts
Normal file
71
src/agents/auth-profiles.runtime-snapshot-save.test.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
activateSecretsRuntimeSnapshot,
|
||||
clearSecretsRuntimeSnapshot,
|
||||
prepareSecretsRuntimeSnapshot,
|
||||
} from "../secrets/runtime.js";
|
||||
import { ensureAuthProfileStore, markAuthProfileUsed } from "./auth-profiles.js";
|
||||
|
||||
describe("auth profile runtime snapshot persistence", () => {
|
||||
it("does not write resolved plaintext keys during usage updates", async () => {
|
||||
const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-auth-runtime-save-"));
|
||||
const agentDir = path.join(stateDir, "agents", "main", "agent");
|
||||
const authPath = path.join(agentDir, "auth-profiles.json");
|
||||
try {
|
||||
await fs.mkdir(agentDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
authPath,
|
||||
`${JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openai:default": {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
keyRef: { source: "env", id: "OPENAI_API_KEY" },
|
||||
},
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
|
||||
const snapshot = await prepareSecretsRuntimeSnapshot({
|
||||
config: {},
|
||||
env: { OPENAI_API_KEY: "sk-runtime-openai" },
|
||||
agentDirs: [agentDir],
|
||||
});
|
||||
activateSecretsRuntimeSnapshot(snapshot);
|
||||
|
||||
const runtimeStore = ensureAuthProfileStore(agentDir);
|
||||
expect(runtimeStore.profiles["openai:default"]).toMatchObject({
|
||||
type: "api_key",
|
||||
key: "sk-runtime-openai",
|
||||
keyRef: { source: "env", id: "OPENAI_API_KEY" },
|
||||
});
|
||||
|
||||
await markAuthProfileUsed({
|
||||
store: runtimeStore,
|
||||
profileId: "openai:default",
|
||||
agentDir,
|
||||
});
|
||||
|
||||
const persisted = JSON.parse(await fs.readFile(authPath, "utf8")) as {
|
||||
profiles: Record<string, { key?: string; keyRef?: unknown }>;
|
||||
};
|
||||
expect(persisted.profiles["openai:default"]?.key).toBeUndefined();
|
||||
expect(persisted.profiles["openai:default"]?.keyRef).toEqual({
|
||||
source: "env",
|
||||
id: "OPENAI_API_KEY",
|
||||
});
|
||||
} finally {
|
||||
clearSecretsRuntimeSnapshot();
|
||||
await fs.rm(stateDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
62
src/agents/auth-profiles.store.save.test.ts
Normal file
62
src/agents/auth-profiles.store.save.test.ts
Normal file
@@ -0,0 +1,62 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { resolveAuthStorePath } from "./auth-profiles/paths.js";
|
||||
import { saveAuthProfileStore } from "./auth-profiles/store.js";
|
||||
import type { AuthProfileStore } from "./auth-profiles/types.js";
|
||||
|
||||
describe("saveAuthProfileStore", () => {
|
||||
it("strips plaintext when keyRef/tokenRef are present", async () => {
|
||||
const agentDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-auth-save-"));
|
||||
try {
|
||||
const store: AuthProfileStore = {
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openai:default": {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
key: "sk-runtime-value",
|
||||
keyRef: { source: "env", id: "OPENAI_API_KEY" },
|
||||
},
|
||||
"github-copilot:default": {
|
||||
type: "token",
|
||||
provider: "github-copilot",
|
||||
token: "gh-runtime-token",
|
||||
tokenRef: { source: "env", id: "GITHUB_TOKEN" },
|
||||
},
|
||||
"anthropic:default": {
|
||||
type: "api_key",
|
||||
provider: "anthropic",
|
||||
key: "sk-anthropic-plain",
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
saveAuthProfileStore(store, agentDir);
|
||||
|
||||
const parsed = JSON.parse(await fs.readFile(resolveAuthStorePath(agentDir), "utf8")) as {
|
||||
profiles: Record<
|
||||
string,
|
||||
{ key?: string; keyRef?: unknown; token?: string; tokenRef?: unknown }
|
||||
>;
|
||||
};
|
||||
|
||||
expect(parsed.profiles["openai:default"]?.key).toBeUndefined();
|
||||
expect(parsed.profiles["openai:default"]?.keyRef).toEqual({
|
||||
source: "env",
|
||||
id: "OPENAI_API_KEY",
|
||||
});
|
||||
|
||||
expect(parsed.profiles["github-copilot:default"]?.token).toBeUndefined();
|
||||
expect(parsed.profiles["github-copilot:default"]?.tokenRef).toEqual({
|
||||
source: "env",
|
||||
id: "GITHUB_TOKEN",
|
||||
});
|
||||
|
||||
expect(parsed.profiles["anthropic:default"]?.key).toBe("sk-anthropic-plain");
|
||||
} finally {
|
||||
await fs.rm(agentDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -17,7 +17,10 @@ export {
|
||||
suggestOAuthProfileIdForLegacyDefault,
|
||||
} from "./auth-profiles/repair.js";
|
||||
export {
|
||||
clearRuntimeAuthProfileStoreSnapshots,
|
||||
ensureAuthProfileStore,
|
||||
loadAuthProfileStoreForRuntime,
|
||||
replaceRuntimeAuthProfileStoreSnapshots,
|
||||
loadAuthProfileStore,
|
||||
saveAuthProfileStore,
|
||||
} from "./auth-profiles/store.js";
|
||||
|
||||
@@ -18,6 +18,65 @@ function _syncAuthProfileStore(target: AuthProfileStore, source: AuthProfileStor
|
||||
target.usageStats = source.usageStats;
|
||||
}
|
||||
|
||||
const runtimeAuthStoreSnapshots = new Map<string, AuthProfileStore>();
|
||||
|
||||
function resolveRuntimeStoreKey(agentDir?: string): string {
|
||||
return resolveAuthStorePath(agentDir);
|
||||
}
|
||||
|
||||
function cloneAuthProfileStore(store: AuthProfileStore): AuthProfileStore {
|
||||
return structuredClone(store);
|
||||
}
|
||||
|
||||
function resolveRuntimeAuthProfileStore(agentDir?: string): AuthProfileStore | null {
|
||||
if (runtimeAuthStoreSnapshots.size === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const mainKey = resolveRuntimeStoreKey(undefined);
|
||||
const requestedKey = resolveRuntimeStoreKey(agentDir);
|
||||
const mainStore = runtimeAuthStoreSnapshots.get(mainKey);
|
||||
const requestedStore = runtimeAuthStoreSnapshots.get(requestedKey);
|
||||
|
||||
if (!agentDir || requestedKey === mainKey) {
|
||||
if (!mainStore) {
|
||||
return null;
|
||||
}
|
||||
return cloneAuthProfileStore(mainStore);
|
||||
}
|
||||
|
||||
if (mainStore && requestedStore) {
|
||||
return mergeAuthProfileStores(
|
||||
cloneAuthProfileStore(mainStore),
|
||||
cloneAuthProfileStore(requestedStore),
|
||||
);
|
||||
}
|
||||
if (requestedStore) {
|
||||
return cloneAuthProfileStore(requestedStore);
|
||||
}
|
||||
if (mainStore) {
|
||||
return cloneAuthProfileStore(mainStore);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
export function replaceRuntimeAuthProfileStoreSnapshots(
|
||||
entries: Array<{ agentDir?: string; store: AuthProfileStore }>,
|
||||
): void {
|
||||
runtimeAuthStoreSnapshots.clear();
|
||||
for (const entry of entries) {
|
||||
runtimeAuthStoreSnapshots.set(
|
||||
resolveRuntimeStoreKey(entry.agentDir),
|
||||
cloneAuthProfileStore(entry.store),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export function clearRuntimeAuthProfileStoreSnapshots(): void {
|
||||
runtimeAuthStoreSnapshots.clear();
|
||||
}
|
||||
|
||||
export async function updateAuthProfileStoreWithLock(params: {
|
||||
agentDir?: string;
|
||||
updater: (store: AuthProfileStore) => boolean;
|
||||
@@ -316,10 +375,30 @@ function loadAuthProfileStoreForAgent(
|
||||
return store;
|
||||
}
|
||||
|
||||
export function loadAuthProfileStoreForRuntime(
|
||||
agentDir?: string,
|
||||
options?: { allowKeychainPrompt?: boolean },
|
||||
): AuthProfileStore {
|
||||
const store = loadAuthProfileStoreForAgent(agentDir, options);
|
||||
const authPath = resolveAuthStorePath(agentDir);
|
||||
const mainAuthPath = resolveAuthStorePath();
|
||||
if (!agentDir || authPath === mainAuthPath) {
|
||||
return store;
|
||||
}
|
||||
|
||||
const mainStore = loadAuthProfileStoreForAgent(undefined, options);
|
||||
return mergeAuthProfileStores(mainStore, store);
|
||||
}
|
||||
|
||||
export function ensureAuthProfileStore(
|
||||
agentDir?: string,
|
||||
options?: { allowKeychainPrompt?: boolean },
|
||||
): AuthProfileStore {
|
||||
const runtimeStore = resolveRuntimeAuthProfileStore(agentDir);
|
||||
if (runtimeStore) {
|
||||
return runtimeStore;
|
||||
}
|
||||
|
||||
const store = loadAuthProfileStoreForAgent(agentDir, options);
|
||||
const authPath = resolveAuthStorePath(agentDir);
|
||||
const mainAuthPath = resolveAuthStorePath();
|
||||
@@ -335,9 +414,24 @@ export function ensureAuthProfileStore(
|
||||
|
||||
export function saveAuthProfileStore(store: AuthProfileStore, agentDir?: string): void {
|
||||
const authPath = resolveAuthStorePath(agentDir);
|
||||
const profiles = Object.fromEntries(
|
||||
Object.entries(store.profiles).map(([profileId, credential]) => {
|
||||
if (credential.type === "api_key" && credential.keyRef && credential.key !== undefined) {
|
||||
const sanitized = { ...credential } as Record<string, unknown>;
|
||||
delete sanitized.key;
|
||||
return [profileId, sanitized];
|
||||
}
|
||||
if (credential.type === "token" && credential.tokenRef && credential.token !== undefined) {
|
||||
const sanitized = { ...credential } as Record<string, unknown>;
|
||||
delete sanitized.token;
|
||||
return [profileId, sanitized];
|
||||
}
|
||||
return [profileId, credential];
|
||||
}),
|
||||
) as AuthProfileStore["profiles"];
|
||||
const payload = {
|
||||
version: AUTH_STORE_VERSION,
|
||||
profiles: store.profiles,
|
||||
profiles,
|
||||
order: store.order ?? undefined,
|
||||
lastGood: store.lastGood ?? undefined,
|
||||
usageStats: store.usageStats ?? undefined,
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import type { OAuthCredentials } from "@mariozechner/pi-ai";
|
||||
import type { OpenClawConfig } from "../../config/config.js";
|
||||
import type { SecretRef } from "../../config/types.secrets.js";
|
||||
|
||||
export type ApiKeyCredential = {
|
||||
type: "api_key";
|
||||
provider: string;
|
||||
key?: string;
|
||||
keyRef?: SecretRef;
|
||||
email?: string;
|
||||
/** Optional provider-specific metadata (e.g., account IDs, gateway IDs). */
|
||||
metadata?: Record<string, string>;
|
||||
@@ -18,6 +20,7 @@ export type TokenCredential = {
|
||||
type: "token";
|
||||
provider: string;
|
||||
token: string;
|
||||
tokenRef?: SecretRef;
|
||||
/** Optional expiry timestamp (ms since epoch). */
|
||||
expires?: number;
|
||||
email?: string;
|
||||
|
||||
@@ -31,6 +31,7 @@ export function mockCatalogImportFailThenRecover() {
|
||||
throw new Error("boom");
|
||||
}
|
||||
return {
|
||||
discoverAuthStorage: () => ({}),
|
||||
AuthStorage: class {},
|
||||
ModelRegistry: class {
|
||||
getAll() {
|
||||
|
||||
@@ -38,6 +38,7 @@ describe("loadModelCatalog", () => {
|
||||
__setModelCatalogImportForTest(
|
||||
async () =>
|
||||
({
|
||||
discoverAuthStorage: () => ({}),
|
||||
AuthStorage: class {},
|
||||
ModelRegistry: class {
|
||||
getAll() {
|
||||
@@ -69,6 +70,7 @@ describe("loadModelCatalog", () => {
|
||||
__setModelCatalogImportForTest(
|
||||
async () =>
|
||||
({
|
||||
discoverAuthStorage: () => ({}),
|
||||
AuthStorage: class {},
|
||||
ModelRegistry: class {
|
||||
getAll() {
|
||||
|
||||
@@ -70,14 +70,6 @@ export function __setModelCatalogImportForTest(loader?: () => Promise<PiSdkModul
|
||||
importPiSdk = loader ?? defaultImportPiSdk;
|
||||
}
|
||||
|
||||
function createAuthStorage(AuthStorageLike: unknown, path: string) {
|
||||
const withFactory = AuthStorageLike as { create?: (path: string) => unknown };
|
||||
if (typeof withFactory.create === "function") {
|
||||
return withFactory.create(path);
|
||||
}
|
||||
return new (AuthStorageLike as { new (path: string): unknown })(path);
|
||||
}
|
||||
|
||||
export async function loadModelCatalog(params?: {
|
||||
config?: OpenClawConfig;
|
||||
useCache?: boolean;
|
||||
@@ -102,9 +94,6 @@ export async function loadModelCatalog(params?: {
|
||||
try {
|
||||
const cfg = params?.config ?? loadConfig();
|
||||
await ensureOpenClawModelsJson(cfg);
|
||||
await (
|
||||
await import("./pi-auth-json.js")
|
||||
).ensurePiAuthJsonFromAuthProfiles(resolveOpenClawAgentDir());
|
||||
// IMPORTANT: keep the dynamic import *inside* the try/catch.
|
||||
// If this fails once (e.g. during a pnpm install that temporarily swaps node_modules),
|
||||
// we must not poison the cache with a rejected promise (otherwise all channel handlers
|
||||
@@ -112,7 +101,7 @@ export async function loadModelCatalog(params?: {
|
||||
const piSdk = await importPiSdk();
|
||||
const agentDir = resolveOpenClawAgentDir();
|
||||
const { join } = await import("node:path");
|
||||
const authStorage = createAuthStorage(piSdk.AuthStorage, join(agentDir, "auth.json"));
|
||||
const authStorage = piSdk.discoverAuthStorage(agentDir);
|
||||
const registry = new (piSdk.ModelRegistry as unknown as {
|
||||
new (
|
||||
authStorage: unknown,
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
DEFAULT_COPILOT_API_BASE_URL,
|
||||
resolveCopilotApiToken,
|
||||
} from "../providers/github-copilot-token.js";
|
||||
import { normalizeOptionalSecretInput } from "../utils/normalize-secret-input.js";
|
||||
import { ensureAuthProfileStore, listProfilesForProvider } from "./auth-profiles.js";
|
||||
import { discoverBedrockModels } from "./bedrock-discovery.js";
|
||||
import {
|
||||
@@ -387,16 +388,17 @@ export function normalizeProviders(params: {
|
||||
for (const [key, provider] of Object.entries(providers)) {
|
||||
const normalizedKey = key.trim();
|
||||
let normalizedProvider = provider;
|
||||
const configuredApiKey = normalizedProvider.apiKey;
|
||||
|
||||
// Fix common misconfig: apiKey set to "${ENV_VAR}" instead of "ENV_VAR".
|
||||
if (
|
||||
normalizedProvider.apiKey &&
|
||||
normalizeApiKeyConfig(normalizedProvider.apiKey) !== normalizedProvider.apiKey
|
||||
typeof configuredApiKey === "string" &&
|
||||
normalizeApiKeyConfig(configuredApiKey) !== configuredApiKey
|
||||
) {
|
||||
mutated = true;
|
||||
normalizedProvider = {
|
||||
...normalizedProvider,
|
||||
apiKey: normalizeApiKeyConfig(normalizedProvider.apiKey),
|
||||
apiKey: normalizeApiKeyConfig(configuredApiKey),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -404,7 +406,9 @@ export function normalizeProviders(params: {
|
||||
// Fill it from the environment or auth profiles when possible.
|
||||
const hasModels =
|
||||
Array.isArray(normalizedProvider.models) && normalizedProvider.models.length > 0;
|
||||
if (hasModels && !normalizedProvider.apiKey?.trim()) {
|
||||
const normalizedApiKey = normalizeOptionalSecretInput(normalizedProvider.apiKey);
|
||||
const hasConfiguredApiKey = Boolean(normalizedApiKey || normalizedProvider.apiKey);
|
||||
if (hasModels && !hasConfiguredApiKey) {
|
||||
const authMode =
|
||||
normalizedProvider.auth ?? (normalizedKey === "amazon-bedrock" ? "aws-sdk" : undefined);
|
||||
if (authMode === "aws-sdk") {
|
||||
@@ -957,17 +961,8 @@ export async function resolveImplicitCopilotProvider(params: {
|
||||
}
|
||||
}
|
||||
|
||||
// pi-coding-agent's ModelRegistry marks a model "available" only if its
|
||||
// `AuthStorage` has auth configured for that provider (via auth.json/env/etc).
|
||||
// Our Copilot auth lives in OpenClaw's auth-profiles store instead, so we also
|
||||
// write a runtime-only auth.json entry for pi-coding-agent to pick up.
|
||||
//
|
||||
// This is safe because it's (1) within OpenClaw's agent dir, (2) contains the
|
||||
// GitHub token (not the exchanged Copilot token), and (3) matches existing
|
||||
// patterns for OAuth-like providers in pi-coding-agent.
|
||||
// Note: we deliberately do not write pi-coding-agent's `auth.json` here.
|
||||
// OpenClaw uses its own auth store and exchanges tokens at runtime.
|
||||
// `models list` uses OpenClaw's auth heuristics for availability.
|
||||
// We deliberately do not write pi-coding-agent auth.json here.
|
||||
// OpenClaw keeps auth in auth-profiles and resolves runtime availability from that store.
|
||||
|
||||
// We intentionally do NOT define custom models for Copilot in models.json.
|
||||
// pi-coding-agent treats providers with models as replacements requiring apiKey.
|
||||
|
||||
@@ -244,40 +244,6 @@ describe("parseNdjsonStream", () => {
|
||||
// Final done:true chunk has no tool_calls
|
||||
expect(chunks[2].message.tool_calls).toBeUndefined();
|
||||
});
|
||||
|
||||
it("preserves unsafe integer tool arguments as exact strings", async () => {
|
||||
const reader = mockNdjsonReader([
|
||||
'{"model":"m","created_at":"t","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"send","arguments":{"target":1234567890123456789,"nested":{"thread":9223372036854775807}}}}]},"done":false}',
|
||||
]);
|
||||
|
||||
const chunks = [];
|
||||
for await (const chunk of parseNdjsonStream(reader)) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
|
||||
const args = chunks[0]?.message.tool_calls?.[0]?.function.arguments as
|
||||
| { target?: unknown; nested?: { thread?: unknown } }
|
||||
| undefined;
|
||||
expect(args?.target).toBe("1234567890123456789");
|
||||
expect(args?.nested?.thread).toBe("9223372036854775807");
|
||||
});
|
||||
|
||||
it("keeps safe integer tool arguments as numbers", async () => {
|
||||
const reader = mockNdjsonReader([
|
||||
'{"model":"m","created_at":"t","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"send","arguments":{"retries":3,"delayMs":2500}}}]},"done":false}',
|
||||
]);
|
||||
|
||||
const chunks = [];
|
||||
for await (const chunk of parseNdjsonStream(reader)) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
|
||||
const args = chunks[0]?.message.tool_calls?.[0]?.function.arguments as
|
||||
| { retries?: unknown; delayMs?: unknown }
|
||||
| undefined;
|
||||
expect(args?.retries).toBe(3);
|
||||
expect(args?.delayMs).toBe(2500);
|
||||
});
|
||||
});
|
||||
|
||||
describe("createOllamaStreamFn", () => {
|
||||
|
||||
@@ -49,130 +49,6 @@ interface OllamaToolCall {
|
||||
};
|
||||
}
|
||||
|
||||
const MAX_SAFE_INTEGER_ABS_STR = String(Number.MAX_SAFE_INTEGER);
|
||||
|
||||
function isAsciiDigit(ch: string | undefined): boolean {
|
||||
return ch !== undefined && ch >= "0" && ch <= "9";
|
||||
}
|
||||
|
||||
function parseJsonNumberToken(
|
||||
input: string,
|
||||
start: number,
|
||||
): { token: string; end: number; isInteger: boolean } | null {
|
||||
let idx = start;
|
||||
if (input[idx] === "-") {
|
||||
idx += 1;
|
||||
}
|
||||
if (idx >= input.length) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (input[idx] === "0") {
|
||||
idx += 1;
|
||||
} else if (isAsciiDigit(input[idx]) && input[idx] !== "0") {
|
||||
while (isAsciiDigit(input[idx])) {
|
||||
idx += 1;
|
||||
}
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
|
||||
let isInteger = true;
|
||||
if (input[idx] === ".") {
|
||||
isInteger = false;
|
||||
idx += 1;
|
||||
if (!isAsciiDigit(input[idx])) {
|
||||
return null;
|
||||
}
|
||||
while (isAsciiDigit(input[idx])) {
|
||||
idx += 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (input[idx] === "e" || input[idx] === "E") {
|
||||
isInteger = false;
|
||||
idx += 1;
|
||||
if (input[idx] === "+" || input[idx] === "-") {
|
||||
idx += 1;
|
||||
}
|
||||
if (!isAsciiDigit(input[idx])) {
|
||||
return null;
|
||||
}
|
||||
while (isAsciiDigit(input[idx])) {
|
||||
idx += 1;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
token: input.slice(start, idx),
|
||||
end: idx,
|
||||
isInteger,
|
||||
};
|
||||
}
|
||||
|
||||
function isUnsafeIntegerLiteral(token: string): boolean {
|
||||
const digits = token[0] === "-" ? token.slice(1) : token;
|
||||
if (digits.length < MAX_SAFE_INTEGER_ABS_STR.length) {
|
||||
return false;
|
||||
}
|
||||
if (digits.length > MAX_SAFE_INTEGER_ABS_STR.length) {
|
||||
return true;
|
||||
}
|
||||
return digits > MAX_SAFE_INTEGER_ABS_STR;
|
||||
}
|
||||
|
||||
function quoteUnsafeIntegerLiterals(input: string): string {
|
||||
let out = "";
|
||||
let inString = false;
|
||||
let escaped = false;
|
||||
let idx = 0;
|
||||
|
||||
while (idx < input.length) {
|
||||
const ch = input[idx] ?? "";
|
||||
if (inString) {
|
||||
out += ch;
|
||||
if (escaped) {
|
||||
escaped = false;
|
||||
} else if (ch === "\\") {
|
||||
escaped = true;
|
||||
} else if (ch === '"') {
|
||||
inString = false;
|
||||
}
|
||||
idx += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === '"') {
|
||||
inString = true;
|
||||
out += ch;
|
||||
idx += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === "-" || isAsciiDigit(ch)) {
|
||||
const parsed = parseJsonNumberToken(input, idx);
|
||||
if (parsed) {
|
||||
if (parsed.isInteger && isUnsafeIntegerLiteral(parsed.token)) {
|
||||
out += `"${parsed.token}"`;
|
||||
} else {
|
||||
out += parsed.token;
|
||||
}
|
||||
idx = parsed.end;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
out += ch;
|
||||
idx += 1;
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
function parseJsonPreservingUnsafeIntegers(input: string): unknown {
|
||||
return JSON.parse(quoteUnsafeIntegerLiterals(input)) as unknown;
|
||||
}
|
||||
|
||||
// ── Ollama /api/chat response types ─────────────────────────────────────────
|
||||
|
||||
interface OllamaChatResponse {
|
||||
@@ -386,7 +262,7 @@ export async function* parseNdjsonStream(
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
yield parseJsonPreservingUnsafeIntegers(trimmed) as OllamaChatResponse;
|
||||
yield JSON.parse(trimmed) as OllamaChatResponse;
|
||||
} catch {
|
||||
log.warn(`Skipping malformed NDJSON line: ${trimmed.slice(0, 120)}`);
|
||||
}
|
||||
@@ -395,7 +271,7 @@ export async function* parseNdjsonStream(
|
||||
|
||||
if (buffer.trim()) {
|
||||
try {
|
||||
yield parseJsonPreservingUnsafeIntegers(buffer.trim()) as OllamaChatResponse;
|
||||
yield JSON.parse(buffer.trim()) as OllamaChatResponse;
|
||||
} catch {
|
||||
log.warn(`Skipping malformed trailing data: ${buffer.trim().slice(0, 120)}`);
|
||||
}
|
||||
|
||||
@@ -4,6 +4,10 @@ import { ensureAuthProfileStore } from "./auth-profiles.js";
|
||||
import type { AuthProfileCredential } from "./auth-profiles/types.js";
|
||||
import { normalizeProviderId } from "./model-selection.js";
|
||||
|
||||
/**
|
||||
* @deprecated Legacy bridge for older flows that still expect `agentDir/auth.json`.
|
||||
* Runtime auth resolution uses auth-profiles directly and should not depend on this module.
|
||||
*/
|
||||
type AuthJsonCredential =
|
||||
| {
|
||||
type: "api_key";
|
||||
@@ -110,6 +114,8 @@ function credentialsEqual(a: AuthJsonCredential | undefined, b: AuthJsonCredenti
|
||||
* registry/catalog output.
|
||||
*
|
||||
* Syncs all credential types: api_key, token (as api_key), and oauth.
|
||||
*
|
||||
* @deprecated Runtime auth now comes from OpenClaw auth-profiles snapshots.
|
||||
*/
|
||||
export async function ensurePiAuthJsonFromAuthProfiles(agentDir: string): Promise<{
|
||||
wrote: boolean;
|
||||
|
||||
@@ -377,11 +377,4 @@ describe("classifyFailoverReason", () => {
|
||||
),
|
||||
).toBe("rate_limit");
|
||||
});
|
||||
it("classifies JSON api_error internal server failures as timeout", () => {
|
||||
expect(
|
||||
classifyFailoverReason(
|
||||
'{"type":"error","error":{"type":"api_error","message":"Internal server error"}}',
|
||||
),
|
||||
).toBe("timeout");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -686,16 +686,6 @@ export function isOverloadedErrorMessage(raw: string): boolean {
|
||||
return matchesErrorPatterns(raw, ERROR_PATTERNS.overloaded);
|
||||
}
|
||||
|
||||
function isJsonApiInternalServerError(raw: string): boolean {
|
||||
if (!raw) {
|
||||
return false;
|
||||
}
|
||||
const value = raw.toLowerCase();
|
||||
// Anthropic often wraps transient 500s in JSON payloads like:
|
||||
// {"type":"error","error":{"type":"api_error","message":"Internal server error"}}
|
||||
return value.includes('"type":"api_error"') && value.includes("internal server error");
|
||||
}
|
||||
|
||||
export function parseImageDimensionError(raw: string): {
|
||||
maxDimensionPx?: number;
|
||||
messageIndex?: number;
|
||||
@@ -804,9 +794,6 @@ export function classifyFailoverReason(raw: string): FailoverReason | null {
|
||||
// Treat transient 5xx provider failures as retryable transport issues.
|
||||
return "timeout";
|
||||
}
|
||||
if (isJsonApiInternalServerError(raw)) {
|
||||
return "timeout";
|
||||
}
|
||||
if (isRateLimitErrorMessage(raw)) {
|
||||
return "rate_limit";
|
||||
}
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { createInlineCodeState } from "../markdown/code-spans.js";
|
||||
import { handleAgentEnd } from "./pi-embedded-subscribe.handlers.lifecycle.js";
|
||||
import type { EmbeddedPiSubscribeContext } from "./pi-embedded-subscribe.handlers.types.js";
|
||||
|
||||
vi.mock("../infra/agent-events.js", () => ({
|
||||
emitAgentEvent: vi.fn(),
|
||||
}));
|
||||
|
||||
function createContext(
|
||||
lastAssistant: unknown,
|
||||
overrides?: { onAgentEvent?: (event: unknown) => void },
|
||||
): EmbeddedPiSubscribeContext {
|
||||
return {
|
||||
params: {
|
||||
runId: "run-1",
|
||||
config: {},
|
||||
sessionKey: "agent:main:main",
|
||||
onAgentEvent: overrides?.onAgentEvent,
|
||||
},
|
||||
state: {
|
||||
lastAssistant: lastAssistant as EmbeddedPiSubscribeContext["state"]["lastAssistant"],
|
||||
pendingCompactionRetry: 0,
|
||||
blockState: {
|
||||
thinking: true,
|
||||
final: true,
|
||||
inlineCode: createInlineCodeState(),
|
||||
},
|
||||
},
|
||||
log: {
|
||||
debug: vi.fn(),
|
||||
warn: vi.fn(),
|
||||
},
|
||||
flushBlockReplyBuffer: vi.fn(),
|
||||
resolveCompactionRetry: vi.fn(),
|
||||
maybeResolveCompactionWait: vi.fn(),
|
||||
} as unknown as EmbeddedPiSubscribeContext;
|
||||
}
|
||||
|
||||
describe("handleAgentEnd", () => {
|
||||
it("logs the resolved error message when run ends with assistant error", () => {
|
||||
const onAgentEvent = vi.fn();
|
||||
const ctx = createContext(
|
||||
{
|
||||
role: "assistant",
|
||||
stopReason: "error",
|
||||
errorMessage: "connection refused",
|
||||
content: [{ type: "text", text: "" }],
|
||||
},
|
||||
{ onAgentEvent },
|
||||
);
|
||||
|
||||
handleAgentEnd(ctx);
|
||||
|
||||
const warn = vi.mocked(ctx.log.warn);
|
||||
expect(warn).toHaveBeenCalledTimes(1);
|
||||
expect(warn.mock.calls[0]?.[0]).toContain("runId=run-1");
|
||||
expect(warn.mock.calls[0]?.[0]).toContain("error=connection refused");
|
||||
expect(onAgentEvent).toHaveBeenCalledWith({
|
||||
stream: "lifecycle",
|
||||
data: {
|
||||
phase: "error",
|
||||
error: "connection refused",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps non-error run-end logging on debug only", () => {
|
||||
const ctx = createContext(undefined);
|
||||
|
||||
handleAgentEnd(ctx);
|
||||
|
||||
expect(ctx.log.warn).not.toHaveBeenCalled();
|
||||
expect(ctx.log.debug).toHaveBeenCalledWith("embedded run agent end: runId=run-1 isError=false");
|
||||
});
|
||||
});
|
||||
@@ -29,6 +29,8 @@ export function handleAgentEnd(ctx: EmbeddedPiSubscribeContext) {
|
||||
const lastAssistant = ctx.state.lastAssistant;
|
||||
const isError = isAssistantMessage(lastAssistant) && lastAssistant.stopReason === "error";
|
||||
|
||||
ctx.log.debug(`embedded run agent end: runId=${ctx.params.runId} isError=${isError}`);
|
||||
|
||||
if (isError && lastAssistant) {
|
||||
const friendlyError = formatAssistantErrorText(lastAssistant, {
|
||||
cfg: ctx.params.config,
|
||||
@@ -36,16 +38,12 @@ export function handleAgentEnd(ctx: EmbeddedPiSubscribeContext) {
|
||||
provider: lastAssistant.provider,
|
||||
model: lastAssistant.model,
|
||||
});
|
||||
const errorText = (friendlyError || lastAssistant.errorMessage || "LLM request failed.").trim();
|
||||
ctx.log.warn(
|
||||
`embedded run agent end: runId=${ctx.params.runId} isError=true error=${errorText}`,
|
||||
);
|
||||
emitAgentEvent({
|
||||
runId: ctx.params.runId,
|
||||
stream: "lifecycle",
|
||||
data: {
|
||||
phase: "error",
|
||||
error: errorText,
|
||||
error: friendlyError || lastAssistant.errorMessage || "LLM request failed.",
|
||||
endedAt: Date.now(),
|
||||
},
|
||||
});
|
||||
@@ -53,11 +51,10 @@ export function handleAgentEnd(ctx: EmbeddedPiSubscribeContext) {
|
||||
stream: "lifecycle",
|
||||
data: {
|
||||
phase: "error",
|
||||
error: errorText,
|
||||
error: friendlyError || lastAssistant.errorMessage || "LLM request failed.",
|
||||
},
|
||||
});
|
||||
} else {
|
||||
ctx.log.debug(`embedded run agent end: runId=${ctx.params.runId} isError=${isError}`);
|
||||
emitAgentEvent({
|
||||
runId: ctx.params.runId,
|
||||
stream: "lifecycle",
|
||||
|
||||
116
src/agents/pi-model-discovery.auth.test.ts
Normal file
116
src/agents/pi-model-discovery.auth.test.ts
Normal file
@@ -0,0 +1,116 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { saveAuthProfileStore } from "./auth-profiles.js";
|
||||
import { discoverAuthStorage } from "./pi-model-discovery.js";
|
||||
|
||||
async function createAgentDir(): Promise<string> {
|
||||
return await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-pi-auth-storage-"));
|
||||
}
|
||||
|
||||
async function pathExists(pathname: string): Promise<boolean> {
|
||||
try {
|
||||
await fs.stat(pathname);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
describe("discoverAuthStorage", () => {
|
||||
it("loads runtime credentials from auth-profiles without writing auth.json", async () => {
|
||||
const agentDir = await createAgentDir();
|
||||
try {
|
||||
saveAuthProfileStore(
|
||||
{
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openrouter:default": {
|
||||
type: "api_key",
|
||||
provider: "openrouter",
|
||||
key: "sk-or-v1-runtime",
|
||||
},
|
||||
"anthropic:default": {
|
||||
type: "token",
|
||||
provider: "anthropic",
|
||||
token: "sk-ant-runtime",
|
||||
},
|
||||
"openai-codex:default": {
|
||||
type: "oauth",
|
||||
provider: "openai-codex",
|
||||
access: "oauth-access",
|
||||
refresh: "oauth-refresh",
|
||||
expires: Date.now() + 60_000,
|
||||
},
|
||||
},
|
||||
},
|
||||
agentDir,
|
||||
);
|
||||
|
||||
const authStorage = discoverAuthStorage(agentDir);
|
||||
|
||||
expect(authStorage.hasAuth("openrouter")).toBe(true);
|
||||
expect(authStorage.hasAuth("anthropic")).toBe(true);
|
||||
expect(authStorage.hasAuth("openai-codex")).toBe(true);
|
||||
await expect(authStorage.getApiKey("openrouter")).resolves.toBe("sk-or-v1-runtime");
|
||||
await expect(authStorage.getApiKey("anthropic")).resolves.toBe("sk-ant-runtime");
|
||||
expect(authStorage.get("openai-codex")).toMatchObject({
|
||||
type: "oauth",
|
||||
access: "oauth-access",
|
||||
});
|
||||
|
||||
expect(await pathExists(path.join(agentDir, "auth.json"))).toBe(false);
|
||||
} finally {
|
||||
await fs.rm(agentDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("scrubs static api_key entries from legacy auth.json and keeps oauth entries", async () => {
|
||||
const agentDir = await createAgentDir();
|
||||
try {
|
||||
saveAuthProfileStore(
|
||||
{
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openrouter:default": {
|
||||
type: "api_key",
|
||||
provider: "openrouter",
|
||||
key: "sk-or-v1-runtime",
|
||||
},
|
||||
},
|
||||
},
|
||||
agentDir,
|
||||
);
|
||||
await fs.writeFile(
|
||||
path.join(agentDir, "auth.json"),
|
||||
JSON.stringify(
|
||||
{
|
||||
openrouter: { type: "api_key", key: "legacy-static-key" },
|
||||
"openai-codex": {
|
||||
type: "oauth",
|
||||
access: "oauth-access",
|
||||
refresh: "oauth-refresh",
|
||||
expires: Date.now() + 60_000,
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
|
||||
discoverAuthStorage(agentDir);
|
||||
|
||||
const parsed = JSON.parse(await fs.readFile(path.join(agentDir, "auth.json"), "utf8")) as {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
expect(parsed.openrouter).toBeUndefined();
|
||||
expect(parsed["openai-codex"]).toMatchObject({
|
||||
type: "oauth",
|
||||
access: "oauth-access",
|
||||
});
|
||||
} finally {
|
||||
await fs.rm(agentDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,19 +1,172 @@
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { AuthStorage, ModelRegistry } from "@mariozechner/pi-coding-agent";
|
||||
import {
|
||||
AuthStorage,
|
||||
InMemoryAuthStorageBackend,
|
||||
ModelRegistry,
|
||||
} from "@mariozechner/pi-coding-agent";
|
||||
import { ensureAuthProfileStore } from "./auth-profiles.js";
|
||||
import type { AuthProfileCredential } from "./auth-profiles.js";
|
||||
import { normalizeProviderId } from "./model-selection.js";
|
||||
|
||||
export { AuthStorage, ModelRegistry } from "@mariozechner/pi-coding-agent";
|
||||
|
||||
function createAuthStorage(AuthStorageLike: unknown, path: string) {
|
||||
const withFactory = AuthStorageLike as { create?: (path: string) => unknown };
|
||||
if (typeof withFactory.create === "function") {
|
||||
return withFactory.create(path) as AuthStorage;
|
||||
type PiApiKeyCredential = { type: "api_key"; key: string };
|
||||
type PiOAuthCredential = {
|
||||
type: "oauth";
|
||||
access: string;
|
||||
refresh: string;
|
||||
expires: number;
|
||||
};
|
||||
|
||||
type PiCredential = PiApiKeyCredential | PiOAuthCredential;
|
||||
type PiCredentialMap = Record<string, PiCredential>;
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function scrubLegacyStaticAuthJsonEntries(pathname: string): void {
|
||||
if (!fs.existsSync(pathname)) {
|
||||
return;
|
||||
}
|
||||
return new (AuthStorageLike as { new (path: string): unknown })(path) as AuthStorage;
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(fs.readFileSync(pathname, "utf8")) as unknown;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!isRecord(parsed)) {
|
||||
return;
|
||||
}
|
||||
|
||||
let changed = false;
|
||||
for (const [provider, value] of Object.entries(parsed)) {
|
||||
if (!isRecord(value)) {
|
||||
continue;
|
||||
}
|
||||
if (value.type !== "api_key") {
|
||||
continue;
|
||||
}
|
||||
delete parsed[provider];
|
||||
changed = true;
|
||||
}
|
||||
|
||||
if (!changed) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (Object.keys(parsed).length === 0) {
|
||||
fs.rmSync(pathname, { force: true });
|
||||
return;
|
||||
}
|
||||
|
||||
fs.writeFileSync(pathname, `${JSON.stringify(parsed, null, 2)}\n`, "utf8");
|
||||
fs.chmodSync(pathname, 0o600);
|
||||
}
|
||||
|
||||
function createAuthStorage(AuthStorageLike: unknown, path: string, creds: PiCredentialMap) {
|
||||
const withInMemory = AuthStorageLike as { inMemory?: (data?: unknown) => unknown };
|
||||
if (typeof withInMemory.inMemory === "function") {
|
||||
return withInMemory.inMemory(creds) as AuthStorage;
|
||||
}
|
||||
|
||||
const withFromStorage = AuthStorageLike as {
|
||||
fromStorage?: (storage: unknown) => unknown;
|
||||
};
|
||||
if (typeof withFromStorage.fromStorage === "function") {
|
||||
const backend = new InMemoryAuthStorageBackend();
|
||||
backend.withLock(() => ({
|
||||
result: undefined,
|
||||
next: JSON.stringify(creds, null, 2),
|
||||
}));
|
||||
return withFromStorage.fromStorage(backend) as AuthStorage;
|
||||
}
|
||||
|
||||
const withFactory = AuthStorageLike as { create?: (path: string) => unknown };
|
||||
const withRuntimeOverride = (
|
||||
typeof withFactory.create === "function"
|
||||
? withFactory.create(path)
|
||||
: new (AuthStorageLike as { new (path: string): unknown })(path)
|
||||
) as AuthStorage & {
|
||||
setRuntimeApiKey?: (provider: string, apiKey: string) => void;
|
||||
};
|
||||
if (typeof withRuntimeOverride.setRuntimeApiKey === "function") {
|
||||
for (const [provider, credential] of Object.entries(creds)) {
|
||||
if (credential.type === "api_key") {
|
||||
withRuntimeOverride.setRuntimeApiKey(provider, credential.key);
|
||||
continue;
|
||||
}
|
||||
withRuntimeOverride.setRuntimeApiKey(provider, credential.access);
|
||||
}
|
||||
}
|
||||
return withRuntimeOverride;
|
||||
}
|
||||
|
||||
function convertAuthProfileCredential(cred: AuthProfileCredential): PiCredential | null {
|
||||
if (cred.type === "api_key") {
|
||||
const key = typeof cred.key === "string" ? cred.key.trim() : "";
|
||||
if (!key) {
|
||||
return null;
|
||||
}
|
||||
return { type: "api_key", key };
|
||||
}
|
||||
|
||||
if (cred.type === "token") {
|
||||
const token = typeof cred.token === "string" ? cred.token.trim() : "";
|
||||
if (!token) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
typeof cred.expires === "number" &&
|
||||
Number.isFinite(cred.expires) &&
|
||||
Date.now() >= cred.expires
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return { type: "api_key", key: token };
|
||||
}
|
||||
|
||||
if (cred.type === "oauth") {
|
||||
const access = typeof cred.access === "string" ? cred.access.trim() : "";
|
||||
const refresh = typeof cred.refresh === "string" ? cred.refresh.trim() : "";
|
||||
if (!access || !refresh || !Number.isFinite(cred.expires) || cred.expires <= 0) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
type: "oauth",
|
||||
access,
|
||||
refresh,
|
||||
expires: cred.expires,
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function resolvePiCredentials(agentDir: string): PiCredentialMap {
|
||||
const store = ensureAuthProfileStore(agentDir, { allowKeychainPrompt: false });
|
||||
const credentials: PiCredentialMap = {};
|
||||
for (const credential of Object.values(store.profiles)) {
|
||||
const provider = normalizeProviderId(String(credential.provider ?? "")).trim();
|
||||
if (!provider || credentials[provider]) {
|
||||
continue;
|
||||
}
|
||||
const converted = convertAuthProfileCredential(credential);
|
||||
if (converted) {
|
||||
credentials[provider] = converted;
|
||||
}
|
||||
}
|
||||
return credentials;
|
||||
}
|
||||
|
||||
// Compatibility helpers for pi-coding-agent 0.50+ (discover* helpers removed).
|
||||
export function discoverAuthStorage(agentDir: string): AuthStorage {
|
||||
return createAuthStorage(AuthStorage, path.join(agentDir, "auth.json"));
|
||||
const credentials = resolvePiCredentials(agentDir);
|
||||
const authPath = path.join(agentDir, "auth.json");
|
||||
scrubLegacyStaticAuthJsonEntries(authPath);
|
||||
return createAuthStorage(AuthStorage, authPath, credentials);
|
||||
}
|
||||
|
||||
export function discoverModels(authStorage: AuthStorage, agentDir: string): ModelRegistry {
|
||||
|
||||
@@ -105,9 +105,10 @@ function applySkillConfigEnvOverrides(params: {
|
||||
}
|
||||
}
|
||||
|
||||
if (normalizedPrimaryEnv && skillConfig.apiKey && !process.env[normalizedPrimaryEnv]) {
|
||||
const resolvedApiKey = typeof skillConfig.apiKey === "string" ? skillConfig.apiKey.trim() : "";
|
||||
if (normalizedPrimaryEnv && resolvedApiKey && !process.env[normalizedPrimaryEnv]) {
|
||||
if (!pendingOverrides[normalizedPrimaryEnv]) {
|
||||
pendingOverrides[normalizedPrimaryEnv] = skillConfig.apiKey;
|
||||
pendingOverrides[normalizedPrimaryEnv] = resolvedApiKey;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import type { OpenClawConfig } from "../config/config.js";
|
||||
import { loadConfig, writeConfigFile } from "../config/config.js";
|
||||
import { resolveStateDir } from "../config/paths.js";
|
||||
import { resolveArchiveKind } from "../infra/archive.js";
|
||||
import { enablePluginInConfig } from "../plugins/enable.js";
|
||||
import { installPluginFromNpmSpec, installPluginFromPath } from "../plugins/install.js";
|
||||
import { recordPluginInstall } from "../plugins/installs.js";
|
||||
import { clearPluginManifestRegistryCache } from "../plugins/manifest-registry.js";
|
||||
@@ -136,6 +135,22 @@ function createPluginInstallLogger(): { info: (msg: string) => void; warn: (msg:
|
||||
};
|
||||
}
|
||||
|
||||
function enablePluginInConfig(config: OpenClawConfig, pluginId: string): OpenClawConfig {
|
||||
return {
|
||||
...config,
|
||||
plugins: {
|
||||
...config.plugins,
|
||||
entries: {
|
||||
...config.plugins?.entries,
|
||||
[pluginId]: {
|
||||
...(config.plugins?.entries?.[pluginId] as object | undefined),
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function logSlotWarnings(warnings: string[]) {
|
||||
if (warnings.length === 0) {
|
||||
return;
|
||||
@@ -337,21 +352,24 @@ export function registerPluginsCli(program: Command) {
|
||||
.argument("<id>", "Plugin id")
|
||||
.action(async (id: string) => {
|
||||
const cfg = loadConfig();
|
||||
const enableResult = enablePluginInConfig(cfg, id);
|
||||
let next: OpenClawConfig = enableResult.config;
|
||||
let next: OpenClawConfig = {
|
||||
...cfg,
|
||||
plugins: {
|
||||
...cfg.plugins,
|
||||
entries: {
|
||||
...cfg.plugins?.entries,
|
||||
[id]: {
|
||||
...(cfg.plugins?.entries as Record<string, { enabled?: boolean }> | undefined)?.[id],
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const slotResult = applySlotSelectionForPlugin(next, id);
|
||||
next = slotResult.config;
|
||||
await writeConfigFile(next);
|
||||
logSlotWarnings(slotResult.warnings);
|
||||
if (enableResult.enabled) {
|
||||
defaultRuntime.log(`Enabled plugin "${id}". Restart the gateway to apply.`);
|
||||
return;
|
||||
}
|
||||
defaultRuntime.log(
|
||||
theme.warn(
|
||||
`Plugin "${id}" could not be enabled (${enableResult.reason ?? "unknown reason"}).`,
|
||||
),
|
||||
);
|
||||
defaultRuntime.log(`Enabled plugin "${id}". Restart the gateway to apply.`);
|
||||
});
|
||||
|
||||
plugins
|
||||
@@ -550,7 +568,7 @@ export function registerPluginsCli(program: Command) {
|
||||
},
|
||||
},
|
||||
probe.pluginId,
|
||||
).config;
|
||||
);
|
||||
next = recordPluginInstall(next, {
|
||||
pluginId: probe.pluginId,
|
||||
source: "path",
|
||||
@@ -579,7 +597,7 @@ export function registerPluginsCli(program: Command) {
|
||||
// force a rescan so config validation sees the freshly installed plugin.
|
||||
clearPluginManifestRegistryCache();
|
||||
|
||||
let next = enablePluginInConfig(cfg, result.pluginId).config;
|
||||
let next = enablePluginInConfig(cfg, result.pluginId);
|
||||
const source: "archive" | "path" = resolveArchiveKind(resolved) ? "archive" : "path";
|
||||
next = recordPluginInstall(next, {
|
||||
pluginId: result.pluginId,
|
||||
@@ -630,7 +648,7 @@ export function registerPluginsCli(program: Command) {
|
||||
// Ensure config validation sees newly installed plugin(s) even if the cache was warmed at startup.
|
||||
clearPluginManifestRegistryCache();
|
||||
|
||||
let next = enablePluginInConfig(cfg, result.pluginId).config;
|
||||
let next = enablePluginInConfig(cfg, result.pluginId);
|
||||
const resolvedSpec = result.npmResolution?.resolvedSpec;
|
||||
const recordSpec = opts.pin && resolvedSpec ? resolvedSpec : raw;
|
||||
if (opts.pin && !resolvedSpec) {
|
||||
|
||||
@@ -260,6 +260,15 @@ const entries: SubCliEntry[] = [
|
||||
mod.registerSecurityCli(program);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "secrets",
|
||||
description: "Secrets runtime reload controls",
|
||||
hasSubcommands: true,
|
||||
register: async (program) => {
|
||||
const mod = await import("../secrets-cli.js");
|
||||
mod.registerSecretsCli(program);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "skills",
|
||||
description: "List and inspect available skills",
|
||||
|
||||
96
src/cli/secrets-cli.test.ts
Normal file
96
src/cli/secrets-cli.test.ts
Normal file
@@ -0,0 +1,96 @@
|
||||
import { Command } from "commander";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { createCliRuntimeCapture } from "./test-runtime-capture.js";
|
||||
|
||||
const callGatewayFromCli = vi.fn();
|
||||
const runSecretsMigration = vi.fn();
|
||||
const rollbackSecretsMigration = vi.fn();
|
||||
|
||||
const { defaultRuntime, runtimeLogs, runtimeErrors, resetRuntimeCapture } =
|
||||
createCliRuntimeCapture();
|
||||
|
||||
vi.mock("./gateway-rpc.js", () => ({
|
||||
addGatewayClientOptions: (cmd: Command) => cmd,
|
||||
callGatewayFromCli: (method: string, opts: unknown, params?: unknown, extra?: unknown) =>
|
||||
callGatewayFromCli(method, opts, params, extra),
|
||||
}));
|
||||
|
||||
vi.mock("../runtime.js", () => ({
|
||||
defaultRuntime,
|
||||
}));
|
||||
|
||||
vi.mock("../secrets/migrate.js", () => ({
|
||||
runSecretsMigration: (options: unknown) => runSecretsMigration(options),
|
||||
rollbackSecretsMigration: (options: unknown) => rollbackSecretsMigration(options),
|
||||
}));
|
||||
|
||||
const { registerSecretsCli } = await import("./secrets-cli.js");
|
||||
|
||||
describe("secrets CLI", () => {
|
||||
const createProgram = () => {
|
||||
const program = new Command();
|
||||
program.exitOverride();
|
||||
registerSecretsCli(program);
|
||||
return program;
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
resetRuntimeCapture();
|
||||
callGatewayFromCli.mockReset();
|
||||
runSecretsMigration.mockReset();
|
||||
rollbackSecretsMigration.mockReset();
|
||||
});
|
||||
|
||||
it("calls secrets.reload and prints human output", async () => {
|
||||
callGatewayFromCli.mockResolvedValue({ ok: true, warningCount: 1 });
|
||||
await createProgram().parseAsync(["secrets", "reload"], { from: "user" });
|
||||
expect(callGatewayFromCli).toHaveBeenCalledWith(
|
||||
"secrets.reload",
|
||||
expect.anything(),
|
||||
undefined,
|
||||
expect.objectContaining({ expectFinal: false }),
|
||||
);
|
||||
expect(runtimeLogs.at(-1)).toBe("Secrets reloaded with 1 warning(s).");
|
||||
expect(runtimeErrors).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("prints JSON when requested", async () => {
|
||||
callGatewayFromCli.mockResolvedValue({ ok: true, warningCount: 0 });
|
||||
await createProgram().parseAsync(["secrets", "reload", "--json"], { from: "user" });
|
||||
expect(runtimeLogs.at(-1)).toContain('"ok": true');
|
||||
});
|
||||
|
||||
it("runs secrets migrate as dry-run by default", async () => {
|
||||
runSecretsMigration.mockResolvedValue({
|
||||
mode: "dry-run",
|
||||
changed: true,
|
||||
secretsFilePath: "/tmp/secrets.enc.json",
|
||||
counters: { secretsWritten: 3 },
|
||||
changedFiles: ["/tmp/openclaw.json"],
|
||||
});
|
||||
|
||||
await createProgram().parseAsync(["secrets", "migrate"], { from: "user" });
|
||||
|
||||
expect(runSecretsMigration).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ write: false, scrubEnv: true }),
|
||||
);
|
||||
expect(runtimeLogs.at(-1)).toContain("dry run");
|
||||
});
|
||||
|
||||
it("runs rollback when --rollback is provided", async () => {
|
||||
rollbackSecretsMigration.mockResolvedValue({
|
||||
backupId: "20260221T010203Z",
|
||||
restoredFiles: 2,
|
||||
deletedFiles: 1,
|
||||
});
|
||||
|
||||
await createProgram().parseAsync(["secrets", "migrate", "--rollback", "20260221T010203Z"], {
|
||||
from: "user",
|
||||
});
|
||||
|
||||
expect(rollbackSecretsMigration).toHaveBeenCalledWith({
|
||||
backupId: "20260221T010203Z",
|
||||
});
|
||||
expect(runtimeLogs.at(-1)).toContain("rollback complete");
|
||||
});
|
||||
});
|
||||
121
src/cli/secrets-cli.ts
Normal file
121
src/cli/secrets-cli.ts
Normal file
@@ -0,0 +1,121 @@
|
||||
import type { Command } from "commander";
|
||||
import { danger } from "../globals.js";
|
||||
import { defaultRuntime } from "../runtime.js";
|
||||
import {
|
||||
rollbackSecretsMigration,
|
||||
runSecretsMigration,
|
||||
type SecretsMigrationRollbackResult,
|
||||
type SecretsMigrationRunResult,
|
||||
} from "../secrets/migrate.js";
|
||||
import { formatDocsLink } from "../terminal/links.js";
|
||||
import { theme } from "../terminal/theme.js";
|
||||
import { addGatewayClientOptions, callGatewayFromCli, type GatewayRpcOpts } from "./gateway-rpc.js";
|
||||
|
||||
type SecretsReloadOptions = GatewayRpcOpts & { json?: boolean };
|
||||
type SecretsMigrateOptions = {
|
||||
write?: boolean;
|
||||
rollback?: string;
|
||||
scrubEnv?: boolean;
|
||||
json?: boolean;
|
||||
};
|
||||
|
||||
function printMigrationResult(
|
||||
result: SecretsMigrationRunResult | SecretsMigrationRollbackResult,
|
||||
json: boolean,
|
||||
): void {
|
||||
if (json) {
|
||||
defaultRuntime.log(JSON.stringify(result, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
if ("restoredFiles" in result) {
|
||||
defaultRuntime.log(
|
||||
`Secrets rollback complete for backup ${result.backupId}. Restored ${result.restoredFiles} file(s), deleted ${result.deletedFiles} file(s).`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (result.mode === "dry-run") {
|
||||
if (!result.changed) {
|
||||
defaultRuntime.log("Secrets migrate dry run: no changes needed.");
|
||||
return;
|
||||
}
|
||||
defaultRuntime.log(
|
||||
`Secrets migrate dry run: ${result.changedFiles.length} file(s) would change, ${result.counters.secretsWritten} secret value(s) would move to ${result.secretsFilePath}.`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!result.changed) {
|
||||
defaultRuntime.log("Secrets migrate: no changes applied.");
|
||||
return;
|
||||
}
|
||||
defaultRuntime.log(
|
||||
`Secrets migrated. Backup: ${result.backupId}. Moved ${result.counters.secretsWritten} secret value(s) into ${result.secretsFilePath}.`,
|
||||
);
|
||||
}
|
||||
|
||||
export function registerSecretsCli(program: Command) {
|
||||
const secrets = program
|
||||
.command("secrets")
|
||||
.description("Secrets runtime controls")
|
||||
.addHelpText(
|
||||
"after",
|
||||
() =>
|
||||
`\n${theme.muted("Docs:")} ${formatDocsLink("/gateway/security", "docs.openclaw.ai/gateway/security")}\n`,
|
||||
);
|
||||
|
||||
addGatewayClientOptions(
|
||||
secrets
|
||||
.command("reload")
|
||||
.description("Re-resolve secret references and atomically swap runtime snapshot")
|
||||
.option("--json", "Output JSON", false),
|
||||
).action(async (opts: SecretsReloadOptions) => {
|
||||
try {
|
||||
const result = await callGatewayFromCli("secrets.reload", opts, undefined, {
|
||||
expectFinal: false,
|
||||
});
|
||||
if (opts.json) {
|
||||
defaultRuntime.log(JSON.stringify(result, null, 2));
|
||||
return;
|
||||
}
|
||||
const warningCount = Number(
|
||||
(result as { warningCount?: unknown } | undefined)?.warningCount ?? 0,
|
||||
);
|
||||
if (Number.isFinite(warningCount) && warningCount > 0) {
|
||||
defaultRuntime.log(`Secrets reloaded with ${warningCount} warning(s).`);
|
||||
return;
|
||||
}
|
||||
defaultRuntime.log("Secrets reloaded.");
|
||||
} catch (err) {
|
||||
defaultRuntime.error(danger(String(err)));
|
||||
defaultRuntime.exit(1);
|
||||
}
|
||||
});
|
||||
|
||||
secrets
|
||||
.command("migrate")
|
||||
.description("Migrate plaintext secrets to file-backed SecretRefs (sops)")
|
||||
.option("--write", "Apply migration changes (default is dry-run)", false)
|
||||
.option("--rollback <backup-id>", "Rollback a previous migration backup id")
|
||||
.option("--no-scrub-env", "Keep matching plaintext values in ~/.openclaw/.env")
|
||||
.option("--json", "Output JSON", false)
|
||||
.action(async (opts: SecretsMigrateOptions) => {
|
||||
try {
|
||||
if (typeof opts.rollback === "string" && opts.rollback.trim()) {
|
||||
const result = await rollbackSecretsMigration({ backupId: opts.rollback.trim() });
|
||||
printMigrationResult(result, Boolean(opts.json));
|
||||
return;
|
||||
}
|
||||
|
||||
const result = await runSecretsMigration({
|
||||
write: Boolean(opts.write),
|
||||
scrubEnv: opts.scrubEnv ?? true,
|
||||
});
|
||||
printMigrationResult(result, Boolean(opts.json));
|
||||
} catch (err) {
|
||||
defaultRuntime.error(danger(String(err)));
|
||||
defaultRuntime.exit(1);
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -95,7 +95,7 @@ describe("models list auth-profile sync", () => {
|
||||
const openrouter = getProviderRow(String(runtime.log.mock.calls[0]?.[0]), "openrouter/");
|
||||
expect(openrouter).toBeDefined();
|
||||
expect(openrouter?.available).toBe(true);
|
||||
expect(await pathExists(authPath)).toBe(true);
|
||||
expect(await pathExists(authPath)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -6,9 +6,6 @@ let toModelRow: typeof import("./models/list.registry.js").toModelRow;
|
||||
|
||||
const loadConfig = vi.fn();
|
||||
const ensureOpenClawModelsJson = vi.fn().mockResolvedValue(undefined);
|
||||
const ensurePiAuthJsonFromAuthProfiles = vi
|
||||
.fn()
|
||||
.mockResolvedValue({ wrote: false, authPath: "/tmp/openclaw-agent/auth.json" });
|
||||
const resolveOpenClawAgentDir = vi.fn().mockReturnValue("/tmp/openclaw-agent");
|
||||
const ensureAuthProfileStore = vi.fn().mockReturnValue({ version: 1, profiles: {} });
|
||||
const listProfilesForProvider = vi.fn().mockReturnValue([]);
|
||||
@@ -38,10 +35,6 @@ vi.mock("../agents/models-config.js", () => ({
|
||||
ensureOpenClawModelsJson,
|
||||
}));
|
||||
|
||||
vi.mock("../agents/pi-auth-json.js", () => ({
|
||||
ensurePiAuthJsonFromAuthProfiles,
|
||||
}));
|
||||
|
||||
vi.mock("../agents/agent-paths.js", () => ({
|
||||
resolveOpenClawAgentDir,
|
||||
}));
|
||||
@@ -110,7 +103,6 @@ beforeEach(() => {
|
||||
modelRegistryState.getAllError = undefined;
|
||||
modelRegistryState.getAvailableError = undefined;
|
||||
listProfilesForProvider.mockReturnValue([]);
|
||||
ensurePiAuthJsonFromAuthProfiles.mockClear();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
@@ -279,13 +271,12 @@ describe("models list/status", () => {
|
||||
({ loadModelRegistry, toModelRow } = await import("./models/list.registry.js"));
|
||||
});
|
||||
|
||||
it("models list syncs auth-profiles into auth.json before availability checks", async () => {
|
||||
it("models list runs model discovery without auth.json sync", async () => {
|
||||
setDefaultZaiRegistry();
|
||||
const runtime = makeRuntime();
|
||||
|
||||
await modelsListCommand({ all: true, json: true }, runtime);
|
||||
|
||||
expect(ensurePiAuthJsonFromAuthProfiles).toHaveBeenCalledWith("/tmp/openclaw-agent");
|
||||
expect(runtime.error).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("models list outputs canonical zai key for configured z.ai model", async () => {
|
||||
|
||||
@@ -12,7 +12,6 @@ import {
|
||||
resolveForwardCompatModel,
|
||||
} from "../../agents/model-forward-compat.js";
|
||||
import { ensureOpenClawModelsJson } from "../../agents/models-config.js";
|
||||
import { ensurePiAuthJsonFromAuthProfiles } from "../../agents/pi-auth-json.js";
|
||||
import type { ModelRegistry } from "../../agents/pi-model-discovery.js";
|
||||
import { discoverAuthStorage, discoverModels } from "../../agents/pi-model-discovery.js";
|
||||
import type { OpenClawConfig } from "../../config/config.js";
|
||||
@@ -102,7 +101,6 @@ function loadAvailableModels(registry: ModelRegistry): Model<Api>[] {
|
||||
export async function loadModelRegistry(cfg: OpenClawConfig) {
|
||||
await ensureOpenClawModelsJson(cfg);
|
||||
const agentDir = resolveOpenClawAgentDir();
|
||||
await ensurePiAuthJsonFromAuthProfiles(agentDir);
|
||||
const authStorage = discoverAuthStorage(agentDir);
|
||||
const registry = discoverModels(authStorage, agentDir);
|
||||
const appended = appendAntigravityForwardCompatModels(registry.getAll(), registry);
|
||||
|
||||
@@ -4,6 +4,7 @@ import type { OpenClawConfig } from "../config/config.js";
|
||||
import type { ModelProviderConfig } from "../config/types.models.js";
|
||||
import type { RuntimeEnv } from "../runtime.js";
|
||||
import { fetchWithTimeout } from "../utils/fetch-timeout.js";
|
||||
import { normalizeOptionalSecretInput } from "../utils/normalize-secret-input.js";
|
||||
import type { WizardPrompter } from "../wizard/prompts.js";
|
||||
import { applyPrimaryModel } from "./model-picker.js";
|
||||
import { normalizeAlias } from "./models/shared.js";
|
||||
@@ -521,7 +522,7 @@ export function applyCustomApiConfig(params: ApplyCustomApiConfigParams): Custom
|
||||
const mergedModels = hasModel ? existingModels : [...existingModels, nextModel];
|
||||
const { apiKey: existingApiKey, ...existingProviderRest } = existingProvider ?? {};
|
||||
const normalizedApiKey =
|
||||
params.apiKey?.trim() || (existingApiKey ? existingApiKey.trim() : undefined);
|
||||
normalizeOptionalSecretInput(params.apiKey) ?? normalizeOptionalSecretInput(existingApiKey);
|
||||
|
||||
let config: OpenClawConfig = {
|
||||
...params.config,
|
||||
|
||||
74
src/config/config.secrets-schema.test.ts
Normal file
74
src/config/config.secrets-schema.test.ts
Normal file
@@ -0,0 +1,74 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { validateConfigObjectRaw } from "./validation.js";
|
||||
|
||||
describe("config secret refs schema", () => {
|
||||
it("accepts top-level secrets sources and model apiKey refs", () => {
|
||||
const result = validateConfigObjectRaw({
|
||||
secrets: {
|
||||
sources: {
|
||||
env: { type: "env" },
|
||||
file: { type: "sops", path: "~/.openclaw/secrets.enc.json", timeoutMs: 10_000 },
|
||||
},
|
||||
},
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "env", id: "OPENAI_API_KEY" },
|
||||
models: [{ id: "gpt-5", name: "gpt-5" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
});
|
||||
|
||||
it("accepts googlechat serviceAccount refs", () => {
|
||||
const result = validateConfigObjectRaw({
|
||||
channels: {
|
||||
googlechat: {
|
||||
serviceAccountRef: { source: "file", id: "/channels/googlechat/serviceAccount" },
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
});
|
||||
|
||||
it("accepts skills entry apiKey refs", () => {
|
||||
const result = validateConfigObjectRaw({
|
||||
skills: {
|
||||
entries: {
|
||||
"review-pr": {
|
||||
enabled: true,
|
||||
apiKey: { source: "env", id: "SKILL_REVIEW_PR_API_KEY" },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects invalid secret ref id", () => {
|
||||
const result = validateConfigObjectRaw({
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "env", id: "bad id with spaces" },
|
||||
models: [{ id: "gpt-5", name: "gpt-5" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) {
|
||||
expect(
|
||||
result.issues.some((issue) => issue.path.includes("models.providers.openai.apiKey")),
|
||||
).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,11 +1,14 @@
|
||||
export {
|
||||
clearConfigCache,
|
||||
clearRuntimeConfigSnapshot,
|
||||
createConfigIO,
|
||||
getRuntimeConfigSnapshot,
|
||||
loadConfig,
|
||||
parseConfigJson5,
|
||||
readConfigFileSnapshot,
|
||||
readConfigFileSnapshotForWrite,
|
||||
resolveConfigSnapshotHash,
|
||||
setRuntimeConfigSnapshot,
|
||||
writeConfigFile,
|
||||
} from "./io.js";
|
||||
export { migrateLegacyConfig } from "./legacy-migrate.js";
|
||||
|
||||
63
src/config/io.runtime-snapshot-write.test.ts
Normal file
63
src/config/io.runtime-snapshot-write.test.ts
Normal file
@@ -0,0 +1,63 @@
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { withTempHome } from "./home-env.test-harness.js";
|
||||
import {
|
||||
clearConfigCache,
|
||||
clearRuntimeConfigSnapshot,
|
||||
loadConfig,
|
||||
setRuntimeConfigSnapshot,
|
||||
writeConfigFile,
|
||||
} from "./io.js";
|
||||
import type { OpenClawConfig } from "./types.js";
|
||||
|
||||
describe("runtime config snapshot writes", () => {
|
||||
it("preserves source secret refs when writeConfigFile receives runtime-resolved config", async () => {
|
||||
await withTempHome("openclaw-config-runtime-write-", async (home) => {
|
||||
const configPath = path.join(home, ".openclaw", "openclaw.json");
|
||||
const sourceConfig: OpenClawConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "env", id: "OPENAI_API_KEY" },
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const runtimeConfig: OpenClawConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: "sk-runtime-resolved",
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
await fs.mkdir(path.dirname(configPath), { recursive: true });
|
||||
await fs.writeFile(configPath, `${JSON.stringify(sourceConfig, null, 2)}\n`, "utf8");
|
||||
|
||||
try {
|
||||
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
|
||||
expect(loadConfig().models?.providers?.openai?.apiKey).toBe("sk-runtime-resolved");
|
||||
|
||||
await writeConfigFile(loadConfig());
|
||||
|
||||
const persisted = JSON.parse(await fs.readFile(configPath, "utf8")) as {
|
||||
models?: { providers?: { openai?: { apiKey?: unknown } } };
|
||||
};
|
||||
expect(persisted.models?.providers?.openai?.apiKey).toEqual({
|
||||
source: "env",
|
||||
id: "OPENAI_API_KEY",
|
||||
});
|
||||
} finally {
|
||||
clearRuntimeConfigSnapshot();
|
||||
clearConfigCache();
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1061,6 +1061,8 @@ let configCache: {
|
||||
expiresAt: number;
|
||||
config: OpenClawConfig;
|
||||
} | null = null;
|
||||
let runtimeConfigSnapshot: OpenClawConfig | null = null;
|
||||
let runtimeConfigSourceSnapshot: OpenClawConfig | null = null;
|
||||
|
||||
function resolveConfigCacheMs(env: NodeJS.ProcessEnv): number {
|
||||
const raw = env.OPENCLAW_CONFIG_CACHE_MS?.trim();
|
||||
@@ -1088,7 +1090,29 @@ export function clearConfigCache(): void {
|
||||
configCache = null;
|
||||
}
|
||||
|
||||
export function setRuntimeConfigSnapshot(
|
||||
config: OpenClawConfig,
|
||||
sourceConfig?: OpenClawConfig,
|
||||
): void {
|
||||
runtimeConfigSnapshot = config;
|
||||
runtimeConfigSourceSnapshot = sourceConfig ?? null;
|
||||
clearConfigCache();
|
||||
}
|
||||
|
||||
export function clearRuntimeConfigSnapshot(): void {
|
||||
runtimeConfigSnapshot = null;
|
||||
runtimeConfigSourceSnapshot = null;
|
||||
clearConfigCache();
|
||||
}
|
||||
|
||||
export function getRuntimeConfigSnapshot(): OpenClawConfig | null {
|
||||
return runtimeConfigSnapshot;
|
||||
}
|
||||
|
||||
export function loadConfig(): OpenClawConfig {
|
||||
if (runtimeConfigSnapshot) {
|
||||
return runtimeConfigSnapshot;
|
||||
}
|
||||
const io = createConfigIO();
|
||||
const configPath = io.configPath;
|
||||
const now = Date.now();
|
||||
@@ -1125,9 +1149,14 @@ export async function writeConfigFile(
|
||||
options: ConfigWriteOptions = {},
|
||||
): Promise<void> {
|
||||
const io = createConfigIO();
|
||||
let nextCfg = cfg;
|
||||
if (runtimeConfigSnapshot && runtimeConfigSourceSnapshot) {
|
||||
const runtimePatch = createMergePatch(runtimeConfigSnapshot, cfg);
|
||||
nextCfg = coerceConfig(applyMergePatch(runtimeConfigSourceSnapshot, runtimePatch));
|
||||
}
|
||||
const sameConfigPath =
|
||||
options.expectedConfigPath === undefined || options.expectedConfigPath === io.configPath;
|
||||
await io.writeConfigFile(cfg, {
|
||||
await io.writeConfigFile(nextCfg, {
|
||||
envSnapshotForRestore: sameConfigPath ? options.envSnapshotForRestore : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -74,7 +74,6 @@ describe("redactConfigSnapshot", () => {
|
||||
},
|
||||
shortSecret: { token: "short" },
|
||||
});
|
||||
|
||||
const result = redactConfigSnapshot(snapshot);
|
||||
const cfg = result.config as typeof snapshot.config;
|
||||
|
||||
@@ -91,6 +90,45 @@ describe("redactConfigSnapshot", () => {
|
||||
expect(cfg.shortSecret.token).toBe(REDACTED_SENTINEL);
|
||||
});
|
||||
|
||||
it("redacts googlechat serviceAccount object payloads", () => {
|
||||
const snapshot = makeSnapshot({
|
||||
channels: {
|
||||
googlechat: {
|
||||
serviceAccount: {
|
||||
type: "service_account",
|
||||
client_email: "bot@example.iam.gserviceaccount.com",
|
||||
private_key: "-----BEGIN PRIVATE KEY-----secret-----END PRIVATE KEY-----",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const result = redactConfigSnapshot(snapshot);
|
||||
const channels = result.config.channels as Record<string, Record<string, unknown>>;
|
||||
expect(channels.googlechat.serviceAccount).toBe(REDACTED_SENTINEL);
|
||||
});
|
||||
|
||||
it("redacts object-valued apiKey refs in model providers", () => {
|
||||
const snapshot = makeSnapshot({
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: { source: "env", id: "OPENAI_API_KEY" },
|
||||
baseUrl: "https://api.openai.com",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const result = redactConfigSnapshot(snapshot);
|
||||
const models = result.config.models as Record<string, Record<string, Record<string, unknown>>>;
|
||||
expect(models.providers.openai.apiKey).toEqual({
|
||||
source: REDACTED_SENTINEL,
|
||||
id: REDACTED_SENTINEL,
|
||||
});
|
||||
expect(models.providers.openai.baseUrl).toBe("https://api.openai.com");
|
||||
});
|
||||
|
||||
it("preserves non-sensitive fields", () => {
|
||||
const snapshot = makeSnapshot({
|
||||
ui: { seamColor: "#0088cc" },
|
||||
|
||||
@@ -17,6 +17,31 @@ function isEnvVarPlaceholder(value: string): boolean {
|
||||
return ENV_VAR_PLACEHOLDER_PATTERN.test(value.trim());
|
||||
}
|
||||
|
||||
function isWholeObjectSensitivePath(path: string): boolean {
|
||||
const lowered = path.toLowerCase();
|
||||
return lowered.endsWith("serviceaccount") || lowered.endsWith("serviceaccountref");
|
||||
}
|
||||
|
||||
function collectSensitiveStrings(value: unknown, values: string[]): void {
|
||||
if (typeof value === "string") {
|
||||
if (!isEnvVarPlaceholder(value)) {
|
||||
values.push(value);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) {
|
||||
collectSensitiveStrings(item, values);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
for (const item of Object.values(value as Record<string, unknown>)) {
|
||||
collectSensitiveStrings(item, values);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function isExtensionPath(path: string): boolean {
|
||||
return (
|
||||
path === "plugins" ||
|
||||
@@ -159,7 +184,19 @@ function redactObjectWithLookup(
|
||||
result[key] = REDACTED_SENTINEL;
|
||||
values.push(value);
|
||||
} else if (typeof value === "object" && value !== null) {
|
||||
result[key] = redactObjectWithLookup(value, lookup, candidate, values, hints);
|
||||
if (hints[candidate]?.sensitive === true && !Array.isArray(value)) {
|
||||
collectSensitiveStrings(value, values);
|
||||
result[key] = REDACTED_SENTINEL;
|
||||
} else {
|
||||
result[key] = redactObjectWithLookup(value, lookup, candidate, values, hints);
|
||||
}
|
||||
} else if (
|
||||
hints[candidate]?.sensitive === true &&
|
||||
value !== undefined &&
|
||||
value !== null
|
||||
) {
|
||||
// Keep primitives at explicitly-sensitive paths fully redacted.
|
||||
result[key] = REDACTED_SENTINEL;
|
||||
}
|
||||
break;
|
||||
}
|
||||
@@ -228,6 +265,16 @@ function redactObjectGuessing(
|
||||
) {
|
||||
result[key] = REDACTED_SENTINEL;
|
||||
values.push(value);
|
||||
} else if (
|
||||
!isExplicitlyNonSensitivePath(hints, [dotPath, wildcardPath]) &&
|
||||
isSensitivePath(dotPath) &&
|
||||
isWholeObjectSensitivePath(dotPath) &&
|
||||
value &&
|
||||
typeof value === "object" &&
|
||||
!Array.isArray(value)
|
||||
) {
|
||||
collectSensitiveStrings(value, values);
|
||||
result[key] = REDACTED_SENTINEL;
|
||||
} else if (typeof value === "object" && value !== null) {
|
||||
result[key] = redactObjectGuessing(value, dotPath, values, hints);
|
||||
} else {
|
||||
|
||||
@@ -236,13 +236,6 @@ export const FIELD_HELP: Record<string, string> = {
|
||||
"memory.backend": 'Memory backend ("builtin" for OpenClaw embeddings, "qmd" for QMD sidecar).',
|
||||
"memory.citations": 'Default citation behavior ("auto", "on", or "off").',
|
||||
"memory.qmd.command": "Path to the qmd binary (default: resolves from PATH).",
|
||||
"memory.qmd.mcporter":
|
||||
"Optional: route QMD searches through mcporter (MCP runtime) instead of spawning `qmd` per query. Intended to avoid per-search cold starts when QMD models are large.",
|
||||
"memory.qmd.mcporter.enabled": "Enable mcporter-backed QMD searches (default: false).",
|
||||
"memory.qmd.mcporter.serverName":
|
||||
"mcporter server name to call (default: qmd). Server should run `qmd mcp` with lifecycle keep-alive.",
|
||||
"memory.qmd.mcporter.startDaemon":
|
||||
"Start `mcporter daemon start` automatically when enabled (default: true).",
|
||||
"memory.qmd.includeDefaultMemory":
|
||||
"Whether to automatically index MEMORY.md + memory/**/*.md (default: true).",
|
||||
"memory.qmd.paths":
|
||||
|
||||
@@ -109,6 +109,7 @@ describe("mapSensitivePaths", () => {
|
||||
expect(hints["agents.defaults.memorySearch.remote.apiKey"]?.sensitive).toBe(true);
|
||||
expect(hints["agents.list[].memorySearch.remote.apiKey"]?.sensitive).toBe(true);
|
||||
expect(hints["channels.discord.accounts.*.token"]?.sensitive).toBe(true);
|
||||
expect(hints["channels.googlechat.serviceAccount"]?.sensitive).toBe(true);
|
||||
expect(hints["gateway.auth.token"]?.sensitive).toBe(true);
|
||||
expect(hints["skills.entries.*.apiKey"]?.sensitive).toBe(true);
|
||||
});
|
||||
|
||||
@@ -107,7 +107,13 @@ const NORMALIZED_SENSITIVE_KEY_WHITELIST_SUFFIXES = SENSITIVE_KEY_WHITELIST_SUFF
|
||||
suffix.toLowerCase(),
|
||||
);
|
||||
|
||||
const SENSITIVE_PATTERNS = [/token$/i, /password/i, /secret/i, /api.?key/i];
|
||||
const SENSITIVE_PATTERNS = [
|
||||
/token$/i,
|
||||
/password/i,
|
||||
/secret/i,
|
||||
/api.?key/i,
|
||||
/serviceaccount(?:ref)?$/i,
|
||||
];
|
||||
|
||||
function isWhitelistedSensitivePath(path: string): boolean {
|
||||
const lowerPath = path.toLowerCase();
|
||||
|
||||
@@ -5,6 +5,7 @@ import type {
|
||||
ReplyToMode,
|
||||
} from "./types.base.js";
|
||||
import type { DmConfig } from "./types.messages.js";
|
||||
import type { SecretRef } from "./types.secrets.js";
|
||||
|
||||
export type GoogleChatDmConfig = {
|
||||
/** If false, ignore all incoming Google Chat DMs. Default: true. */
|
||||
@@ -58,8 +59,10 @@ export type GoogleChatAccountConfig = {
|
||||
defaultTo?: string;
|
||||
/** Per-space configuration keyed by space id or name. */
|
||||
groups?: Record<string, GoogleChatGroupConfig>;
|
||||
/** Service account JSON (inline string or object). */
|
||||
serviceAccount?: string | Record<string, unknown>;
|
||||
/** Service account JSON (inline string, object, or secret reference). */
|
||||
serviceAccount?: string | Record<string, unknown> | SecretRef;
|
||||
/** Explicit secret reference for service account JSON. */
|
||||
serviceAccountRef?: SecretRef;
|
||||
/** Service account JSON file path. */
|
||||
serviceAccountFile?: string;
|
||||
/** Webhook audience type (app-url or project-number). */
|
||||
|
||||
@@ -12,7 +12,6 @@ export type MemoryConfig = {
|
||||
|
||||
export type MemoryQmdConfig = {
|
||||
command?: string;
|
||||
mcporter?: MemoryQmdMcporterConfig;
|
||||
searchMode?: MemoryQmdSearchMode;
|
||||
includeDefaultMemory?: boolean;
|
||||
paths?: MemoryQmdIndexPath[];
|
||||
@@ -22,20 +21,6 @@ export type MemoryQmdConfig = {
|
||||
scope?: SessionSendPolicyConfig;
|
||||
};
|
||||
|
||||
export type MemoryQmdMcporterConfig = {
|
||||
/**
|
||||
* Route QMD searches through mcporter (MCP runtime) instead of spawning `qmd` per query.
|
||||
* Requires:
|
||||
* - `mcporter` installed and on PATH
|
||||
* - A configured mcporter server that runs `qmd mcp` with `lifecycle: keep-alive`
|
||||
*/
|
||||
enabled?: boolean;
|
||||
/** mcporter server name (defaults to "qmd") */
|
||||
serverName?: string;
|
||||
/** Start the mcporter daemon automatically (defaults to true when enabled). */
|
||||
startDaemon?: boolean;
|
||||
};
|
||||
|
||||
export type MemoryQmdIndexPath = {
|
||||
path: string;
|
||||
name?: string;
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { SecretInput } from "./types.secrets.js";
|
||||
|
||||
export type ModelApi =
|
||||
| "openai-completions"
|
||||
| "openai-responses"
|
||||
@@ -43,7 +45,7 @@ export type ModelDefinitionConfig = {
|
||||
|
||||
export type ModelProviderConfig = {
|
||||
baseUrl: string;
|
||||
apiKey?: string;
|
||||
apiKey?: SecretInput;
|
||||
auth?: ModelProviderAuthMode;
|
||||
api?: ModelApi;
|
||||
headers?: Record<string, string>;
|
||||
|
||||
@@ -22,6 +22,7 @@ import type {
|
||||
import type { ModelsConfig } from "./types.models.js";
|
||||
import type { NodeHostConfig } from "./types.node-host.js";
|
||||
import type { PluginsConfig } from "./types.plugins.js";
|
||||
import type { SecretsConfig } from "./types.secrets.js";
|
||||
import type { SkillsConfig } from "./types.skills.js";
|
||||
import type { ToolsConfig } from "./types.tools.js";
|
||||
|
||||
@@ -75,6 +76,7 @@ export type OpenClawConfig = {
|
||||
avatar?: string;
|
||||
};
|
||||
};
|
||||
secrets?: SecretsConfig;
|
||||
skills?: SkillsConfig;
|
||||
plugins?: PluginsConfig;
|
||||
models?: ModelsConfig;
|
||||
|
||||
49
src/config/types.secrets.ts
Normal file
49
src/config/types.secrets.ts
Normal file
@@ -0,0 +1,49 @@
|
||||
export type SecretRefSource = "env" | "file";
|
||||
|
||||
/**
|
||||
* Stable identifier for a secret in a configured source.
|
||||
* Examples:
|
||||
* - env source: "OPENAI_API_KEY"
|
||||
* - file source: "/providers/openai/api_key" (JSON pointer)
|
||||
*/
|
||||
export type SecretRef = {
|
||||
source: SecretRefSource;
|
||||
id: string;
|
||||
};
|
||||
|
||||
export type SecretInput = string | SecretRef;
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
export function isSecretRef(value: unknown): value is SecretRef {
|
||||
if (!isRecord(value)) {
|
||||
return false;
|
||||
}
|
||||
if (Object.keys(value).length !== 2) {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
(value.source === "env" || value.source === "file") &&
|
||||
typeof value.id === "string" &&
|
||||
value.id.trim().length > 0
|
||||
);
|
||||
}
|
||||
|
||||
export type EnvSecretSourceConfig = {
|
||||
type?: "env";
|
||||
};
|
||||
|
||||
export type SopsSecretSourceConfig = {
|
||||
type: "sops";
|
||||
path: string;
|
||||
timeoutMs?: number;
|
||||
};
|
||||
|
||||
export type SecretsConfig = {
|
||||
sources?: {
|
||||
env?: EnvSecretSourceConfig;
|
||||
file?: SopsSecretSourceConfig;
|
||||
};
|
||||
};
|
||||
@@ -1,6 +1,8 @@
|
||||
import type { SecretInput } from "./types.secrets.js";
|
||||
|
||||
export type SkillConfig = {
|
||||
enabled?: boolean;
|
||||
apiKey?: string;
|
||||
apiKey?: SecretInput;
|
||||
env?: Record<string, string>;
|
||||
config?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
@@ -22,6 +22,7 @@ export * from "./types.msteams.js";
|
||||
export * from "./types.plugins.js";
|
||||
export * from "./types.queue.js";
|
||||
export * from "./types.sandbox.js";
|
||||
export * from "./types.secrets.js";
|
||||
export * from "./types.signal.js";
|
||||
export * from "./types.skills.js";
|
||||
export * from "./types.slack.js";
|
||||
|
||||
@@ -3,6 +3,49 @@ import { isSafeExecutableValue } from "../infra/exec-safety.js";
|
||||
import { createAllowDenyChannelRulesSchema } from "./zod-schema.allowdeny.js";
|
||||
import { sensitive } from "./zod-schema.sensitive.js";
|
||||
|
||||
const SECRET_REF_ID_PATTERN = /^[A-Za-z0-9_./:=-](?:[A-Za-z0-9_./:=~-]{0,127})$/;
|
||||
|
||||
export const SecretRefSchema = z
|
||||
.object({
|
||||
source: z.enum(["env", "file"]),
|
||||
id: z
|
||||
.string()
|
||||
.regex(
|
||||
SECRET_REF_ID_PATTERN,
|
||||
"Secret reference id must match /^[A-Za-z0-9_./:=-](?:[A-Za-z0-9_./:=~-]{0,127})$/",
|
||||
),
|
||||
})
|
||||
.strict();
|
||||
|
||||
export const SecretInputSchema = z.union([z.string(), SecretRefSchema]);
|
||||
|
||||
const SecretsEnvSourceSchema = z
|
||||
.object({
|
||||
type: z.literal("env").optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
const SecretsFileSourceSchema = z
|
||||
.object({
|
||||
type: z.literal("sops"),
|
||||
path: z.string().min(1),
|
||||
timeoutMs: z.number().int().positive().max(120000).optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
export const SecretsConfigSchema = z
|
||||
.object({
|
||||
sources: z
|
||||
.object({
|
||||
env: SecretsEnvSourceSchema.optional(),
|
||||
file: SecretsFileSourceSchema.optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional();
|
||||
|
||||
export const ModelApiSchema = z.union([
|
||||
z.literal("openai-completions"),
|
||||
z.literal("openai-responses"),
|
||||
@@ -58,7 +101,7 @@ export const ModelDefinitionSchema = z
|
||||
export const ModelProviderSchema = z
|
||||
.object({
|
||||
baseUrl: z.string().min(1),
|
||||
apiKey: z.string().optional().register(sensitive),
|
||||
apiKey: SecretInputSchema.optional().register(sensitive),
|
||||
auth: z
|
||||
.union([z.literal("api-key"), z.literal("aws-sdk"), z.literal("oauth"), z.literal("token")])
|
||||
.optional(),
|
||||
|
||||
@@ -25,6 +25,7 @@ import {
|
||||
MarkdownConfigSchema,
|
||||
MSTeamsReplyStyleSchema,
|
||||
ProviderCommandsSchema,
|
||||
SecretRefSchema,
|
||||
ReplyToModeSchema,
|
||||
RetryConfigSchema,
|
||||
TtsConfigSchema,
|
||||
@@ -519,7 +520,11 @@ export const GoogleChatAccountSchema = z
|
||||
groupAllowFrom: z.array(z.union([z.string(), z.number()])).optional(),
|
||||
groups: z.record(z.string(), GoogleChatGroupSchema.optional()).optional(),
|
||||
defaultTo: z.string().optional(),
|
||||
serviceAccount: z.union([z.string(), z.record(z.string(), z.unknown())]).optional(),
|
||||
serviceAccount: z
|
||||
.union([z.string(), z.record(z.string(), z.unknown()), SecretRefSchema])
|
||||
.optional()
|
||||
.register(sensitive),
|
||||
serviceAccountRef: SecretRefSchema.optional().register(sensitive),
|
||||
serviceAccountFile: z.string().optional(),
|
||||
audienceType: z.enum(["app-url", "project-number"]).optional(),
|
||||
audience: z.string().optional(),
|
||||
|
||||
@@ -2,7 +2,12 @@ import { z } from "zod";
|
||||
import { ToolsSchema } from "./zod-schema.agent-runtime.js";
|
||||
import { AgentsSchema, AudioSchema, BindingsSchema, BroadcastSchema } from "./zod-schema.agents.js";
|
||||
import { ApprovalsSchema } from "./zod-schema.approvals.js";
|
||||
import { HexColorSchema, ModelsConfigSchema } from "./zod-schema.core.js";
|
||||
import {
|
||||
HexColorSchema,
|
||||
ModelsConfigSchema,
|
||||
SecretInputSchema,
|
||||
SecretsConfigSchema,
|
||||
} from "./zod-schema.core.js";
|
||||
import { HookMappingSchema, HooksGmailSchema, InternalHooksSchema } from "./zod-schema.hooks.js";
|
||||
import { InstallRecordShape } from "./zod-schema.installs.js";
|
||||
import { ChannelsSchema } from "./zod-schema.providers.js";
|
||||
@@ -72,18 +77,9 @@ const MemoryQmdLimitsSchema = z
|
||||
})
|
||||
.strict();
|
||||
|
||||
const MemoryQmdMcporterSchema = z
|
||||
.object({
|
||||
enabled: z.boolean().optional(),
|
||||
serverName: z.string().optional(),
|
||||
startDaemon: z.boolean().optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
const MemoryQmdSchema = z
|
||||
.object({
|
||||
command: z.string().optional(),
|
||||
mcporter: MemoryQmdMcporterSchema.optional(),
|
||||
searchMode: z.union([z.literal("query"), z.literal("search"), z.literal("vsearch")]).optional(),
|
||||
includeDefaultMemory: z.boolean().optional(),
|
||||
paths: z.array(MemoryQmdPathSchema).optional(),
|
||||
@@ -272,6 +268,7 @@ export const OpenClawSchema = z
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
secrets: SecretsConfigSchema,
|
||||
auth: z
|
||||
.object({
|
||||
profiles: z
|
||||
@@ -621,7 +618,7 @@ export const OpenClawSchema = z
|
||||
z
|
||||
.object({
|
||||
enabled: z.boolean().optional(),
|
||||
apiKey: z.string().optional().register(sensitive),
|
||||
apiKey: SecretInputSchema.optional().register(sensitive),
|
||||
env: z.record(z.string(), z.string()).optional(),
|
||||
config: z.record(z.string(), z.unknown()).optional(),
|
||||
})
|
||||
|
||||
@@ -13,18 +13,6 @@ describe("cron schedule", () => {
|
||||
expect(next).toBe(Date.parse("2025-12-17T17:00:00.000Z"));
|
||||
});
|
||||
|
||||
it("throws a clear error when cron expr is missing at runtime", () => {
|
||||
const nowMs = Date.parse("2025-12-13T00:00:00.000Z");
|
||||
expect(() =>
|
||||
computeNextRunAtMs(
|
||||
{
|
||||
kind: "cron",
|
||||
} as unknown as { kind: "cron"; expr: string; tz?: string },
|
||||
nowMs,
|
||||
),
|
||||
).toThrow("invalid cron schedule: expr is required");
|
||||
});
|
||||
|
||||
it("computes next run for every schedule", () => {
|
||||
const anchor = Date.parse("2025-12-13T00:00:00.000Z");
|
||||
const now = anchor + 10_000;
|
||||
|
||||
@@ -41,11 +41,7 @@ export function computeNextRunAtMs(schedule: CronSchedule, nowMs: number): numbe
|
||||
return anchor + steps * everyMs;
|
||||
}
|
||||
|
||||
const exprSource = (schedule as { expr?: unknown }).expr;
|
||||
if (typeof exprSource !== "string") {
|
||||
throw new Error("invalid cron schedule: expr is required");
|
||||
}
|
||||
const expr = exprSource.trim();
|
||||
const expr = schedule.expr.trim();
|
||||
if (!expr) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -11,22 +11,6 @@ const noopLogger = {
|
||||
error: vi.fn(),
|
||||
};
|
||||
|
||||
async function withTimeout<T>(promise: Promise<T>, timeoutMs: number, label: string): Promise<T> {
|
||||
let timeout: NodeJS.Timeout | undefined;
|
||||
try {
|
||||
return await Promise.race([
|
||||
promise,
|
||||
new Promise<T>((_resolve, reject) => {
|
||||
timeout = setTimeout(() => reject(new Error(`${label} timed out`)), timeoutMs);
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
if (timeout) {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function makeStorePath() {
|
||||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-cron-"));
|
||||
return {
|
||||
@@ -151,86 +135,4 @@ describe("CronService read ops while job is running", () => {
|
||||
await store.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps list and status responsive during startup catch-up runs", async () => {
|
||||
const store = await makeStorePath();
|
||||
const enqueueSystemEvent = vi.fn();
|
||||
const requestHeartbeatNow = vi.fn();
|
||||
const nowMs = Date.parse("2025-12-13T00:00:00.000Z");
|
||||
|
||||
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
|
||||
await fs.writeFile(
|
||||
store.storePath,
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
jobs: [
|
||||
{
|
||||
id: "startup-catchup",
|
||||
name: "startup catch-up",
|
||||
enabled: true,
|
||||
createdAtMs: nowMs - 86_400_000,
|
||||
updatedAtMs: nowMs - 86_400_000,
|
||||
schedule: { kind: "at", at: new Date(nowMs - 60_000).toISOString() },
|
||||
sessionTarget: "isolated",
|
||||
wakeMode: "next-heartbeat",
|
||||
payload: { kind: "agentTurn", message: "startup replay" },
|
||||
delivery: { mode: "none" },
|
||||
state: { nextRunAtMs: nowMs - 60_000 },
|
||||
},
|
||||
],
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
let resolveRun:
|
||||
| ((value: { status: "ok" | "error" | "skipped"; summary?: string; error?: string }) => void)
|
||||
| undefined;
|
||||
let resolveRunStarted: (() => void) | undefined;
|
||||
const runStarted = new Promise<void>((resolve) => {
|
||||
resolveRunStarted = resolve;
|
||||
});
|
||||
|
||||
const runIsolatedAgentJob = vi.fn(async () => {
|
||||
resolveRunStarted?.();
|
||||
return await new Promise<{
|
||||
status: "ok" | "error" | "skipped";
|
||||
summary?: string;
|
||||
error?: string;
|
||||
}>((resolve) => {
|
||||
resolveRun = resolve;
|
||||
});
|
||||
});
|
||||
|
||||
const cron = new CronService({
|
||||
storePath: store.storePath,
|
||||
cronEnabled: true,
|
||||
log: noopLogger,
|
||||
nowMs: () => nowMs,
|
||||
enqueueSystemEvent,
|
||||
requestHeartbeatNow,
|
||||
runIsolatedAgentJob,
|
||||
});
|
||||
|
||||
try {
|
||||
const startPromise = cron.start();
|
||||
await runStarted;
|
||||
|
||||
await expect(
|
||||
withTimeout(cron.list({ includeDisabled: true }), 300, "cron.list during startup"),
|
||||
).resolves.toBeTypeOf("object");
|
||||
await expect(withTimeout(cron.status(), 300, "cron.status during startup")).resolves.toEqual(
|
||||
expect.objectContaining({ enabled: true, storePath: store.storePath }),
|
||||
);
|
||||
|
||||
resolveRun?.({ status: "ok", summary: "done" });
|
||||
await startPromise;
|
||||
|
||||
const jobs = await cron.list({ includeDisabled: true });
|
||||
expect(jobs[0]?.state.lastStatus).toBe("ok");
|
||||
expect(jobs[0]?.state.runningAtMs).toBeUndefined();
|
||||
} finally {
|
||||
cron.stop();
|
||||
await store.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -186,19 +186,4 @@ describe("cron schedule error isolation", () => {
|
||||
expect(badJob.state.lastError).toMatch(/^schedule error:/);
|
||||
expect(badJob.state.lastError).toBeTruthy();
|
||||
});
|
||||
|
||||
it("records a clear schedule error when cron expr is missing", () => {
|
||||
const badJob = createJob({
|
||||
id: "missing-expr",
|
||||
name: "Missing Expr",
|
||||
schedule: { kind: "cron" } as unknown as CronJob["schedule"],
|
||||
});
|
||||
const state = createMockState([badJob]);
|
||||
|
||||
recomputeNextRuns(state);
|
||||
|
||||
expect(badJob.state.lastError).toContain("invalid cron schedule: expr is required");
|
||||
expect(badJob.state.lastError).not.toContain("Cannot read properties of undefined");
|
||||
expect(badJob.state.scheduleErrorCount).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -28,15 +28,14 @@ async function ensureLoadedForRead(state: CronServiceState) {
|
||||
}
|
||||
|
||||
export async function start(state: CronServiceState) {
|
||||
if (!state.deps.cronEnabled) {
|
||||
state.deps.log.info({ enabled: false }, "cron: disabled");
|
||||
return;
|
||||
}
|
||||
|
||||
const startupInterruptedJobIds = new Set<string>();
|
||||
await locked(state, async () => {
|
||||
if (!state.deps.cronEnabled) {
|
||||
state.deps.log.info({ enabled: false }, "cron: disabled");
|
||||
return;
|
||||
}
|
||||
await ensureLoaded(state, { skipRecompute: true });
|
||||
const jobs = state.store?.jobs ?? [];
|
||||
const startupInterruptedJobIds = new Set<string>();
|
||||
for (const job of jobs) {
|
||||
if (typeof job.state.runningAtMs === "number") {
|
||||
state.deps.log.warn(
|
||||
@@ -47,13 +46,7 @@ export async function start(state: CronServiceState) {
|
||||
startupInterruptedJobIds.add(job.id);
|
||||
}
|
||||
}
|
||||
await persist(state);
|
||||
});
|
||||
|
||||
await runMissedJobs(state, { skipJobIds: startupInterruptedJobIds });
|
||||
|
||||
await locked(state, async () => {
|
||||
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
|
||||
await runMissedJobs(state, { skipJobIds: startupInterruptedJobIds });
|
||||
recomputeNextRuns(state);
|
||||
await persist(state);
|
||||
armTimer(state);
|
||||
|
||||
@@ -458,97 +458,22 @@ export async function runMissedJobs(
|
||||
state: CronServiceState,
|
||||
opts?: { skipJobIds?: ReadonlySet<string> },
|
||||
) {
|
||||
const startupCandidates = await locked(state, async () => {
|
||||
await ensureLoaded(state, { skipRecompute: true });
|
||||
if (!state.store) {
|
||||
return [] as Array<{ jobId: string; job: CronJob }>;
|
||||
}
|
||||
const now = state.deps.nowMs();
|
||||
const skipJobIds = opts?.skipJobIds;
|
||||
const missed = collectRunnableJobs(state, now, { skipJobIds, skipAtIfAlreadyRan: true });
|
||||
if (missed.length === 0) {
|
||||
return [] as Array<{ jobId: string; job: CronJob }>;
|
||||
}
|
||||
if (!state.store) {
|
||||
return;
|
||||
}
|
||||
const now = state.deps.nowMs();
|
||||
const skipJobIds = opts?.skipJobIds;
|
||||
const missed = collectRunnableJobs(state, now, { skipJobIds, skipAtIfAlreadyRan: true });
|
||||
|
||||
if (missed.length > 0) {
|
||||
state.deps.log.info(
|
||||
{ count: missed.length, jobIds: missed.map((j) => j.id) },
|
||||
"cron: running missed jobs after restart",
|
||||
);
|
||||
for (const job of missed) {
|
||||
job.state.runningAtMs = now;
|
||||
job.state.lastError = undefined;
|
||||
}
|
||||
await persist(state);
|
||||
return missed.map((job) => ({ jobId: job.id, job }));
|
||||
});
|
||||
|
||||
if (startupCandidates.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
const outcomes: Array<TimedCronRunOutcome> = [];
|
||||
for (const candidate of startupCandidates) {
|
||||
const startedAt = state.deps.nowMs();
|
||||
emit(state, { jobId: candidate.job.id, action: "started", runAtMs: startedAt });
|
||||
try {
|
||||
const result = await executeJobCore(state, candidate.job);
|
||||
outcomes.push({
|
||||
jobId: candidate.jobId,
|
||||
status: result.status,
|
||||
error: result.error,
|
||||
summary: result.summary,
|
||||
delivered: result.delivered,
|
||||
sessionId: result.sessionId,
|
||||
sessionKey: result.sessionKey,
|
||||
model: result.model,
|
||||
provider: result.provider,
|
||||
usage: result.usage,
|
||||
startedAt,
|
||||
endedAt: state.deps.nowMs(),
|
||||
});
|
||||
} catch (err) {
|
||||
outcomes.push({
|
||||
jobId: candidate.jobId,
|
||||
status: "error",
|
||||
error: String(err),
|
||||
startedAt,
|
||||
endedAt: state.deps.nowMs(),
|
||||
});
|
||||
await executeJob(state, job, now, { forced: false });
|
||||
}
|
||||
}
|
||||
|
||||
await locked(state, async () => {
|
||||
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
|
||||
if (!state.store) {
|
||||
return;
|
||||
}
|
||||
|
||||
for (const result of outcomes) {
|
||||
const job = state.store.jobs.find((entry) => entry.id === result.jobId);
|
||||
if (!job) {
|
||||
continue;
|
||||
}
|
||||
const shouldDelete = applyJobResult(state, job, {
|
||||
status: result.status,
|
||||
error: result.error,
|
||||
delivered: result.delivered,
|
||||
startedAt: result.startedAt,
|
||||
endedAt: result.endedAt,
|
||||
});
|
||||
|
||||
emitJobFinished(state, job, result, result.startedAt);
|
||||
|
||||
if (shouldDelete) {
|
||||
state.store.jobs = state.store.jobs.filter((entry) => entry.id !== job.id);
|
||||
emit(state, { jobId: job.id, action: "removed" });
|
||||
}
|
||||
}
|
||||
|
||||
// Preserve any new past-due nextRunAtMs values that became due while
|
||||
// startup catch-up was running. They should execute on a future tick
|
||||
// instead of being silently advanced.
|
||||
recomputeNextRunsForMaintenance(state);
|
||||
await persist(state);
|
||||
});
|
||||
}
|
||||
|
||||
export async function runDueJobs(state: CronServiceState) {
|
||||
|
||||
@@ -33,13 +33,4 @@ describe("cron stagger helpers", () => {
|
||||
expect(resolveCronStaggerMs({ kind: "cron", expr: "0 * * * *", staggerMs: 0 })).toBe(0);
|
||||
expect(resolveCronStaggerMs({ kind: "cron", expr: "15 * * * *" })).toBe(0);
|
||||
});
|
||||
|
||||
it("handles missing runtime expr values without throwing", () => {
|
||||
expect(() =>
|
||||
resolveCronStaggerMs({ kind: "cron" } as unknown as { kind: "cron"; expr: string }),
|
||||
).not.toThrow();
|
||||
expect(
|
||||
resolveCronStaggerMs({ kind: "cron" } as unknown as { kind: "cron"; expr: string }),
|
||||
).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -41,7 +41,5 @@ export function resolveCronStaggerMs(schedule: Extract<CronSchedule, { kind: "cr
|
||||
if (explicit !== undefined) {
|
||||
return explicit;
|
||||
}
|
||||
const expr = (schedule as { expr?: unknown }).expr;
|
||||
const cronExpr = typeof expr === "string" ? expr : "";
|
||||
return resolveDefaultCronStaggerMs(cronExpr) ?? 0;
|
||||
return resolveDefaultCronStaggerMs(schedule.expr) ?? 0;
|
||||
}
|
||||
|
||||
@@ -23,48 +23,6 @@ describe("diffConfigPaths", () => {
|
||||
const paths = diffConfigPaths(prev, next);
|
||||
expect(paths).toContain("messages.groupChat.mentionPatterns");
|
||||
});
|
||||
|
||||
it("does not report unchanged arrays of objects as changed", () => {
|
||||
const prev = {
|
||||
memory: {
|
||||
qmd: {
|
||||
paths: [{ path: "~/docs", pattern: "**/*.md", name: "docs" }],
|
||||
scope: {
|
||||
rules: [{ when: { channel: "slack" }, include: ["docs"] }],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const next = {
|
||||
memory: {
|
||||
qmd: {
|
||||
paths: [{ path: "~/docs", pattern: "**/*.md", name: "docs" }],
|
||||
scope: {
|
||||
rules: [{ when: { channel: "slack" }, include: ["docs"] }],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(diffConfigPaths(prev, next)).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports changed arrays of objects", () => {
|
||||
const prev = {
|
||||
memory: {
|
||||
qmd: {
|
||||
paths: [{ path: "~/docs", pattern: "**/*.md", name: "docs" }],
|
||||
},
|
||||
},
|
||||
};
|
||||
const next = {
|
||||
memory: {
|
||||
qmd: {
|
||||
paths: [{ path: "~/docs", pattern: "**/*.txt", name: "docs" }],
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(diffConfigPaths(prev, next)).toContain("memory.qmd.paths");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildGatewayReloadPlan", () => {
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import chokidar from "chokidar";
|
||||
import { type ChannelId, listChannelPlugins } from "../channels/plugins/index.js";
|
||||
import type { OpenClawConfig, ConfigFileSnapshot, GatewayReloadMode } from "../config/config.js";
|
||||
@@ -151,9 +150,7 @@ export function diffConfigPaths(prev: unknown, next: unknown, prefix = ""): stri
|
||||
return paths;
|
||||
}
|
||||
if (Array.isArray(prev) && Array.isArray(next)) {
|
||||
// Arrays can contain object entries (for example memory.qmd.paths/scope.rules);
|
||||
// compare structurally so identical values are not reported as changed.
|
||||
if (isDeepStrictEqual(prev, next)) {
|
||||
if (prev.length === next.length && prev.every((val, idx) => val === next[idx])) {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -253,7 +250,7 @@ export function startGatewayConfigReloader(opts: {
|
||||
initialConfig: OpenClawConfig;
|
||||
readSnapshot: () => Promise<ConfigFileSnapshot>;
|
||||
onHotReload: (plan: GatewayReloadPlan, nextConfig: OpenClawConfig) => Promise<void>;
|
||||
onRestart: (plan: GatewayReloadPlan, nextConfig: OpenClawConfig) => void;
|
||||
onRestart: (plan: GatewayReloadPlan, nextConfig: OpenClawConfig) => void | Promise<void>;
|
||||
log: {
|
||||
info: (msg: string) => void;
|
||||
warn: (msg: string) => void;
|
||||
@@ -304,42 +301,53 @@ export function startGatewayConfigReloader(opts: {
|
||||
}
|
||||
const nextConfig = snapshot.config;
|
||||
const changedPaths = diffConfigPaths(currentConfig, nextConfig);
|
||||
currentConfig = nextConfig;
|
||||
settings = resolveGatewayReloadSettings(nextConfig);
|
||||
const nextSettings = resolveGatewayReloadSettings(nextConfig);
|
||||
if (changedPaths.length === 0) {
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
return;
|
||||
}
|
||||
|
||||
opts.log.info(`config change detected; evaluating reload (${changedPaths.join(", ")})`);
|
||||
const plan = buildGatewayReloadPlan(changedPaths);
|
||||
if (settings.mode === "off") {
|
||||
if (nextSettings.mode === "off") {
|
||||
opts.log.info("config reload disabled (gateway.reload.mode=off)");
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
return;
|
||||
}
|
||||
if (settings.mode === "restart") {
|
||||
if (nextSettings.mode === "restart") {
|
||||
if (!restartQueued) {
|
||||
restartQueued = true;
|
||||
opts.onRestart(plan, nextConfig);
|
||||
await opts.onRestart(plan, nextConfig);
|
||||
}
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
return;
|
||||
}
|
||||
if (plan.restartGateway) {
|
||||
if (settings.mode === "hot") {
|
||||
if (nextSettings.mode === "hot") {
|
||||
opts.log.warn(
|
||||
`config reload requires gateway restart; hot mode ignoring (${plan.restartReasons.join(
|
||||
", ",
|
||||
)})`,
|
||||
);
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
return;
|
||||
}
|
||||
if (!restartQueued) {
|
||||
restartQueued = true;
|
||||
opts.onRestart(plan, nextConfig);
|
||||
await opts.onRestart(plan, nextConfig);
|
||||
}
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
return;
|
||||
}
|
||||
|
||||
await opts.onHotReload(plan, nextConfig);
|
||||
currentConfig = nextConfig;
|
||||
settings = nextSettings;
|
||||
} catch (err) {
|
||||
opts.log.error(`config reload failed: ${String(err)}`);
|
||||
} finally {
|
||||
|
||||
@@ -97,6 +97,7 @@ const METHOD_SCOPE_GROUPS: Record<OperatorScope, readonly string[]> = {
|
||||
"agents.delete",
|
||||
"skills.install",
|
||||
"skills.update",
|
||||
"secrets.reload",
|
||||
"cron.add",
|
||||
"cron.update",
|
||||
"cron.remove",
|
||||
|
||||
@@ -48,6 +48,7 @@ const BASE_METHODS = [
|
||||
"update.run",
|
||||
"voicewake.get",
|
||||
"voicewake.set",
|
||||
"secrets.reload",
|
||||
"sessions.list",
|
||||
"sessions.preview",
|
||||
"sessions.patch",
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
import { DEFAULT_PROVIDER } from "../../agents/defaults.js";
|
||||
import { buildAllowedModelSet } from "../../agents/model-selection.js";
|
||||
import { loadConfig } from "../../config/config.js";
|
||||
import {
|
||||
ErrorCodes,
|
||||
errorShape,
|
||||
@@ -23,14 +20,7 @@ export const modelsHandlers: GatewayRequestHandlers = {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const catalog = await context.loadGatewayModelCatalog();
|
||||
const cfg = loadConfig();
|
||||
const { allowedCatalog } = buildAllowedModelSet({
|
||||
cfg,
|
||||
catalog,
|
||||
defaultProvider: DEFAULT_PROVIDER,
|
||||
});
|
||||
const models = allowedCatalog.length > 0 ? allowedCatalog : catalog;
|
||||
const models = await context.loadGatewayModelCatalog();
|
||||
respond(true, { models }, undefined);
|
||||
} catch (err) {
|
||||
respond(false, undefined, errorShape(ErrorCodes.UNAVAILABLE, String(err)));
|
||||
|
||||
43
src/gateway/server-methods/secrets.test.ts
Normal file
43
src/gateway/server-methods/secrets.test.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { createSecretsHandlers } from "./secrets.js";
|
||||
|
||||
describe("secrets handlers", () => {
|
||||
it("responds with warning count on successful reload", async () => {
|
||||
const handlers = createSecretsHandlers({
|
||||
reloadSecrets: vi.fn().mockResolvedValue({ warningCount: 2 }),
|
||||
});
|
||||
const respond = vi.fn();
|
||||
await handlers["secrets.reload"]({
|
||||
req: { type: "req", id: "1", method: "secrets.reload" },
|
||||
params: {},
|
||||
client: null,
|
||||
isWebchatConnect: () => false,
|
||||
respond,
|
||||
context: {} as never,
|
||||
});
|
||||
expect(respond).toHaveBeenCalledWith(true, { ok: true, warningCount: 2 });
|
||||
});
|
||||
|
||||
it("returns unavailable when reload fails", async () => {
|
||||
const handlers = createSecretsHandlers({
|
||||
reloadSecrets: vi.fn().mockRejectedValue(new Error("reload failed")),
|
||||
});
|
||||
const respond = vi.fn();
|
||||
await handlers["secrets.reload"]({
|
||||
req: { type: "req", id: "1", method: "secrets.reload" },
|
||||
params: {},
|
||||
client: null,
|
||||
isWebchatConnect: () => false,
|
||||
respond,
|
||||
context: {} as never,
|
||||
});
|
||||
expect(respond).toHaveBeenCalledWith(
|
||||
false,
|
||||
undefined,
|
||||
expect.objectContaining({
|
||||
code: "UNAVAILABLE",
|
||||
message: "Error: reload failed",
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
17
src/gateway/server-methods/secrets.ts
Normal file
17
src/gateway/server-methods/secrets.ts
Normal file
@@ -0,0 +1,17 @@
|
||||
import { ErrorCodes, errorShape } from "../protocol/index.js";
|
||||
import type { GatewayRequestHandlers } from "./types.js";
|
||||
|
||||
export function createSecretsHandlers(params: {
|
||||
reloadSecrets: () => Promise<{ warningCount: number }>;
|
||||
}): GatewayRequestHandlers {
|
||||
return {
|
||||
"secrets.reload": async ({ respond }) => {
|
||||
try {
|
||||
const result = await params.reloadSecrets();
|
||||
respond(true, { ok: true, warningCount: result.warningCount });
|
||||
} catch (err) {
|
||||
respond(false, undefined, errorShape(ErrorCodes.UNAVAILABLE, String(err)));
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import { createDefaultDeps } from "../cli/deps.js";
|
||||
import { isRestartEnabled } from "../config/commands.js";
|
||||
import {
|
||||
CONFIG_PATH,
|
||||
type OpenClawConfig,
|
||||
isNixMode,
|
||||
loadConfig,
|
||||
migrateLegacyConfig,
|
||||
@@ -45,6 +46,12 @@ import { createEmptyPluginRegistry } from "../plugins/registry.js";
|
||||
import type { PluginServicesHandle } from "../plugins/services.js";
|
||||
import { getTotalQueueSize } from "../process/command-queue.js";
|
||||
import type { RuntimeEnv } from "../runtime.js";
|
||||
import {
|
||||
activateSecretsRuntimeSnapshot,
|
||||
clearSecretsRuntimeSnapshot,
|
||||
getActiveSecretsRuntimeSnapshot,
|
||||
prepareSecretsRuntimeSnapshot,
|
||||
} from "../secrets/runtime.js";
|
||||
import { runOnboardingWizard } from "../wizard/onboarding.js";
|
||||
import { createAuthRateLimiter, type AuthRateLimiter } from "./auth-rate-limit.js";
|
||||
import { startChannelHealthMonitor } from "./channel-health-monitor.js";
|
||||
@@ -68,6 +75,7 @@ import { GATEWAY_EVENTS, listGatewayMethods } from "./server-methods-list.js";
|
||||
import { coreGatewayHandlers } from "./server-methods.js";
|
||||
import { createExecApprovalHandlers } from "./server-methods/exec-approval.js";
|
||||
import { safeParseJson } from "./server-methods/nodes.helpers.js";
|
||||
import { createSecretsHandlers } from "./server-methods/secrets.js";
|
||||
import { hasConnectedMobileNode } from "./server-mobile-nodes.js";
|
||||
import { loadGatewayModelCatalog } from "./server-model-catalog.js";
|
||||
import { createNodeSubscriptionManager } from "./server-node-subscriptions.js";
|
||||
@@ -107,6 +115,7 @@ const logReload = log.child("reload");
|
||||
const logHooks = log.child("hooks");
|
||||
const logPlugins = log.child("plugins");
|
||||
const logWsControl = log.child("ws");
|
||||
const logSecrets = log.child("secrets");
|
||||
const gatewayRuntime = runtimeForLogger(log);
|
||||
const canvasRuntime = runtimeForLogger(logCanvas);
|
||||
|
||||
@@ -233,7 +242,63 @@ export async function startGatewayServer(
|
||||
}
|
||||
}
|
||||
|
||||
let cfgAtStart = loadConfig();
|
||||
let secretsDegraded = false;
|
||||
const activateRuntimeSecrets = async (
|
||||
config: OpenClawConfig,
|
||||
params: { reason: "startup" | "reload" | "restart-check"; activate: boolean },
|
||||
) => {
|
||||
try {
|
||||
const prepared = await prepareSecretsRuntimeSnapshot({ config });
|
||||
if (params.activate) {
|
||||
activateSecretsRuntimeSnapshot(prepared);
|
||||
}
|
||||
for (const warning of prepared.warnings) {
|
||||
logSecrets.warn(`[${warning.code}] ${warning.message}`);
|
||||
}
|
||||
if (secretsDegraded) {
|
||||
logSecrets.info(
|
||||
"[SECRETS_RELOADER_RECOVERED] Secret resolution recovered; runtime remained on last-known-good during the outage.",
|
||||
);
|
||||
}
|
||||
secretsDegraded = false;
|
||||
return prepared;
|
||||
} catch (err) {
|
||||
const details = String(err);
|
||||
if (!secretsDegraded) {
|
||||
logSecrets.error(`[SECRETS_RELOADER_DEGRADED] ${details}`);
|
||||
} else {
|
||||
logSecrets.warn(`[SECRETS_RELOADER_DEGRADED] ${details}`);
|
||||
}
|
||||
secretsDegraded = true;
|
||||
if (params.reason === "startup") {
|
||||
throw new Error(`Startup failed: required secrets are unavailable. ${details}`, {
|
||||
cause: err,
|
||||
});
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
};
|
||||
|
||||
// Fail fast before startup if required refs are unresolved.
|
||||
let cfgAtStart: OpenClawConfig;
|
||||
{
|
||||
const freshSnapshot = await readConfigFileSnapshot();
|
||||
if (!freshSnapshot.valid) {
|
||||
const issues =
|
||||
freshSnapshot.issues.length > 0
|
||||
? freshSnapshot.issues
|
||||
.map((issue) => `${issue.path || "<root>"}: ${issue.message}`)
|
||||
.join("\n")
|
||||
: "Unknown validation issue.";
|
||||
throw new Error(`Invalid config at ${freshSnapshot.path}.\n${issues}`);
|
||||
}
|
||||
await activateRuntimeSecrets(freshSnapshot.config, {
|
||||
reason: "startup",
|
||||
activate: false,
|
||||
});
|
||||
}
|
||||
|
||||
cfgAtStart = loadConfig();
|
||||
const authBootstrap = await ensureGatewayStartupAuth({
|
||||
cfg: cfgAtStart,
|
||||
env: process.env,
|
||||
@@ -253,6 +318,12 @@ export async function startGatewayServer(
|
||||
);
|
||||
}
|
||||
}
|
||||
cfgAtStart = (
|
||||
await activateRuntimeSecrets(cfgAtStart, {
|
||||
reason: "startup",
|
||||
activate: true,
|
||||
})
|
||||
).config;
|
||||
const diagnosticsEnabled = isDiagnosticsEnabled(cfgAtStart);
|
||||
if (diagnosticsEnabled) {
|
||||
startDiagnosticHeartbeat();
|
||||
@@ -560,6 +631,19 @@ export async function startGatewayServer(
|
||||
const execApprovalHandlers = createExecApprovalHandlers(execApprovalManager, {
|
||||
forwarder: execApprovalForwarder,
|
||||
});
|
||||
const secretsHandlers = createSecretsHandlers({
|
||||
reloadSecrets: async () => {
|
||||
const active = getActiveSecretsRuntimeSnapshot();
|
||||
if (!active) {
|
||||
throw new Error("Secrets runtime snapshot is not active.");
|
||||
}
|
||||
const prepared = await activateRuntimeSecrets(active.sourceConfig, {
|
||||
reason: "reload",
|
||||
activate: true,
|
||||
});
|
||||
return { warningCount: prepared.warnings.length };
|
||||
},
|
||||
});
|
||||
|
||||
const canvasHostServerPort = (canvasHostServer as CanvasHostServer | null)?.port;
|
||||
|
||||
@@ -580,6 +664,7 @@ export async function startGatewayServer(
|
||||
extraHandlers: {
|
||||
...pluginRegistry.gatewayHandlers,
|
||||
...execApprovalHandlers,
|
||||
...secretsHandlers,
|
||||
},
|
||||
broadcast,
|
||||
context: {
|
||||
@@ -710,8 +795,27 @@ export async function startGatewayServer(
|
||||
return startGatewayConfigReloader({
|
||||
initialConfig: cfgAtStart,
|
||||
readSnapshot: readConfigFileSnapshot,
|
||||
onHotReload: applyHotReload,
|
||||
onRestart: requestGatewayRestart,
|
||||
onHotReload: async (plan, nextConfig) => {
|
||||
const previousSnapshot = getActiveSecretsRuntimeSnapshot();
|
||||
const prepared = await activateRuntimeSecrets(nextConfig, {
|
||||
reason: "reload",
|
||||
activate: true,
|
||||
});
|
||||
try {
|
||||
await applyHotReload(plan, prepared.config);
|
||||
} catch (err) {
|
||||
if (previousSnapshot) {
|
||||
activateSecretsRuntimeSnapshot(previousSnapshot);
|
||||
} else {
|
||||
clearSecretsRuntimeSnapshot();
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
},
|
||||
onRestart: async (plan, nextConfig) => {
|
||||
await activateRuntimeSecrets(nextConfig, { reason: "restart-check", activate: false });
|
||||
requestGatewayRestart(plan, nextConfig);
|
||||
},
|
||||
log: {
|
||||
info: (msg) => logReload.info(msg),
|
||||
warn: (msg) => logReload.warn(msg),
|
||||
@@ -764,6 +868,7 @@ export async function startGatewayServer(
|
||||
skillsChangeUnsub();
|
||||
authRateLimiter?.dispose();
|
||||
channelHealthMonitor?.stop();
|
||||
clearSecretsRuntimeSnapshot();
|
||||
await close(opts);
|
||||
},
|
||||
};
|
||||
|
||||
@@ -5,7 +5,6 @@ import { afterAll, beforeAll, describe, expect, test } from "vitest";
|
||||
import { WebSocket } from "ws";
|
||||
import { getChannelPlugin } from "../channels/plugins/index.js";
|
||||
import type { ChannelOutboundAdapter } from "../channels/plugins/types.js";
|
||||
import { clearConfigCache } from "../config/config.js";
|
||||
import { resolveCanvasHostUrl } from "../infra/canvas-host-url.js";
|
||||
import { GatewayLockError } from "../infra/gateway-lock.js";
|
||||
import { getActivePluginRegistry, setActivePluginRegistry } from "../plugins/runtime.js";
|
||||
@@ -252,99 +251,6 @@ describe("gateway server models + voicewake", () => {
|
||||
expect(piSdkMock.discoverCalls).toBe(1);
|
||||
});
|
||||
|
||||
test("models.list filters to allowlisted configured models by default", async () => {
|
||||
const configPath = process.env.OPENCLAW_CONFIG_PATH;
|
||||
if (!configPath) {
|
||||
throw new Error("Missing OPENCLAW_CONFIG_PATH");
|
||||
}
|
||||
let previousConfig: string | undefined;
|
||||
try {
|
||||
previousConfig = await fs.readFile(configPath, "utf-8");
|
||||
} catch (err) {
|
||||
const code = (err as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code !== "ENOENT") {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
try {
|
||||
await fs.mkdir(path.dirname(configPath), { recursive: true });
|
||||
await fs.writeFile(
|
||||
configPath,
|
||||
JSON.stringify(
|
||||
{
|
||||
agents: {
|
||||
defaults: {
|
||||
model: { primary: "openai/gpt-test-z" },
|
||||
models: {
|
||||
"openai/gpt-test-z": {},
|
||||
"anthropic/claude-test-a": {},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
"utf-8",
|
||||
);
|
||||
clearConfigCache();
|
||||
|
||||
piSdkMock.enabled = true;
|
||||
piSdkMock.models = [
|
||||
{ id: "gpt-test-z", provider: "openai", contextWindow: 0 },
|
||||
{
|
||||
id: "gpt-test-a",
|
||||
name: "A-Model",
|
||||
provider: "openai",
|
||||
contextWindow: 8000,
|
||||
},
|
||||
{
|
||||
id: "claude-test-b",
|
||||
name: "B-Model",
|
||||
provider: "anthropic",
|
||||
contextWindow: 1000,
|
||||
},
|
||||
{
|
||||
id: "claude-test-a",
|
||||
name: "A-Model",
|
||||
provider: "anthropic",
|
||||
contextWindow: 200_000,
|
||||
},
|
||||
];
|
||||
|
||||
const res = await rpcReq<{
|
||||
models: Array<{
|
||||
id: string;
|
||||
name: string;
|
||||
provider: string;
|
||||
contextWindow?: number;
|
||||
}>;
|
||||
}>(ws, "models.list");
|
||||
|
||||
expect(res.ok).toBe(true);
|
||||
expect(res.payload?.models).toEqual([
|
||||
{
|
||||
id: "claude-test-a",
|
||||
name: "A-Model",
|
||||
provider: "anthropic",
|
||||
contextWindow: 200_000,
|
||||
},
|
||||
{
|
||||
id: "gpt-test-z",
|
||||
name: "gpt-test-z",
|
||||
provider: "openai",
|
||||
},
|
||||
]);
|
||||
} finally {
|
||||
if (previousConfig === undefined) {
|
||||
await fs.rm(configPath, { force: true });
|
||||
} else {
|
||||
await fs.writeFile(configPath, previousConfig, "utf-8");
|
||||
}
|
||||
clearConfigCache();
|
||||
}
|
||||
});
|
||||
|
||||
test("models.list rejects unknown params", async () => {
|
||||
piSdkMock.enabled = true;
|
||||
piSdkMock.models = [{ id: "gpt-test-a", name: "A", provider: "openai" }];
|
||||
|
||||
@@ -122,26 +122,6 @@ describe("device pairing tokens", () => {
|
||||
expect(paired?.tokens?.operator?.scopes).toEqual(["operator.read"]);
|
||||
});
|
||||
|
||||
test("preserves existing token scopes when approving a repair without requested scopes", async () => {
|
||||
const baseDir = await mkdtemp(join(tmpdir(), "openclaw-device-pairing-"));
|
||||
await setupPairedOperatorDevice(baseDir, ["operator.admin"]);
|
||||
|
||||
const repair = await requestDevicePairing(
|
||||
{
|
||||
deviceId: "device-1",
|
||||
publicKey: "public-key-1",
|
||||
role: "operator",
|
||||
},
|
||||
baseDir,
|
||||
);
|
||||
await approveDevicePairing(repair.request.requestId, baseDir);
|
||||
|
||||
const paired = await getPairedDevice("device-1", baseDir);
|
||||
expect(paired?.scopes).toEqual(["operator.admin"]);
|
||||
expect(paired?.approvedScopes).toEqual(["operator.admin"]);
|
||||
expect(paired?.tokens?.operator?.scopes).toEqual(["operator.admin"]);
|
||||
});
|
||||
|
||||
test("rejects scope escalation when rotating a token and leaves state unchanged", async () => {
|
||||
const baseDir = await mkdtemp(join(tmpdir(), "openclaw-device-pairing-"));
|
||||
await setupPairedOperatorDevice(baseDir, ["operator.read"]);
|
||||
@@ -188,7 +168,7 @@ describe("device pairing tokens", () => {
|
||||
expect(mismatch.reason).toBe("token-mismatch");
|
||||
});
|
||||
|
||||
test("accepts operator.read/operator.write requests with an operator.admin token scope", async () => {
|
||||
test("accepts operator.read requests with an operator.admin token scope", async () => {
|
||||
const baseDir = await mkdtemp(join(tmpdir(), "openclaw-device-pairing-"));
|
||||
await setupPairedOperatorDevice(baseDir, ["operator.admin"]);
|
||||
const paired = await getPairedDevice("device-1", baseDir);
|
||||
@@ -203,14 +183,14 @@ describe("device pairing tokens", () => {
|
||||
});
|
||||
expect(readOk.ok).toBe(true);
|
||||
|
||||
const writeOk = await verifyDeviceToken({
|
||||
const writeMismatch = await verifyDeviceToken({
|
||||
deviceId: "device-1",
|
||||
token,
|
||||
role: "operator",
|
||||
scopes: ["operator.write"],
|
||||
baseDir,
|
||||
});
|
||||
expect(writeOk.ok).toBe(true);
|
||||
expect(writeMismatch).toEqual({ ok: false, reason: "scope-mismatch" });
|
||||
});
|
||||
|
||||
test("treats multibyte same-length token input as mismatch without throwing", async () => {
|
||||
|
||||
@@ -332,17 +332,8 @@ export async function approveDevicePairing(
|
||||
const tokens = existing?.tokens ? { ...existing.tokens } : {};
|
||||
const roleForToken = normalizeRole(pending.role);
|
||||
if (roleForToken) {
|
||||
const nextScopes = normalizeDeviceAuthScopes(pending.scopes);
|
||||
const existingToken = tokens[roleForToken];
|
||||
const requestedScopes = normalizeDeviceAuthScopes(pending.scopes);
|
||||
const nextScopes =
|
||||
requestedScopes.length > 0
|
||||
? requestedScopes
|
||||
: normalizeDeviceAuthScopes(
|
||||
existingToken?.scopes ??
|
||||
approvedScopes ??
|
||||
existing?.approvedScopes ??
|
||||
existing?.scopes,
|
||||
);
|
||||
const now = Date.now();
|
||||
tokens[roleForToken] = {
|
||||
token: newToken(),
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
validateSafeBinArgv,
|
||||
} from "./exec-safe-bin-policy.js";
|
||||
import { isTrustedSafeBinPath } from "./exec-safe-bin-trust.js";
|
||||
|
||||
export function normalizeSafeBins(entries?: string[]): Set<string> {
|
||||
if (!Array.isArray(entries)) {
|
||||
return new Set();
|
||||
|
||||
@@ -196,55 +196,6 @@ describe("gateway lock", () => {
|
||||
staleSpy.mockRestore();
|
||||
});
|
||||
|
||||
it("keeps lock when fs.stat fails until payload is stale", async () => {
|
||||
vi.useRealTimers();
|
||||
const env = await makeEnv();
|
||||
const { lockPath, configPath } = resolveLockPath(env);
|
||||
const payload = createLockPayload({ configPath, startTime: 111 });
|
||||
await fs.writeFile(lockPath, JSON.stringify(payload), "utf8");
|
||||
|
||||
const procSpy = mockProcStatRead({
|
||||
onProcRead: () => {
|
||||
throw new Error("EACCES");
|
||||
},
|
||||
});
|
||||
const statSpy = vi
|
||||
.spyOn(fs, "stat")
|
||||
.mockRejectedValue(Object.assign(new Error("EPERM"), { code: "EPERM" }));
|
||||
|
||||
const pending = acquireForTest(env, {
|
||||
timeoutMs: 20,
|
||||
staleMs: 10_000,
|
||||
platform: "linux",
|
||||
});
|
||||
await expect(pending).rejects.toBeInstanceOf(GatewayLockError);
|
||||
|
||||
procSpy.mockRestore();
|
||||
|
||||
const stalePayload = createLockPayload({
|
||||
configPath,
|
||||
startTime: 111,
|
||||
createdAt: new Date(0).toISOString(),
|
||||
});
|
||||
await fs.writeFile(lockPath, JSON.stringify(stalePayload), "utf8");
|
||||
|
||||
const staleProcSpy = mockProcStatRead({
|
||||
onProcRead: () => {
|
||||
throw new Error("EACCES");
|
||||
},
|
||||
});
|
||||
|
||||
const lock = await acquireForTest(env, {
|
||||
staleMs: 1,
|
||||
platform: "linux",
|
||||
});
|
||||
expect(lock).not.toBeNull();
|
||||
|
||||
await lock?.release();
|
||||
staleProcSpy.mockRestore();
|
||||
statSpy.mockRestore();
|
||||
});
|
||||
|
||||
it("returns null when multi-gateway override is enabled", async () => {
|
||||
const env = await makeEnv();
|
||||
const lock = await acquireGatewayLock({
|
||||
|
||||
@@ -231,11 +231,7 @@ export async function acquireGatewayLock(
|
||||
const st = await fs.stat(lockPath);
|
||||
stale = Date.now() - st.mtimeMs > staleMs;
|
||||
} catch {
|
||||
// On Windows or locked filesystems we may be unable to stat the
|
||||
// lock file even though the existing gateway is still healthy.
|
||||
// Treat the lock as non-stale so we keep waiting instead of
|
||||
// forcefully removing another gateway's lock.
|
||||
stale = false;
|
||||
stale = true;
|
||||
}
|
||||
}
|
||||
if (stale) {
|
||||
|
||||
@@ -8,7 +8,6 @@ import type {
|
||||
MemoryCitationsMode,
|
||||
MemoryQmdConfig,
|
||||
MemoryQmdIndexPath,
|
||||
MemoryQmdMcporterConfig,
|
||||
MemoryQmdSearchMode,
|
||||
} from "../config/types.memory.js";
|
||||
import { resolveUserPath } from "../utils.js";
|
||||
@@ -51,15 +50,8 @@ export type ResolvedQmdSessionConfig = {
|
||||
retentionDays?: number;
|
||||
};
|
||||
|
||||
export type ResolvedQmdMcporterConfig = {
|
||||
enabled: boolean;
|
||||
serverName: string;
|
||||
startDaemon: boolean;
|
||||
};
|
||||
|
||||
export type ResolvedQmdConfig = {
|
||||
command: string;
|
||||
mcporter: ResolvedQmdMcporterConfig;
|
||||
searchMode: MemoryQmdSearchMode;
|
||||
collections: ResolvedQmdCollection[];
|
||||
sessions: ResolvedQmdSessionConfig;
|
||||
@@ -87,12 +79,6 @@ const DEFAULT_QMD_LIMITS: ResolvedQmdLimitsConfig = {
|
||||
maxInjectedChars: 4_000,
|
||||
timeoutMs: DEFAULT_QMD_TIMEOUT_MS,
|
||||
};
|
||||
const DEFAULT_QMD_MCPORTER: ResolvedQmdMcporterConfig = {
|
||||
enabled: false,
|
||||
serverName: "qmd",
|
||||
startDaemon: true,
|
||||
};
|
||||
|
||||
const DEFAULT_QMD_SCOPE: SessionSendPolicyConfig = {
|
||||
default: "deny",
|
||||
rules: [
|
||||
@@ -251,27 +237,6 @@ function resolveCustomPaths(
|
||||
return collections;
|
||||
}
|
||||
|
||||
function resolveMcporterConfig(raw?: MemoryQmdMcporterConfig): ResolvedQmdMcporterConfig {
|
||||
const parsed: ResolvedQmdMcporterConfig = { ...DEFAULT_QMD_MCPORTER };
|
||||
if (!raw) {
|
||||
return parsed;
|
||||
}
|
||||
if (raw.enabled !== undefined) {
|
||||
parsed.enabled = raw.enabled;
|
||||
}
|
||||
if (typeof raw.serverName === "string" && raw.serverName.trim()) {
|
||||
parsed.serverName = raw.serverName.trim();
|
||||
}
|
||||
if (raw.startDaemon !== undefined) {
|
||||
parsed.startDaemon = raw.startDaemon;
|
||||
}
|
||||
// When enabled, default startDaemon to true.
|
||||
if (parsed.enabled && raw.startDaemon === undefined) {
|
||||
parsed.startDaemon = true;
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function resolveDefaultCollections(
|
||||
include: boolean,
|
||||
workspaceDir: string,
|
||||
@@ -318,7 +283,6 @@ export function resolveMemoryBackendConfig(params: {
|
||||
const command = parsedCommand?.[0] || rawCommand.split(/\s+/)[0] || "qmd";
|
||||
const resolved: ResolvedQmdConfig = {
|
||||
command,
|
||||
mcporter: resolveMcporterConfig(qmdCfg?.mcporter),
|
||||
searchMode: resolveSearchMode(qmdCfg?.searchMode),
|
||||
collections,
|
||||
includeDefaultMemory,
|
||||
|
||||
@@ -185,9 +185,7 @@ export async function createEmbeddingProvider(
|
||||
continue;
|
||||
}
|
||||
// Non-auth errors (e.g., network) are still fatal
|
||||
const wrapped = new Error(message) as Error & { cause?: unknown };
|
||||
wrapped.cause = err;
|
||||
throw wrapped;
|
||||
throw new Error(message, { cause: err });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -230,9 +228,7 @@ export async function createEmbeddingProvider(
|
||||
};
|
||||
}
|
||||
// Non-auth errors are still fatal
|
||||
const wrapped = new Error(combinedReason) as Error & { cause?: unknown };
|
||||
wrapped.cause = fallbackErr;
|
||||
throw wrapped;
|
||||
throw new Error(combinedReason, { cause: fallbackErr });
|
||||
}
|
||||
}
|
||||
// No fallback configured - check if we should degrade to FTS-only
|
||||
@@ -243,9 +239,7 @@ export async function createEmbeddingProvider(
|
||||
providerUnavailableReason: reason,
|
||||
};
|
||||
}
|
||||
const wrapped = new Error(reason) as Error & { cause?: unknown };
|
||||
wrapped.cause = primaryErr;
|
||||
throw wrapped;
|
||||
throw new Error(reason, { cause: primaryErr });
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -148,8 +148,6 @@ describe("QmdMemoryManager", () => {
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers();
|
||||
delete process.env.OPENCLAW_STATE_DIR;
|
||||
delete (globalThis as Record<string, unknown>).__openclawMcporterDaemonStart;
|
||||
delete (globalThis as Record<string, unknown>).__openclawMcporterColdStartWarned;
|
||||
});
|
||||
|
||||
it("debounces back-to-back sync calls", async () => {
|
||||
@@ -912,171 +910,6 @@ describe("QmdMemoryManager", () => {
|
||||
await manager.close();
|
||||
});
|
||||
|
||||
it("runs qmd searches via mcporter and warns when startDaemon=false", async () => {
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
mcporter: { enabled: true, serverName: "qmd", startDaemon: false },
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
spawnMock.mockImplementation((cmd: string, args: string[]) => {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
if (cmd === "mcporter" && args[0] === "call") {
|
||||
emitAndClose(child, "stdout", JSON.stringify({ results: [] }));
|
||||
return child;
|
||||
}
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
});
|
||||
|
||||
const { manager } = await createManager();
|
||||
|
||||
logWarnMock.mockClear();
|
||||
await expect(
|
||||
manager.search("hello", { sessionKey: "agent:main:slack:dm:u123" }),
|
||||
).resolves.toEqual([]);
|
||||
|
||||
const mcporterCalls = spawnMock.mock.calls.filter((call: unknown[]) => call[0] === "mcporter");
|
||||
expect(mcporterCalls.length).toBeGreaterThan(0);
|
||||
expect(mcporterCalls.some((call: unknown[]) => (call[1] as string[])[0] === "daemon")).toBe(
|
||||
false,
|
||||
);
|
||||
expect(logWarnMock).toHaveBeenCalledWith(expect.stringContaining("cold-start"));
|
||||
|
||||
await manager.close();
|
||||
});
|
||||
|
||||
it("passes manager-scoped XDG env to mcporter commands", async () => {
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
mcporter: { enabled: true, serverName: "qmd", startDaemon: false },
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
spawnMock.mockImplementation((cmd: string, args: string[]) => {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
if (cmd === "mcporter" && args[0] === "call") {
|
||||
emitAndClose(child, "stdout", JSON.stringify({ results: [] }));
|
||||
return child;
|
||||
}
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
});
|
||||
|
||||
const { manager } = await createManager();
|
||||
await manager.search("hello", { sessionKey: "agent:main:slack:dm:u123" });
|
||||
|
||||
const mcporterCall = spawnMock.mock.calls.find(
|
||||
(call: unknown[]) => call[0] === "mcporter" && (call[1] as string[])[0] === "call",
|
||||
);
|
||||
expect(mcporterCall).toBeDefined();
|
||||
const spawnOpts = mcporterCall?.[2] as { env?: NodeJS.ProcessEnv } | undefined;
|
||||
const normalizePath = (value?: string) => value?.replace(/\\/g, "/");
|
||||
expect(normalizePath(spawnOpts?.env?.XDG_CONFIG_HOME)).toContain("/agents/main/qmd/xdg-config");
|
||||
expect(normalizePath(spawnOpts?.env?.XDG_CACHE_HOME)).toContain("/agents/main/qmd/xdg-cache");
|
||||
|
||||
await manager.close();
|
||||
});
|
||||
|
||||
it("retries mcporter daemon start after a failure", async () => {
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
mcporter: { enabled: true, serverName: "qmd", startDaemon: true },
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
let daemonAttempts = 0;
|
||||
spawnMock.mockImplementation((cmd: string, args: string[]) => {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
if (cmd === "mcporter" && args[0] === "daemon") {
|
||||
daemonAttempts += 1;
|
||||
if (daemonAttempts === 1) {
|
||||
emitAndClose(child, "stderr", "failed", 1);
|
||||
} else {
|
||||
emitAndClose(child, "stdout", "");
|
||||
}
|
||||
return child;
|
||||
}
|
||||
if (cmd === "mcporter" && args[0] === "call") {
|
||||
emitAndClose(child, "stdout", JSON.stringify({ results: [] }));
|
||||
return child;
|
||||
}
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
});
|
||||
|
||||
const { manager } = await createManager();
|
||||
|
||||
await manager.search("one", { sessionKey: "agent:main:slack:dm:u123" });
|
||||
await manager.search("two", { sessionKey: "agent:main:slack:dm:u123" });
|
||||
|
||||
expect(daemonAttempts).toBe(2);
|
||||
|
||||
await manager.close();
|
||||
});
|
||||
|
||||
it("starts the mcporter daemon only once when enabled", async () => {
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
mcporter: { enabled: true, serverName: "qmd", startDaemon: true },
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
spawnMock.mockImplementation((cmd: string, args: string[]) => {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
if (cmd === "mcporter" && args[0] === "daemon") {
|
||||
emitAndClose(child, "stdout", "");
|
||||
return child;
|
||||
}
|
||||
if (cmd === "mcporter" && args[0] === "call") {
|
||||
emitAndClose(child, "stdout", JSON.stringify({ results: [] }));
|
||||
return child;
|
||||
}
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
});
|
||||
|
||||
const { manager } = await createManager();
|
||||
|
||||
await manager.search("one", { sessionKey: "agent:main:slack:dm:u123" });
|
||||
await manager.search("two", { sessionKey: "agent:main:slack:dm:u123" });
|
||||
|
||||
const daemonStarts = spawnMock.mock.calls.filter(
|
||||
(call: unknown[]) => call[0] === "mcporter" && (call[1] as string[])[0] === "daemon",
|
||||
);
|
||||
expect(daemonStarts).toHaveLength(1);
|
||||
|
||||
await manager.close();
|
||||
});
|
||||
|
||||
it("fails closed when no managed collections are configured", async () => {
|
||||
cfg = {
|
||||
...cfg,
|
||||
|
||||
@@ -25,11 +25,7 @@ import type {
|
||||
} from "./types.js";
|
||||
|
||||
type SqliteDatabase = import("node:sqlite").DatabaseSync;
|
||||
import type {
|
||||
ResolvedMemoryBackendConfig,
|
||||
ResolvedQmdConfig,
|
||||
ResolvedQmdMcporterConfig,
|
||||
} from "./backend-config.js";
|
||||
import type { ResolvedMemoryBackendConfig, ResolvedQmdConfig } from "./backend-config.js";
|
||||
import { parseQmdQueryJson, type QmdQueryResult } from "./qmd-query-parser.js";
|
||||
|
||||
const log = createSubsystemLogger("memory");
|
||||
@@ -429,37 +425,9 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
return [];
|
||||
}
|
||||
const qmdSearchCommand = this.qmd.searchMode;
|
||||
const mcporterEnabled = this.qmd.mcporter.enabled;
|
||||
let parsed: QmdQueryResult[];
|
||||
try {
|
||||
if (mcporterEnabled) {
|
||||
const tool: "search" | "vector_search" | "deep_search" =
|
||||
qmdSearchCommand === "search"
|
||||
? "search"
|
||||
: qmdSearchCommand === "vsearch"
|
||||
? "vector_search"
|
||||
: "deep_search";
|
||||
const minScore = opts?.minScore ?? 0;
|
||||
if (collectionNames.length > 1) {
|
||||
parsed = await this.runMcporterAcrossCollections({
|
||||
tool,
|
||||
query: trimmed,
|
||||
limit,
|
||||
minScore,
|
||||
collectionNames,
|
||||
});
|
||||
} else {
|
||||
parsed = await this.runQmdSearchViaMcporter({
|
||||
mcporter: this.qmd.mcporter,
|
||||
tool,
|
||||
query: trimmed,
|
||||
limit,
|
||||
minScore,
|
||||
collection: collectionNames[0],
|
||||
timeoutMs: this.qmd.limits.timeoutMs,
|
||||
});
|
||||
}
|
||||
} else if (collectionNames.length > 1) {
|
||||
if (collectionNames.length > 1) {
|
||||
parsed = await this.runQueryAcrossCollections(
|
||||
trimmed,
|
||||
limit,
|
||||
@@ -475,11 +443,7 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
parsed = parseQmdQueryJson(result.stdout, result.stderr);
|
||||
}
|
||||
} catch (err) {
|
||||
if (
|
||||
!mcporterEnabled &&
|
||||
qmdSearchCommand !== "query" &&
|
||||
this.isUnsupportedQmdOptionError(err)
|
||||
) {
|
||||
if (qmdSearchCommand !== "query" && this.isUnsupportedQmdOptionError(err)) {
|
||||
log.warn(
|
||||
`qmd ${qmdSearchCommand} does not support configured flags; retrying search with qmd query`,
|
||||
);
|
||||
@@ -499,8 +463,7 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
throw fallbackErr instanceof Error ? fallbackErr : new Error(String(fallbackErr));
|
||||
}
|
||||
} else {
|
||||
const label = mcporterEnabled ? "mcporter/qmd" : `qmd ${qmdSearchCommand}`;
|
||||
log.warn(`${label} failed: ${String(err)}`);
|
||||
log.warn(`qmd ${qmdSearchCommand} failed: ${String(err)}`);
|
||||
throw err instanceof Error ? err : new Error(String(err));
|
||||
}
|
||||
}
|
||||
@@ -896,169 +859,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
});
|
||||
}
|
||||
|
||||
private async ensureMcporterDaemonStarted(mcporter: ResolvedQmdMcporterConfig): Promise<void> {
|
||||
if (!mcporter.enabled) {
|
||||
return;
|
||||
}
|
||||
if (!mcporter.startDaemon) {
|
||||
type McporterWarnGlobal = typeof globalThis & {
|
||||
__openclawMcporterColdStartWarned?: boolean;
|
||||
};
|
||||
const g: McporterWarnGlobal = globalThis;
|
||||
if (!g.__openclawMcporterColdStartWarned) {
|
||||
g.__openclawMcporterColdStartWarned = true;
|
||||
log.warn(
|
||||
"mcporter qmd bridge enabled but startDaemon=false; each query may cold-start QMD MCP. Consider setting memory.qmd.mcporter.startDaemon=true to keep it warm.",
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
type McporterGlobal = typeof globalThis & {
|
||||
__openclawMcporterDaemonStart?: Promise<void>;
|
||||
};
|
||||
const g: McporterGlobal = globalThis;
|
||||
if (!g.__openclawMcporterDaemonStart) {
|
||||
g.__openclawMcporterDaemonStart = (async () => {
|
||||
try {
|
||||
await this.runMcporter(["daemon", "start"], { timeoutMs: 10_000 });
|
||||
} catch (err) {
|
||||
log.warn(`mcporter daemon start failed: ${String(err)}`);
|
||||
// Allow future searches to retry daemon start on transient failures.
|
||||
delete g.__openclawMcporterDaemonStart;
|
||||
}
|
||||
})();
|
||||
}
|
||||
await g.__openclawMcporterDaemonStart;
|
||||
}
|
||||
|
||||
private async runMcporter(
|
||||
args: string[],
|
||||
opts?: { timeoutMs?: number },
|
||||
): Promise<{ stdout: string; stderr: string }> {
|
||||
return await new Promise((resolve, reject) => {
|
||||
const child = spawn("mcporter", args, {
|
||||
// Keep mcporter and direct qmd commands on the same agent-scoped XDG state.
|
||||
env: this.env,
|
||||
cwd: this.workspaceDir,
|
||||
});
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let stdoutTruncated = false;
|
||||
let stderrTruncated = false;
|
||||
const timer = opts?.timeoutMs
|
||||
? setTimeout(() => {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error(`mcporter ${args.join(" ")} timed out after ${opts.timeoutMs}ms`));
|
||||
}, opts.timeoutMs)
|
||||
: null;
|
||||
child.stdout.on("data", (data) => {
|
||||
const next = appendOutputWithCap(stdout, data.toString("utf8"), this.maxQmdOutputChars);
|
||||
stdout = next.text;
|
||||
stdoutTruncated = stdoutTruncated || next.truncated;
|
||||
});
|
||||
child.stderr.on("data", (data) => {
|
||||
const next = appendOutputWithCap(stderr, data.toString("utf8"), this.maxQmdOutputChars);
|
||||
stderr = next.text;
|
||||
stderrTruncated = stderrTruncated || next.truncated;
|
||||
});
|
||||
child.on("error", (err) => {
|
||||
if (timer) {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
reject(err);
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
if (timer) {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
if (stdoutTruncated || stderrTruncated) {
|
||||
reject(
|
||||
new Error(
|
||||
`mcporter ${args.join(" ")} produced too much output (limit ${this.maxQmdOutputChars} chars)`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (code === 0) {
|
||||
resolve({ stdout, stderr });
|
||||
} else {
|
||||
reject(
|
||||
new Error(`mcporter ${args.join(" ")} failed (code ${code}): ${stderr || stdout}`),
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
private async runQmdSearchViaMcporter(params: {
|
||||
mcporter: ResolvedQmdMcporterConfig;
|
||||
tool: "search" | "vector_search" | "deep_search";
|
||||
query: string;
|
||||
limit: number;
|
||||
minScore: number;
|
||||
collection?: string;
|
||||
timeoutMs: number;
|
||||
}): Promise<QmdQueryResult[]> {
|
||||
await this.ensureMcporterDaemonStarted(params.mcporter);
|
||||
|
||||
const selector = `${params.mcporter.serverName}.${params.tool}`;
|
||||
const callArgs: Record<string, unknown> = {
|
||||
query: params.query,
|
||||
limit: params.limit,
|
||||
minScore: params.minScore,
|
||||
};
|
||||
if (params.collection) {
|
||||
callArgs.collection = params.collection;
|
||||
}
|
||||
|
||||
const result = await this.runMcporter(
|
||||
[
|
||||
"call",
|
||||
selector,
|
||||
"--args",
|
||||
JSON.stringify(callArgs),
|
||||
"--output",
|
||||
"json",
|
||||
"--timeout",
|
||||
String(Math.max(0, params.timeoutMs)),
|
||||
],
|
||||
{ timeoutMs: Math.max(params.timeoutMs + 2_000, 5_000) },
|
||||
);
|
||||
|
||||
const parsedUnknown: unknown = JSON.parse(result.stdout);
|
||||
const isRecord = (value: unknown): value is Record<string, unknown> =>
|
||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
|
||||
const structured =
|
||||
isRecord(parsedUnknown) && isRecord(parsedUnknown.structuredContent)
|
||||
? parsedUnknown.structuredContent
|
||||
: parsedUnknown;
|
||||
|
||||
const results: unknown[] =
|
||||
isRecord(structured) && Array.isArray(structured.results)
|
||||
? (structured.results as unknown[])
|
||||
: Array.isArray(structured)
|
||||
? structured
|
||||
: [];
|
||||
|
||||
const out: QmdQueryResult[] = [];
|
||||
for (const item of results) {
|
||||
if (!isRecord(item)) {
|
||||
continue;
|
||||
}
|
||||
const docidRaw = item.docid;
|
||||
const docid = typeof docidRaw === "string" ? docidRaw.replace(/^#/, "").trim() : "";
|
||||
if (!docid) {
|
||||
continue;
|
||||
}
|
||||
const scoreRaw = item.score;
|
||||
const score = typeof scoreRaw === "number" ? scoreRaw : Number(scoreRaw);
|
||||
const snippet = typeof item.snippet === "string" ? item.snippet : "";
|
||||
out.push({ docid, score: Number.isFinite(score) ? score : 0, snippet });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private async readPartialText(
|
||||
absPath: string,
|
||||
from?: number,
|
||||
@@ -1607,39 +1407,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
return [...bestByDocId.values()].toSorted((a, b) => (b.score ?? 0) - (a.score ?? 0));
|
||||
}
|
||||
|
||||
private async runMcporterAcrossCollections(params: {
|
||||
tool: "search" | "vector_search" | "deep_search";
|
||||
query: string;
|
||||
limit: number;
|
||||
minScore: number;
|
||||
collectionNames: string[];
|
||||
}): Promise<QmdQueryResult[]> {
|
||||
const bestByDocId = new Map<string, QmdQueryResult>();
|
||||
for (const collectionName of params.collectionNames) {
|
||||
const parsed = await this.runQmdSearchViaMcporter({
|
||||
mcporter: this.qmd.mcporter,
|
||||
tool: params.tool,
|
||||
query: params.query,
|
||||
limit: params.limit,
|
||||
minScore: params.minScore,
|
||||
collection: collectionName,
|
||||
timeoutMs: this.qmd.limits.timeoutMs,
|
||||
});
|
||||
for (const entry of parsed) {
|
||||
if (typeof entry.docid !== "string" || !entry.docid.trim()) {
|
||||
continue;
|
||||
}
|
||||
const prev = bestByDocId.get(entry.docid);
|
||||
const prevScore = typeof prev?.score === "number" ? prev.score : Number.NEGATIVE_INFINITY;
|
||||
const nextScore = typeof entry.score === "number" ? entry.score : Number.NEGATIVE_INFINITY;
|
||||
if (!prev || nextScore > prevScore) {
|
||||
bestByDocId.set(entry.docid, entry);
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...bestByDocId.values()].toSorted((a, b) => (b.score ?? 0) - (a.score ?? 0));
|
||||
}
|
||||
|
||||
private listManagedCollectionNames(): string[] {
|
||||
const seen = new Set<string>();
|
||||
const names: string[] = [];
|
||||
|
||||
@@ -38,63 +38,6 @@ describe("extractKeywords", () => {
|
||||
expect(keywords).toContain("bug");
|
||||
});
|
||||
|
||||
it("extracts keywords from Korean conversational query", () => {
|
||||
const keywords = extractKeywords("어제 논의한 배포 전략");
|
||||
expect(keywords).toContain("논의한");
|
||||
expect(keywords).toContain("배포");
|
||||
expect(keywords).toContain("전략");
|
||||
// Should not include stop words
|
||||
expect(keywords).not.toContain("어제");
|
||||
});
|
||||
|
||||
it("strips Korean particles to extract stems", () => {
|
||||
const keywords = extractKeywords("서버에서 발생한 에러를 확인");
|
||||
expect(keywords).toContain("서버");
|
||||
expect(keywords).toContain("에러");
|
||||
expect(keywords).toContain("확인");
|
||||
});
|
||||
|
||||
it("filters Korean stop words including inflected forms", () => {
|
||||
const keywords = extractKeywords("나는 그리고 그래서");
|
||||
expect(keywords).not.toContain("나");
|
||||
expect(keywords).not.toContain("나는");
|
||||
expect(keywords).not.toContain("그리고");
|
||||
expect(keywords).not.toContain("그래서");
|
||||
});
|
||||
|
||||
it("filters inflected Korean stop words not explicitly listed", () => {
|
||||
const keywords = extractKeywords("그녀는 우리는");
|
||||
expect(keywords).not.toContain("그녀는");
|
||||
expect(keywords).not.toContain("우리는");
|
||||
expect(keywords).not.toContain("그녀");
|
||||
expect(keywords).not.toContain("우리");
|
||||
});
|
||||
|
||||
it("does not produce bogus single-char stems from particle stripping", () => {
|
||||
const keywords = extractKeywords("논의");
|
||||
expect(keywords).toContain("논의");
|
||||
expect(keywords).not.toContain("논");
|
||||
});
|
||||
|
||||
it("strips longest Korean trailing particles first", () => {
|
||||
const keywords = extractKeywords("기능으로 설명");
|
||||
expect(keywords).toContain("기능");
|
||||
expect(keywords).not.toContain("기능으");
|
||||
});
|
||||
|
||||
it("keeps stripped ASCII stems for mixed Korean tokens", () => {
|
||||
const keywords = extractKeywords("API를 배포했다");
|
||||
expect(keywords).toContain("api");
|
||||
expect(keywords).toContain("배포했다");
|
||||
});
|
||||
|
||||
it("handles mixed Korean and English query", () => {
|
||||
const keywords = extractKeywords("API 배포에 대한 논의");
|
||||
expect(keywords).toContain("api");
|
||||
expect(keywords).toContain("배포");
|
||||
expect(keywords).toContain("논의");
|
||||
});
|
||||
|
||||
it("handles empty query", () => {
|
||||
expect(extractKeywords("")).toEqual([]);
|
||||
expect(extractKeywords(" ")).toEqual([]);
|
||||
|
||||
@@ -118,161 +118,6 @@ const STOP_WORDS_EN = new Set([
|
||||
"give",
|
||||
]);
|
||||
|
||||
const STOP_WORDS_KO = new Set([
|
||||
// Particles (조사)
|
||||
"은",
|
||||
"는",
|
||||
"이",
|
||||
"가",
|
||||
"을",
|
||||
"를",
|
||||
"의",
|
||||
"에",
|
||||
"에서",
|
||||
"로",
|
||||
"으로",
|
||||
"와",
|
||||
"과",
|
||||
"도",
|
||||
"만",
|
||||
"까지",
|
||||
"부터",
|
||||
"한테",
|
||||
"에게",
|
||||
"께",
|
||||
"처럼",
|
||||
"같이",
|
||||
"보다",
|
||||
"마다",
|
||||
"밖에",
|
||||
"대로",
|
||||
// Pronouns (대명사)
|
||||
"나",
|
||||
"나는",
|
||||
"내가",
|
||||
"나를",
|
||||
"너",
|
||||
"우리",
|
||||
"저",
|
||||
"저희",
|
||||
"그",
|
||||
"그녀",
|
||||
"그들",
|
||||
"이것",
|
||||
"저것",
|
||||
"그것",
|
||||
"여기",
|
||||
"저기",
|
||||
"거기",
|
||||
// Common verbs / auxiliaries (일반 동사/보조 동사)
|
||||
"있다",
|
||||
"없다",
|
||||
"하다",
|
||||
"되다",
|
||||
"이다",
|
||||
"아니다",
|
||||
"보다",
|
||||
"주다",
|
||||
"오다",
|
||||
"가다",
|
||||
// Nouns (의존 명사 / vague)
|
||||
"것",
|
||||
"거",
|
||||
"등",
|
||||
"수",
|
||||
"때",
|
||||
"곳",
|
||||
"중",
|
||||
"분",
|
||||
// Adverbs
|
||||
"잘",
|
||||
"더",
|
||||
"또",
|
||||
"매우",
|
||||
"정말",
|
||||
"아주",
|
||||
"많이",
|
||||
"너무",
|
||||
"좀",
|
||||
// Conjunctions
|
||||
"그리고",
|
||||
"하지만",
|
||||
"그래서",
|
||||
"그런데",
|
||||
"그러나",
|
||||
"또는",
|
||||
"그러면",
|
||||
// Question words
|
||||
"왜",
|
||||
"어떻게",
|
||||
"뭐",
|
||||
"언제",
|
||||
"어디",
|
||||
"누구",
|
||||
"무엇",
|
||||
"어떤",
|
||||
// Time (vague)
|
||||
"어제",
|
||||
"오늘",
|
||||
"내일",
|
||||
"최근",
|
||||
"지금",
|
||||
"아까",
|
||||
"나중",
|
||||
"전에",
|
||||
// Request words
|
||||
"제발",
|
||||
"부탁",
|
||||
]);
|
||||
|
||||
// Common Korean trailing particles to strip from words for tokenization
|
||||
// Sorted by descending length so longest-match-first is guaranteed.
|
||||
const KO_TRAILING_PARTICLES = [
|
||||
"에서",
|
||||
"으로",
|
||||
"에게",
|
||||
"한테",
|
||||
"처럼",
|
||||
"같이",
|
||||
"보다",
|
||||
"까지",
|
||||
"부터",
|
||||
"마다",
|
||||
"밖에",
|
||||
"대로",
|
||||
"은",
|
||||
"는",
|
||||
"이",
|
||||
"가",
|
||||
"을",
|
||||
"를",
|
||||
"의",
|
||||
"에",
|
||||
"로",
|
||||
"와",
|
||||
"과",
|
||||
"도",
|
||||
"만",
|
||||
].toSorted((a, b) => b.length - a.length);
|
||||
|
||||
function stripKoreanTrailingParticle(token: string): string | null {
|
||||
for (const particle of KO_TRAILING_PARTICLES) {
|
||||
if (token.length > particle.length && token.endsWith(particle)) {
|
||||
return token.slice(0, -particle.length);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function isUsefulKoreanStem(stem: string): boolean {
|
||||
// Prevent bogus one-syllable stems from words like "논의" -> "논".
|
||||
if (/[\uac00-\ud7af]/.test(stem)) {
|
||||
return stem.length >= 2;
|
||||
}
|
||||
// Keep stripped ASCII stems for mixed tokens like "API를" -> "api".
|
||||
return /^[a-z0-9_]+$/i.test(stem);
|
||||
}
|
||||
|
||||
const STOP_WORDS_ZH = new Set([
|
||||
// Pronouns
|
||||
"我",
|
||||
@@ -395,7 +240,7 @@ function isValidKeyword(token: string): boolean {
|
||||
}
|
||||
|
||||
/**
|
||||
* Simple tokenizer that handles English, Chinese, and Korean text.
|
||||
* Simple tokenizer that handles both English and Chinese text.
|
||||
* For Chinese, we do character-based splitting since we don't have a proper segmenter.
|
||||
* For English, we split on whitespace and punctuation.
|
||||
*/
|
||||
@@ -407,7 +252,7 @@ function tokenize(text: string): string[] {
|
||||
const segments = normalized.split(/[\s\p{P}]+/u).filter(Boolean);
|
||||
|
||||
for (const segment of segments) {
|
||||
// Check if segment contains CJK characters (Chinese)
|
||||
// Check if segment contains CJK characters
|
||||
if (/[\u4e00-\u9fff]/.test(segment)) {
|
||||
// For Chinese, extract character n-grams (unigrams and bigrams)
|
||||
const chars = Array.from(segment).filter((c) => /[\u4e00-\u9fff]/.test(c));
|
||||
@@ -417,18 +262,6 @@ function tokenize(text: string): string[] {
|
||||
for (let i = 0; i < chars.length - 1; i++) {
|
||||
tokens.push(chars[i] + chars[i + 1]);
|
||||
}
|
||||
} else if (/[\uac00-\ud7af\u3131-\u3163]/.test(segment)) {
|
||||
// For Korean (Hangul syllables and jamo), keep the word as-is unless it is
|
||||
// effectively a stop word once trailing particles are removed.
|
||||
const stem = stripKoreanTrailingParticle(segment);
|
||||
const stemIsStopWord = stem !== null && STOP_WORDS_KO.has(stem);
|
||||
if (!STOP_WORDS_KO.has(segment) && !stemIsStopWord) {
|
||||
tokens.push(segment);
|
||||
}
|
||||
// Also emit particle-stripped stems when they are useful keywords.
|
||||
if (stem && !STOP_WORDS_KO.has(stem) && isUsefulKoreanStem(stem)) {
|
||||
tokens.push(stem);
|
||||
}
|
||||
} else {
|
||||
// For non-CJK, keep as single token
|
||||
tokens.push(segment);
|
||||
@@ -453,7 +286,7 @@ export function extractKeywords(query: string): string[] {
|
||||
|
||||
for (const token of tokens) {
|
||||
// Skip stop words
|
||||
if (STOP_WORDS_EN.has(token) || STOP_WORDS_ZH.has(token) || STOP_WORDS_KO.has(token)) {
|
||||
if (STOP_WORDS_EN.has(token) || STOP_WORDS_ZH.has(token)) {
|
||||
continue;
|
||||
}
|
||||
// Skip invalid keywords
|
||||
|
||||
@@ -211,7 +211,6 @@ export {
|
||||
clearHistoryEntries,
|
||||
clearHistoryEntriesIfEnabled,
|
||||
DEFAULT_GROUP_HISTORY_LIMIT,
|
||||
evictOldHistoryKeys,
|
||||
recordPendingHistoryEntry,
|
||||
recordPendingHistoryEntryIfEnabled,
|
||||
} from "../auto-reply/reply/history.js";
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { OpenClawConfig } from "../config/config.js";
|
||||
import { enablePluginInConfig } from "./enable.js";
|
||||
|
||||
describe("enablePluginInConfig", () => {
|
||||
it("enables a plugin entry", () => {
|
||||
const cfg: OpenClawConfig = {};
|
||||
const result = enablePluginInConfig(cfg, "google-antigravity-auth");
|
||||
expect(result.enabled).toBe(true);
|
||||
expect(result.config.plugins?.entries?.["google-antigravity-auth"]?.enabled).toBe(true);
|
||||
});
|
||||
|
||||
it("adds plugin to allowlist when allowlist is configured", () => {
|
||||
const cfg: OpenClawConfig = {
|
||||
plugins: {
|
||||
allow: ["memory-core"],
|
||||
},
|
||||
};
|
||||
const result = enablePluginInConfig(cfg, "google-antigravity-auth");
|
||||
expect(result.enabled).toBe(true);
|
||||
expect(result.config.plugins?.allow).toEqual(["memory-core", "google-antigravity-auth"]);
|
||||
});
|
||||
|
||||
it("refuses enable when plugin is denylisted", () => {
|
||||
const cfg: OpenClawConfig = {
|
||||
plugins: {
|
||||
deny: ["google-antigravity-auth"],
|
||||
},
|
||||
};
|
||||
const result = enablePluginInConfig(cfg, "google-antigravity-auth");
|
||||
expect(result.enabled).toBe(false);
|
||||
expect(result.reason).toBe("blocked by denylist");
|
||||
});
|
||||
});
|
||||
94
src/secrets/json-pointer.ts
Normal file
94
src/secrets/json-pointer.ts
Normal file
@@ -0,0 +1,94 @@
|
||||
function failOrUndefined(params: { onMissing: "throw" | "undefined"; message: string }): undefined {
|
||||
if (params.onMissing === "throw") {
|
||||
throw new Error(params.message);
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export function decodeJsonPointerToken(token: string): string {
|
||||
return token.replace(/~1/g, "/").replace(/~0/g, "~");
|
||||
}
|
||||
|
||||
export function encodeJsonPointerToken(token: string): string {
|
||||
return token.replace(/~/g, "~0").replace(/\//g, "~1");
|
||||
}
|
||||
|
||||
export function readJsonPointer(
|
||||
root: unknown,
|
||||
pointer: string,
|
||||
options: { onMissing?: "throw" | "undefined" } = {},
|
||||
): unknown {
|
||||
const onMissing = options.onMissing ?? "throw";
|
||||
if (!pointer.startsWith("/")) {
|
||||
return failOrUndefined({
|
||||
onMissing,
|
||||
message:
|
||||
'File-backed secret ids must be absolute JSON pointers (for example: "/providers/openai/apiKey").',
|
||||
});
|
||||
}
|
||||
|
||||
const tokens = pointer
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.map((token) => decodeJsonPointerToken(token));
|
||||
|
||||
let current: unknown = root;
|
||||
for (const token of tokens) {
|
||||
if (Array.isArray(current)) {
|
||||
const index = Number.parseInt(token, 10);
|
||||
if (!Number.isFinite(index) || index < 0 || index >= current.length) {
|
||||
return failOrUndefined({
|
||||
onMissing,
|
||||
message: `JSON pointer segment "${token}" is out of bounds.`,
|
||||
});
|
||||
}
|
||||
current = current[index];
|
||||
continue;
|
||||
}
|
||||
if (typeof current !== "object" || current === null || Array.isArray(current)) {
|
||||
return failOrUndefined({
|
||||
onMissing,
|
||||
message: `JSON pointer segment "${token}" does not exist.`,
|
||||
});
|
||||
}
|
||||
const record = current as Record<string, unknown>;
|
||||
if (!Object.hasOwn(record, token)) {
|
||||
return failOrUndefined({
|
||||
onMissing,
|
||||
message: `JSON pointer segment "${token}" does not exist.`,
|
||||
});
|
||||
}
|
||||
current = record[token];
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
export function setJsonPointer(
|
||||
root: Record<string, unknown>,
|
||||
pointer: string,
|
||||
value: unknown,
|
||||
): void {
|
||||
if (!pointer.startsWith("/")) {
|
||||
throw new Error(`Invalid JSON pointer "${pointer}".`);
|
||||
}
|
||||
|
||||
const tokens = pointer
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.map((token) => decodeJsonPointerToken(token));
|
||||
|
||||
let current: Record<string, unknown> = root;
|
||||
for (let index = 0; index < tokens.length; index += 1) {
|
||||
const token = tokens[index];
|
||||
const isLast = index === tokens.length - 1;
|
||||
if (isLast) {
|
||||
current[token] = value;
|
||||
return;
|
||||
}
|
||||
const child = current[token];
|
||||
if (typeof child !== "object" || child === null || Array.isArray(child)) {
|
||||
current[token] = {};
|
||||
}
|
||||
current = current[token] as Record<string, unknown>;
|
||||
}
|
||||
}
|
||||
204
src/secrets/migrate.test.ts
Normal file
204
src/secrets/migrate.test.ts
Normal file
@@ -0,0 +1,204 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const runExecMock = vi.hoisted(() => vi.fn());
|
||||
|
||||
vi.mock("../process/exec.js", () => ({
|
||||
runExec: runExecMock,
|
||||
}));
|
||||
|
||||
const { rollbackSecretsMigration, runSecretsMigration } = await import("./migrate.js");
|
||||
|
||||
describe("secrets migrate", () => {
|
||||
let baseDir = "";
|
||||
let stateDir = "";
|
||||
let configPath = "";
|
||||
let env: NodeJS.ProcessEnv;
|
||||
let authStorePath = "";
|
||||
let envPath = "";
|
||||
|
||||
beforeEach(async () => {
|
||||
baseDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-secrets-migrate-"));
|
||||
stateDir = path.join(baseDir, ".openclaw");
|
||||
configPath = path.join(stateDir, "openclaw.json");
|
||||
authStorePath = path.join(stateDir, "agents", "main", "agent", "auth-profiles.json");
|
||||
envPath = path.join(stateDir, ".env");
|
||||
env = {
|
||||
...process.env,
|
||||
OPENCLAW_STATE_DIR: stateDir,
|
||||
OPENCLAW_CONFIG_PATH: configPath,
|
||||
};
|
||||
|
||||
await fs.mkdir(path.dirname(configPath), { recursive: true });
|
||||
await fs.mkdir(path.dirname(authStorePath), { recursive: true });
|
||||
|
||||
await fs.writeFile(
|
||||
configPath,
|
||||
`${JSON.stringify(
|
||||
{
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: "sk-openai-plaintext",
|
||||
models: [{ id: "gpt-5", name: "gpt-5" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
skills: {
|
||||
entries: {
|
||||
"review-pr": {
|
||||
enabled: true,
|
||||
apiKey: "sk-skill-plaintext",
|
||||
},
|
||||
},
|
||||
},
|
||||
channels: {
|
||||
googlechat: {
|
||||
serviceAccount: '{"type":"service_account","client_email":"bot@example.com"}',
|
||||
},
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
|
||||
await fs.writeFile(
|
||||
authStorePath,
|
||||
`${JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openai:default": {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
key: "sk-profile-plaintext",
|
||||
},
|
||||
},
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
|
||||
await fs.writeFile(
|
||||
envPath,
|
||||
"OPENAI_API_KEY=sk-openai-plaintext\nSKILL_KEY=sk-skill-plaintext\nUNRELATED=value\n",
|
||||
"utf8",
|
||||
);
|
||||
|
||||
runExecMock.mockReset();
|
||||
runExecMock.mockImplementation(async (_cmd: string, args: string[]) => {
|
||||
if (args[0] === "--encrypt") {
|
||||
const outputPath = args[args.indexOf("--output") + 1];
|
||||
const inputPath = args.at(-1);
|
||||
if (!outputPath || !inputPath) {
|
||||
throw new Error("missing sops encrypt paths");
|
||||
}
|
||||
await fs.copyFile(inputPath, outputPath);
|
||||
return { stdout: "", stderr: "" };
|
||||
}
|
||||
if (args[0] === "--decrypt") {
|
||||
const sourcePath = args.at(-1);
|
||||
if (!sourcePath) {
|
||||
throw new Error("missing sops decrypt source");
|
||||
}
|
||||
const raw = await fs.readFile(sourcePath, "utf8");
|
||||
return { stdout: raw, stderr: "" };
|
||||
}
|
||||
throw new Error(`unexpected sops invocation: ${args.join(" ")}`);
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(baseDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("reports a dry-run without mutating files", async () => {
|
||||
const beforeConfig = await fs.readFile(configPath, "utf8");
|
||||
const beforeAuthStore = await fs.readFile(authStorePath, "utf8");
|
||||
|
||||
const result = await runSecretsMigration({ env });
|
||||
|
||||
expect(result.mode).toBe("dry-run");
|
||||
expect(result.changed).toBe(true);
|
||||
expect(result.counters.secretsWritten).toBeGreaterThanOrEqual(3);
|
||||
|
||||
expect(await fs.readFile(configPath, "utf8")).toBe(beforeConfig);
|
||||
expect(await fs.readFile(authStorePath, "utf8")).toBe(beforeAuthStore);
|
||||
});
|
||||
|
||||
it("migrates plaintext to file-backed refs and can rollback", async () => {
|
||||
const applyResult = await runSecretsMigration({ env, write: true });
|
||||
|
||||
expect(applyResult.mode).toBe("write");
|
||||
expect(applyResult.changed).toBe(true);
|
||||
expect(applyResult.backupId).toBeTruthy();
|
||||
|
||||
const migratedConfig = JSON.parse(await fs.readFile(configPath, "utf8")) as {
|
||||
models: { providers: { openai: { apiKey: unknown } } };
|
||||
skills: { entries: { "review-pr": { apiKey: unknown } } };
|
||||
channels: { googlechat: { serviceAccount?: unknown; serviceAccountRef?: unknown } };
|
||||
secrets: { sources: { file: { type: string; path: string } } };
|
||||
};
|
||||
expect(migratedConfig.models.providers.openai.apiKey).toEqual({
|
||||
source: "file",
|
||||
id: "/providers/openai/apiKey",
|
||||
});
|
||||
expect(migratedConfig.skills.entries["review-pr"].apiKey).toEqual({
|
||||
source: "file",
|
||||
id: "/skills/entries/review-pr/apiKey",
|
||||
});
|
||||
expect(migratedConfig.channels.googlechat.serviceAccount).toBeUndefined();
|
||||
expect(migratedConfig.channels.googlechat.serviceAccountRef).toEqual({
|
||||
source: "file",
|
||||
id: "/channels/googlechat/serviceAccount",
|
||||
});
|
||||
expect(migratedConfig.secrets.sources.file.type).toBe("sops");
|
||||
|
||||
const migratedAuth = JSON.parse(await fs.readFile(authStorePath, "utf8")) as {
|
||||
profiles: { "openai:default": { key?: string; keyRef?: unknown } };
|
||||
};
|
||||
expect(migratedAuth.profiles["openai:default"].key).toBeUndefined();
|
||||
expect(migratedAuth.profiles["openai:default"].keyRef).toEqual({
|
||||
source: "file",
|
||||
id: "/auth-profiles/main/openai:default/key",
|
||||
});
|
||||
|
||||
const migratedEnv = await fs.readFile(envPath, "utf8");
|
||||
expect(migratedEnv).not.toContain("sk-openai-plaintext");
|
||||
expect(migratedEnv).toContain("SKILL_KEY=sk-skill-plaintext");
|
||||
expect(migratedEnv).toContain("UNRELATED=value");
|
||||
|
||||
const secretsPath = path.join(stateDir, "secrets.enc.json");
|
||||
const secretsPayload = JSON.parse(await fs.readFile(secretsPath, "utf8")) as {
|
||||
providers: { openai: { apiKey: string } };
|
||||
skills: { entries: { "review-pr": { apiKey: string } } };
|
||||
channels: { googlechat: { serviceAccount: string } };
|
||||
"auth-profiles": { main: { "openai:default": { key: string } } };
|
||||
};
|
||||
expect(secretsPayload.providers.openai.apiKey).toBe("sk-openai-plaintext");
|
||||
expect(secretsPayload.skills.entries["review-pr"].apiKey).toBe("sk-skill-plaintext");
|
||||
expect(secretsPayload.channels.googlechat.serviceAccount).toContain("service_account");
|
||||
expect(secretsPayload["auth-profiles"].main["openai:default"].key).toBe("sk-profile-plaintext");
|
||||
|
||||
const rollbackResult = await rollbackSecretsMigration({ env, backupId: applyResult.backupId! });
|
||||
expect(rollbackResult.restoredFiles).toBeGreaterThan(0);
|
||||
|
||||
const rolledBackConfig = await fs.readFile(configPath, "utf8");
|
||||
expect(rolledBackConfig).toContain("sk-openai-plaintext");
|
||||
expect(rolledBackConfig).toContain("sk-skill-plaintext");
|
||||
|
||||
const rolledBackAuth = await fs.readFile(authStorePath, "utf8");
|
||||
expect(rolledBackAuth).toContain("sk-profile-plaintext");
|
||||
|
||||
await expect(fs.stat(secretsPath)).rejects.toThrow();
|
||||
const rolledBackEnv = await fs.readFile(envPath, "utf8");
|
||||
expect(rolledBackEnv).toContain("OPENAI_API_KEY=sk-openai-plaintext");
|
||||
});
|
||||
});
|
||||
886
src/secrets/migrate.ts
Normal file
886
src/secrets/migrate.ts
Normal file
@@ -0,0 +1,886 @@
|
||||
import crypto from "node:crypto";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import { listAgentIds, resolveAgentDir } from "../agents/agent-scope.js";
|
||||
import { resolveAuthStorePath } from "../agents/auth-profiles/paths.js";
|
||||
import { createConfigIO, resolveStateDir, type OpenClawConfig } from "../config/config.js";
|
||||
import { isSecretRef } from "../config/types.secrets.js";
|
||||
import { resolveConfigDir, resolveUserPath } from "../utils.js";
|
||||
import {
|
||||
encodeJsonPointerToken,
|
||||
readJsonPointer as readJsonPointerRaw,
|
||||
setJsonPointer,
|
||||
} from "./json-pointer.js";
|
||||
import { listKnownSecretEnvVarNames } from "./provider-env-vars.js";
|
||||
import { decryptSopsJsonFile, encryptSopsJsonFile, DEFAULT_SOPS_TIMEOUT_MS } from "./sops.js";
|
||||
|
||||
const DEFAULT_SECRETS_FILE_PATH = "~/.openclaw/secrets.enc.json";
|
||||
const BACKUP_DIRNAME = "secrets-migrate";
|
||||
const BACKUP_MANIFEST_FILENAME = "manifest.json";
|
||||
const BACKUP_RETENTION = 20;
|
||||
|
||||
type MigrationCounters = {
|
||||
configRefs: number;
|
||||
authProfileRefs: number;
|
||||
plaintextRemoved: number;
|
||||
secretsWritten: number;
|
||||
envEntriesRemoved: number;
|
||||
authStoresChanged: number;
|
||||
};
|
||||
|
||||
type AuthStoreChange = {
|
||||
path: string;
|
||||
nextStore: Record<string, unknown>;
|
||||
};
|
||||
|
||||
type EnvChange = {
|
||||
path: string;
|
||||
nextRaw: string;
|
||||
};
|
||||
|
||||
type BackupManifestEntry = {
|
||||
path: string;
|
||||
existed: boolean;
|
||||
backupPath?: string;
|
||||
mode?: number;
|
||||
};
|
||||
|
||||
type BackupManifest = {
|
||||
version: 1;
|
||||
backupId: string;
|
||||
createdAt: string;
|
||||
entries: BackupManifestEntry[];
|
||||
};
|
||||
|
||||
type MigrationPlan = {
|
||||
changed: boolean;
|
||||
counters: MigrationCounters;
|
||||
stateDir: string;
|
||||
configChanged: boolean;
|
||||
nextConfig: OpenClawConfig;
|
||||
configWriteOptions: Awaited<
|
||||
ReturnType<ReturnType<typeof createConfigIO>["readConfigFileSnapshotForWrite"]>
|
||||
>["writeOptions"];
|
||||
authStoreChanges: AuthStoreChange[];
|
||||
payloadChanged: boolean;
|
||||
nextPayload: Record<string, unknown>;
|
||||
secretsFilePath: string;
|
||||
secretsFileTimeoutMs: number;
|
||||
envChange: EnvChange | null;
|
||||
backupTargets: string[];
|
||||
};
|
||||
|
||||
export type SecretsMigrationRunOptions = {
|
||||
write?: boolean;
|
||||
scrubEnv?: boolean;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
now?: Date;
|
||||
};
|
||||
|
||||
export type SecretsMigrationRunResult = {
|
||||
mode: "dry-run" | "write";
|
||||
changed: boolean;
|
||||
backupId?: string;
|
||||
backupDir?: string;
|
||||
secretsFilePath: string;
|
||||
counters: MigrationCounters;
|
||||
changedFiles: string[];
|
||||
};
|
||||
|
||||
export type SecretsMigrationRollbackOptions = {
|
||||
backupId: string;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
};
|
||||
|
||||
export type SecretsMigrationRollbackResult = {
|
||||
backupId: string;
|
||||
restoredFiles: number;
|
||||
deletedFiles: number;
|
||||
};
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function isNonEmptyString(value: unknown): value is string {
|
||||
return typeof value === "string" && value.trim().length > 0;
|
||||
}
|
||||
|
||||
function normalizeSopsTimeoutMs(value: unknown): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) {
|
||||
return Math.max(1, Math.floor(value));
|
||||
}
|
||||
return DEFAULT_SOPS_TIMEOUT_MS;
|
||||
}
|
||||
|
||||
function readJsonPointer(root: unknown, pointer: string): unknown {
|
||||
return readJsonPointerRaw(root, pointer, { onMissing: "undefined" });
|
||||
}
|
||||
|
||||
function formatBackupId(now: Date): string {
|
||||
const year = now.getUTCFullYear();
|
||||
const month = String(now.getUTCMonth() + 1).padStart(2, "0");
|
||||
const day = String(now.getUTCDate()).padStart(2, "0");
|
||||
const hour = String(now.getUTCHours()).padStart(2, "0");
|
||||
const minute = String(now.getUTCMinutes()).padStart(2, "0");
|
||||
const second = String(now.getUTCSeconds()).padStart(2, "0");
|
||||
return `${year}${month}${day}T${hour}${minute}${second}Z`;
|
||||
}
|
||||
|
||||
function parseEnvValue(raw: string): string {
|
||||
const trimmed = raw.trim();
|
||||
if (
|
||||
(trimmed.startsWith('"') && trimmed.endsWith('"')) ||
|
||||
(trimmed.startsWith("'") && trimmed.endsWith("'"))
|
||||
) {
|
||||
return trimmed.slice(1, -1);
|
||||
}
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
function scrubEnvRaw(
|
||||
raw: string,
|
||||
migratedValues: Set<string>,
|
||||
allowedEnvKeys: Set<string>,
|
||||
): {
|
||||
nextRaw: string;
|
||||
removed: number;
|
||||
} {
|
||||
if (migratedValues.size === 0 || allowedEnvKeys.size === 0) {
|
||||
return { nextRaw: raw, removed: 0 };
|
||||
}
|
||||
const lines = raw.split(/\r?\n/);
|
||||
const nextLines: string[] = [];
|
||||
let removed = 0;
|
||||
for (const line of lines) {
|
||||
const match = line.match(/^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$/);
|
||||
if (!match) {
|
||||
nextLines.push(line);
|
||||
continue;
|
||||
}
|
||||
const envKey = match[1] ?? "";
|
||||
if (!allowedEnvKeys.has(envKey)) {
|
||||
nextLines.push(line);
|
||||
continue;
|
||||
}
|
||||
const parsedValue = parseEnvValue(match[2] ?? "");
|
||||
if (migratedValues.has(parsedValue)) {
|
||||
removed += 1;
|
||||
continue;
|
||||
}
|
||||
nextLines.push(line);
|
||||
}
|
||||
const hadTrailingNewline = raw.endsWith("\n");
|
||||
const joined = nextLines.join("\n");
|
||||
return {
|
||||
nextRaw:
|
||||
hadTrailingNewline || joined.length === 0
|
||||
? `${joined}${joined.endsWith("\n") ? "" : "\n"}`
|
||||
: joined,
|
||||
removed,
|
||||
};
|
||||
}
|
||||
|
||||
function ensureDirForFile(filePath: string): void {
|
||||
fs.mkdirSync(path.dirname(filePath), { recursive: true, mode: 0o700 });
|
||||
}
|
||||
|
||||
function saveJsonFile(pathname: string, value: unknown): void {
|
||||
ensureDirForFile(pathname);
|
||||
fs.writeFileSync(pathname, `${JSON.stringify(value, null, 2)}\n`, "utf8");
|
||||
fs.chmodSync(pathname, 0o600);
|
||||
}
|
||||
|
||||
function resolveFileSource(
|
||||
config: OpenClawConfig,
|
||||
env: NodeJS.ProcessEnv,
|
||||
): {
|
||||
path: string;
|
||||
timeoutMs: number;
|
||||
hadConfiguredSource: boolean;
|
||||
} {
|
||||
const source = config.secrets?.sources?.file;
|
||||
if (source && source.type === "sops" && isNonEmptyString(source.path)) {
|
||||
return {
|
||||
path: resolveUserPath(source.path),
|
||||
timeoutMs: normalizeSopsTimeoutMs(source.timeoutMs),
|
||||
hadConfiguredSource: true,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
path: resolveUserPath(resolveDefaultSecretsConfigPath(env)),
|
||||
timeoutMs: DEFAULT_SOPS_TIMEOUT_MS,
|
||||
hadConfiguredSource: false,
|
||||
};
|
||||
}
|
||||
|
||||
function resolveDefaultSecretsConfigPath(env: NodeJS.ProcessEnv): string {
|
||||
if (env.OPENCLAW_STATE_DIR?.trim() || env.CLAWDBOT_STATE_DIR?.trim()) {
|
||||
return path.join(resolveStateDir(env, os.homedir), "secrets.enc.json");
|
||||
}
|
||||
return DEFAULT_SECRETS_FILE_PATH;
|
||||
}
|
||||
|
||||
async function decryptSopsJson(
|
||||
pathname: string,
|
||||
timeoutMs: number,
|
||||
): Promise<Record<string, unknown>> {
|
||||
if (!fs.existsSync(pathname)) {
|
||||
return {};
|
||||
}
|
||||
const parsed = await decryptSopsJsonFile({
|
||||
path: pathname,
|
||||
timeoutMs,
|
||||
missingBinaryMessage:
|
||||
"sops binary not found in PATH. Install sops >= 3.9.0 to run secrets migrate.",
|
||||
});
|
||||
if (!isRecord(parsed)) {
|
||||
throw new Error("sops decrypt failed: decrypted payload is not a JSON object");
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
async function encryptSopsJson(params: {
|
||||
pathname: string;
|
||||
timeoutMs: number;
|
||||
payload: Record<string, unknown>;
|
||||
}): Promise<void> {
|
||||
await encryptSopsJsonFile({
|
||||
path: params.pathname,
|
||||
payload: params.payload,
|
||||
timeoutMs: params.timeoutMs,
|
||||
missingBinaryMessage:
|
||||
"sops binary not found in PATH. Install sops >= 3.9.0 to run secrets migrate.",
|
||||
});
|
||||
}
|
||||
|
||||
function migrateModelProviderSecrets(params: {
|
||||
config: OpenClawConfig;
|
||||
payload: Record<string, unknown>;
|
||||
counters: MigrationCounters;
|
||||
migratedValues: Set<string>;
|
||||
}): void {
|
||||
const providers = params.config.models?.providers as
|
||||
| Record<string, { apiKey?: unknown }>
|
||||
| undefined;
|
||||
if (!providers) {
|
||||
return;
|
||||
}
|
||||
for (const [providerId, provider] of Object.entries(providers)) {
|
||||
if (isSecretRef(provider.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
if (!isNonEmptyString(provider.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
const value = provider.apiKey.trim();
|
||||
const id = `/providers/${encodeJsonPointerToken(providerId)}/apiKey`;
|
||||
const existing = readJsonPointer(params.payload, id);
|
||||
if (!isDeepStrictEqual(existing, value)) {
|
||||
setJsonPointer(params.payload, id, value);
|
||||
params.counters.secretsWritten += 1;
|
||||
}
|
||||
provider.apiKey = { source: "file", id };
|
||||
params.counters.configRefs += 1;
|
||||
params.migratedValues.add(value);
|
||||
}
|
||||
}
|
||||
|
||||
function migrateSkillEntrySecrets(params: {
|
||||
config: OpenClawConfig;
|
||||
payload: Record<string, unknown>;
|
||||
counters: MigrationCounters;
|
||||
migratedValues: Set<string>;
|
||||
}): void {
|
||||
const entries = params.config.skills?.entries as Record<string, { apiKey?: unknown }> | undefined;
|
||||
if (!entries) {
|
||||
return;
|
||||
}
|
||||
for (const [skillKey, entry] of Object.entries(entries)) {
|
||||
if (!isRecord(entry) || isSecretRef(entry.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
if (!isNonEmptyString(entry.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
const value = entry.apiKey.trim();
|
||||
const id = `/skills/entries/${encodeJsonPointerToken(skillKey)}/apiKey`;
|
||||
const existing = readJsonPointer(params.payload, id);
|
||||
if (!isDeepStrictEqual(existing, value)) {
|
||||
setJsonPointer(params.payload, id, value);
|
||||
params.counters.secretsWritten += 1;
|
||||
}
|
||||
entry.apiKey = { source: "file", id };
|
||||
params.counters.configRefs += 1;
|
||||
params.migratedValues.add(value);
|
||||
}
|
||||
}
|
||||
|
||||
function migrateGoogleChatServiceAccount(params: {
|
||||
account: Record<string, unknown>;
|
||||
pointerId: string;
|
||||
counters: MigrationCounters;
|
||||
payload: Record<string, unknown>;
|
||||
}): void {
|
||||
const explicitRef = isSecretRef(params.account.serviceAccountRef)
|
||||
? params.account.serviceAccountRef
|
||||
: null;
|
||||
const inlineRef = isSecretRef(params.account.serviceAccount)
|
||||
? params.account.serviceAccount
|
||||
: null;
|
||||
if (explicitRef || inlineRef) {
|
||||
if (
|
||||
params.account.serviceAccount !== undefined &&
|
||||
!isSecretRef(params.account.serviceAccount)
|
||||
) {
|
||||
delete params.account.serviceAccount;
|
||||
params.counters.plaintextRemoved += 1;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const value = params.account.serviceAccount;
|
||||
const hasStringValue = isNonEmptyString(value);
|
||||
const hasObjectValue = isRecord(value) && Object.keys(value).length > 0;
|
||||
if (!hasStringValue && !hasObjectValue) {
|
||||
return;
|
||||
}
|
||||
|
||||
const id = `${params.pointerId}/serviceAccount`;
|
||||
const normalizedValue = hasStringValue ? value.trim() : structuredClone(value);
|
||||
const existing = readJsonPointer(params.payload, id);
|
||||
if (!isDeepStrictEqual(existing, normalizedValue)) {
|
||||
setJsonPointer(params.payload, id, normalizedValue);
|
||||
params.counters.secretsWritten += 1;
|
||||
}
|
||||
|
||||
params.account.serviceAccountRef = { source: "file", id };
|
||||
delete params.account.serviceAccount;
|
||||
params.counters.configRefs += 1;
|
||||
}
|
||||
|
||||
function migrateGoogleChatSecrets(params: {
|
||||
config: OpenClawConfig;
|
||||
payload: Record<string, unknown>;
|
||||
counters: MigrationCounters;
|
||||
}): void {
|
||||
const googlechat = params.config.channels?.googlechat;
|
||||
if (!isRecord(googlechat)) {
|
||||
return;
|
||||
}
|
||||
|
||||
migrateGoogleChatServiceAccount({
|
||||
account: googlechat,
|
||||
pointerId: "/channels/googlechat",
|
||||
payload: params.payload,
|
||||
counters: params.counters,
|
||||
});
|
||||
|
||||
if (!isRecord(googlechat.accounts)) {
|
||||
return;
|
||||
}
|
||||
for (const [accountId, accountValue] of Object.entries(googlechat.accounts)) {
|
||||
if (!isRecord(accountValue)) {
|
||||
continue;
|
||||
}
|
||||
migrateGoogleChatServiceAccount({
|
||||
account: accountValue,
|
||||
pointerId: `/channels/googlechat/accounts/${encodeJsonPointerToken(accountId)}`,
|
||||
payload: params.payload,
|
||||
counters: params.counters,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function collectAuthStorePaths(config: OpenClawConfig, stateDir: string): string[] {
|
||||
const paths = new Set<string>();
|
||||
paths.add(resolveUserPath(resolveAuthStorePath()));
|
||||
|
||||
const agentsRoot = path.join(resolveUserPath(stateDir), "agents");
|
||||
if (fs.existsSync(agentsRoot)) {
|
||||
for (const entry of fs.readdirSync(agentsRoot, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory()) {
|
||||
continue;
|
||||
}
|
||||
paths.add(path.join(agentsRoot, entry.name, "agent", "auth-profiles.json"));
|
||||
}
|
||||
}
|
||||
|
||||
for (const agentId of listAgentIds(config)) {
|
||||
const agentDir = resolveAgentDir(config, agentId);
|
||||
paths.add(resolveUserPath(resolveAuthStorePath(agentDir)));
|
||||
}
|
||||
|
||||
return [...paths];
|
||||
}
|
||||
|
||||
function deriveAuthStoreScope(authStorePath: string, stateDir: string): string {
|
||||
const agentsRoot = path.join(resolveUserPath(stateDir), "agents");
|
||||
const relative = path.relative(agentsRoot, authStorePath);
|
||||
if (!relative.startsWith("..")) {
|
||||
const segments = relative.split(path.sep);
|
||||
if (segments.length >= 3 && segments[1] === "agent" && segments[2] === "auth-profiles.json") {
|
||||
const candidate = segments[0]?.trim();
|
||||
if (candidate) {
|
||||
return candidate;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const digest = crypto.createHash("sha1").update(authStorePath).digest("hex").slice(0, 8);
|
||||
return `path-${digest}`;
|
||||
}
|
||||
|
||||
function migrateAuthStoreSecrets(params: {
|
||||
store: Record<string, unknown>;
|
||||
scope: string;
|
||||
payload: Record<string, unknown>;
|
||||
counters: MigrationCounters;
|
||||
migratedValues: Set<string>;
|
||||
}): boolean {
|
||||
const profiles = params.store.profiles;
|
||||
if (!isRecord(profiles)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let changed = false;
|
||||
for (const [profileId, profileValue] of Object.entries(profiles)) {
|
||||
if (!isRecord(profileValue)) {
|
||||
continue;
|
||||
}
|
||||
if (profileValue.type === "api_key") {
|
||||
const keyRef = isSecretRef(profileValue.keyRef) ? profileValue.keyRef : null;
|
||||
const key = isNonEmptyString(profileValue.key) ? profileValue.key.trim() : "";
|
||||
if (keyRef) {
|
||||
if (key) {
|
||||
delete profileValue.key;
|
||||
params.counters.plaintextRemoved += 1;
|
||||
changed = true;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!key) {
|
||||
continue;
|
||||
}
|
||||
const id = `/auth-profiles/${encodeJsonPointerToken(params.scope)}/${encodeJsonPointerToken(profileId)}/key`;
|
||||
const existing = readJsonPointer(params.payload, id);
|
||||
if (!isDeepStrictEqual(existing, key)) {
|
||||
setJsonPointer(params.payload, id, key);
|
||||
params.counters.secretsWritten += 1;
|
||||
}
|
||||
profileValue.keyRef = { source: "file", id };
|
||||
delete profileValue.key;
|
||||
params.counters.authProfileRefs += 1;
|
||||
params.migratedValues.add(key);
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (profileValue.type === "token") {
|
||||
const tokenRef = isSecretRef(profileValue.tokenRef) ? profileValue.tokenRef : null;
|
||||
const token = isNonEmptyString(profileValue.token) ? profileValue.token.trim() : "";
|
||||
if (tokenRef) {
|
||||
if (token) {
|
||||
delete profileValue.token;
|
||||
params.counters.plaintextRemoved += 1;
|
||||
changed = true;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!token) {
|
||||
continue;
|
||||
}
|
||||
const id = `/auth-profiles/${encodeJsonPointerToken(params.scope)}/${encodeJsonPointerToken(profileId)}/token`;
|
||||
const existing = readJsonPointer(params.payload, id);
|
||||
if (!isDeepStrictEqual(existing, token)) {
|
||||
setJsonPointer(params.payload, id, token);
|
||||
params.counters.secretsWritten += 1;
|
||||
}
|
||||
profileValue.tokenRef = { source: "file", id };
|
||||
delete profileValue.token;
|
||||
params.counters.authProfileRefs += 1;
|
||||
params.migratedValues.add(token);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
return changed;
|
||||
}
|
||||
|
||||
function resolveBackupRoot(stateDir: string): string {
|
||||
return path.join(resolveUserPath(stateDir), "backups", BACKUP_DIRNAME);
|
||||
}
|
||||
|
||||
function createBackupManifest(params: {
|
||||
stateDir: string;
|
||||
targets: string[];
|
||||
backupId: string;
|
||||
now: Date;
|
||||
}): { backupDir: string; manifestPath: string; manifest: BackupManifest } {
|
||||
const backupDir = path.join(resolveBackupRoot(params.stateDir), params.backupId);
|
||||
fs.mkdirSync(backupDir, { recursive: true, mode: 0o700 });
|
||||
|
||||
const entries: BackupManifestEntry[] = [];
|
||||
let index = 0;
|
||||
for (const target of params.targets) {
|
||||
const normalized = resolveUserPath(target);
|
||||
const exists = fs.existsSync(normalized);
|
||||
if (!exists) {
|
||||
entries.push({ path: normalized, existed: false });
|
||||
continue;
|
||||
}
|
||||
|
||||
const backupName = `file-${String(index).padStart(4, "0")}.bak`;
|
||||
const backupPath = path.join(backupDir, backupName);
|
||||
fs.copyFileSync(normalized, backupPath);
|
||||
const stats = fs.statSync(normalized);
|
||||
entries.push({
|
||||
path: normalized,
|
||||
existed: true,
|
||||
backupPath,
|
||||
mode: stats.mode & 0o777,
|
||||
});
|
||||
index += 1;
|
||||
}
|
||||
|
||||
const manifest: BackupManifest = {
|
||||
version: 1,
|
||||
backupId: params.backupId,
|
||||
createdAt: params.now.toISOString(),
|
||||
entries,
|
||||
};
|
||||
const manifestPath = path.join(backupDir, BACKUP_MANIFEST_FILENAME);
|
||||
fs.writeFileSync(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`, "utf8");
|
||||
fs.chmodSync(manifestPath, 0o600);
|
||||
|
||||
return { backupDir, manifestPath, manifest };
|
||||
}
|
||||
|
||||
function restoreFromManifest(manifest: BackupManifest): {
|
||||
restoredFiles: number;
|
||||
deletedFiles: number;
|
||||
} {
|
||||
let restoredFiles = 0;
|
||||
let deletedFiles = 0;
|
||||
|
||||
for (const entry of manifest.entries) {
|
||||
if (!entry.existed) {
|
||||
if (fs.existsSync(entry.path)) {
|
||||
fs.rmSync(entry.path, { force: true });
|
||||
deletedFiles += 1;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!entry.backupPath || !fs.existsSync(entry.backupPath)) {
|
||||
throw new Error(`Backup file is missing for ${entry.path}.`);
|
||||
}
|
||||
ensureDirForFile(entry.path);
|
||||
fs.copyFileSync(entry.backupPath, entry.path);
|
||||
fs.chmodSync(entry.path, entry.mode ?? 0o600);
|
||||
restoredFiles += 1;
|
||||
}
|
||||
|
||||
return { restoredFiles, deletedFiles };
|
||||
}
|
||||
|
||||
function pruneOldBackups(stateDir: string): void {
|
||||
const backupRoot = resolveBackupRoot(stateDir);
|
||||
if (!fs.existsSync(backupRoot)) {
|
||||
return;
|
||||
}
|
||||
const dirs = fs
|
||||
.readdirSync(backupRoot, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory())
|
||||
.map((entry) => entry.name)
|
||||
.toSorted();
|
||||
|
||||
if (dirs.length <= BACKUP_RETENTION) {
|
||||
return;
|
||||
}
|
||||
|
||||
const toDelete = dirs.slice(0, Math.max(0, dirs.length - BACKUP_RETENTION));
|
||||
for (const dir of toDelete) {
|
||||
fs.rmSync(path.join(backupRoot, dir), { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function buildMigrationPlan(params: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
scrubEnv: boolean;
|
||||
}): Promise<MigrationPlan> {
|
||||
const io = createConfigIO({ env: params.env });
|
||||
const { snapshot, writeOptions } = await io.readConfigFileSnapshotForWrite();
|
||||
if (!snapshot.valid) {
|
||||
const issues =
|
||||
snapshot.issues.length > 0
|
||||
? snapshot.issues.map((issue) => `${issue.path || "<root>"}: ${issue.message}`).join("\n")
|
||||
: "Unknown validation issue.";
|
||||
throw new Error(`Cannot migrate secrets because config is invalid:\n${issues}`);
|
||||
}
|
||||
|
||||
const stateDir = resolveStateDir(params.env, os.homedir);
|
||||
const nextConfig = structuredClone(snapshot.config);
|
||||
const fileSource = resolveFileSource(nextConfig, params.env);
|
||||
const previousPayload = await decryptSopsJson(fileSource.path, fileSource.timeoutMs);
|
||||
const nextPayload = structuredClone(previousPayload);
|
||||
|
||||
const counters: MigrationCounters = {
|
||||
configRefs: 0,
|
||||
authProfileRefs: 0,
|
||||
plaintextRemoved: 0,
|
||||
secretsWritten: 0,
|
||||
envEntriesRemoved: 0,
|
||||
authStoresChanged: 0,
|
||||
};
|
||||
|
||||
const migratedValues = new Set<string>();
|
||||
|
||||
migrateModelProviderSecrets({
|
||||
config: nextConfig,
|
||||
payload: nextPayload,
|
||||
counters,
|
||||
migratedValues,
|
||||
});
|
||||
migrateSkillEntrySecrets({
|
||||
config: nextConfig,
|
||||
payload: nextPayload,
|
||||
counters,
|
||||
migratedValues,
|
||||
});
|
||||
migrateGoogleChatSecrets({
|
||||
config: nextConfig,
|
||||
payload: nextPayload,
|
||||
counters,
|
||||
});
|
||||
|
||||
const authStoreChanges: AuthStoreChange[] = [];
|
||||
for (const authStorePath of collectAuthStorePaths(nextConfig, stateDir)) {
|
||||
if (!fs.existsSync(authStorePath)) {
|
||||
continue;
|
||||
}
|
||||
const raw = fs.readFileSync(authStorePath, "utf8");
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw) as unknown;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!isRecord(parsed)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const nextStore = structuredClone(parsed);
|
||||
const scope = deriveAuthStoreScope(authStorePath, stateDir);
|
||||
const changed = migrateAuthStoreSecrets({
|
||||
store: nextStore,
|
||||
scope,
|
||||
payload: nextPayload,
|
||||
counters,
|
||||
migratedValues,
|
||||
});
|
||||
if (!changed) {
|
||||
continue;
|
||||
}
|
||||
authStoreChanges.push({ path: authStorePath, nextStore });
|
||||
}
|
||||
counters.authStoresChanged = authStoreChanges.length;
|
||||
|
||||
if (counters.secretsWritten > 0 && !fileSource.hadConfiguredSource) {
|
||||
const defaultConfigPath = resolveDefaultSecretsConfigPath(params.env);
|
||||
nextConfig.secrets ??= {};
|
||||
nextConfig.secrets.sources ??= {};
|
||||
nextConfig.secrets.sources.file = {
|
||||
type: "sops",
|
||||
path: defaultConfigPath,
|
||||
timeoutMs: DEFAULT_SOPS_TIMEOUT_MS,
|
||||
};
|
||||
}
|
||||
|
||||
const configChanged = !isDeepStrictEqual(snapshot.config, nextConfig);
|
||||
const payloadChanged = !isDeepStrictEqual(previousPayload, nextPayload);
|
||||
|
||||
let envChange: EnvChange | null = null;
|
||||
if (params.scrubEnv && migratedValues.size > 0) {
|
||||
const envPath = path.join(resolveConfigDir(params.env, os.homedir), ".env");
|
||||
if (fs.existsSync(envPath)) {
|
||||
const rawEnv = fs.readFileSync(envPath, "utf8");
|
||||
const scrubbed = scrubEnvRaw(rawEnv, migratedValues, new Set(listKnownSecretEnvVarNames()));
|
||||
if (scrubbed.removed > 0 && scrubbed.nextRaw !== rawEnv) {
|
||||
counters.envEntriesRemoved = scrubbed.removed;
|
||||
envChange = {
|
||||
path: envPath,
|
||||
nextRaw: scrubbed.nextRaw,
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const backupTargets = new Set<string>();
|
||||
if (configChanged) {
|
||||
backupTargets.add(io.configPath);
|
||||
}
|
||||
if (payloadChanged) {
|
||||
backupTargets.add(fileSource.path);
|
||||
}
|
||||
for (const change of authStoreChanges) {
|
||||
backupTargets.add(change.path);
|
||||
}
|
||||
if (envChange) {
|
||||
backupTargets.add(envChange.path);
|
||||
}
|
||||
|
||||
return {
|
||||
changed: configChanged || payloadChanged || authStoreChanges.length > 0 || Boolean(envChange),
|
||||
counters,
|
||||
stateDir,
|
||||
configChanged,
|
||||
nextConfig,
|
||||
configWriteOptions: writeOptions,
|
||||
authStoreChanges,
|
||||
payloadChanged,
|
||||
nextPayload,
|
||||
secretsFilePath: fileSource.path,
|
||||
secretsFileTimeoutMs: fileSource.timeoutMs,
|
||||
envChange,
|
||||
backupTargets: [...backupTargets],
|
||||
};
|
||||
}
|
||||
|
||||
export async function runSecretsMigration(
|
||||
options: SecretsMigrationRunOptions = {},
|
||||
): Promise<SecretsMigrationRunResult> {
|
||||
const env = options.env ?? process.env;
|
||||
const scrubEnv = options.scrubEnv ?? true;
|
||||
const plan = await buildMigrationPlan({ env, scrubEnv });
|
||||
|
||||
if (!options.write) {
|
||||
return {
|
||||
mode: "dry-run",
|
||||
changed: plan.changed,
|
||||
secretsFilePath: plan.secretsFilePath,
|
||||
counters: plan.counters,
|
||||
changedFiles: plan.backupTargets,
|
||||
};
|
||||
}
|
||||
|
||||
if (!plan.changed) {
|
||||
return {
|
||||
mode: "write",
|
||||
changed: false,
|
||||
secretsFilePath: plan.secretsFilePath,
|
||||
counters: plan.counters,
|
||||
changedFiles: [],
|
||||
};
|
||||
}
|
||||
|
||||
const now = options.now ?? new Date();
|
||||
const backupId = formatBackupId(now);
|
||||
const backup = createBackupManifest({
|
||||
stateDir: plan.stateDir,
|
||||
targets: plan.backupTargets,
|
||||
backupId,
|
||||
now,
|
||||
});
|
||||
|
||||
try {
|
||||
if (plan.payloadChanged) {
|
||||
await encryptSopsJson({
|
||||
pathname: plan.secretsFilePath,
|
||||
timeoutMs: plan.secretsFileTimeoutMs,
|
||||
payload: plan.nextPayload,
|
||||
});
|
||||
}
|
||||
|
||||
if (plan.configChanged) {
|
||||
const io = createConfigIO({ env });
|
||||
await io.writeConfigFile(plan.nextConfig, plan.configWriteOptions);
|
||||
}
|
||||
|
||||
for (const change of plan.authStoreChanges) {
|
||||
saveJsonFile(change.path, change.nextStore);
|
||||
}
|
||||
|
||||
if (plan.envChange) {
|
||||
ensureDirForFile(plan.envChange.path);
|
||||
fs.writeFileSync(plan.envChange.path, plan.envChange.nextRaw, "utf8");
|
||||
fs.chmodSync(plan.envChange.path, 0o600);
|
||||
}
|
||||
} catch (err) {
|
||||
restoreFromManifest(backup.manifest);
|
||||
throw new Error(
|
||||
`Secrets migration failed and was rolled back from backup ${backupId}: ${String(err)}`,
|
||||
{
|
||||
cause: err,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
pruneOldBackups(plan.stateDir);
|
||||
|
||||
return {
|
||||
mode: "write",
|
||||
changed: true,
|
||||
backupId,
|
||||
backupDir: backup.backupDir,
|
||||
secretsFilePath: plan.secretsFilePath,
|
||||
counters: plan.counters,
|
||||
changedFiles: plan.backupTargets,
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveSecretsMigrationBackupRoot(env: NodeJS.ProcessEnv = process.env): string {
|
||||
return resolveBackupRoot(resolveStateDir(env, os.homedir));
|
||||
}
|
||||
|
||||
export function listSecretsMigrationBackups(env: NodeJS.ProcessEnv = process.env): string[] {
|
||||
const root = resolveSecretsMigrationBackupRoot(env);
|
||||
if (!fs.existsSync(root)) {
|
||||
return [];
|
||||
}
|
||||
return fs
|
||||
.readdirSync(root, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory())
|
||||
.map((entry) => entry.name)
|
||||
.toSorted();
|
||||
}
|
||||
|
||||
export async function rollbackSecretsMigration(
|
||||
options: SecretsMigrationRollbackOptions,
|
||||
): Promise<SecretsMigrationRollbackResult> {
|
||||
const env = options.env ?? process.env;
|
||||
const backupDir = path.join(resolveSecretsMigrationBackupRoot(env), options.backupId);
|
||||
const manifestPath = path.join(backupDir, BACKUP_MANIFEST_FILENAME);
|
||||
if (!fs.existsSync(manifestPath)) {
|
||||
const available = listSecretsMigrationBackups(env);
|
||||
const suffix =
|
||||
available.length > 0
|
||||
? ` Available backups: ${available.slice(-10).join(", ")}`
|
||||
: " No backups were found.";
|
||||
throw new Error(`Backup "${options.backupId}" was not found.${suffix}`);
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(fs.readFileSync(manifestPath, "utf8")) as unknown;
|
||||
} catch (err) {
|
||||
throw new Error(`Failed to read backup manifest at ${manifestPath}: ${String(err)}`, {
|
||||
cause: err,
|
||||
});
|
||||
}
|
||||
|
||||
if (!isRecord(parsed) || !Array.isArray(parsed.entries)) {
|
||||
throw new Error(`Backup manifest at ${manifestPath} is invalid.`);
|
||||
}
|
||||
|
||||
const manifest = parsed as BackupManifest;
|
||||
const restored = restoreFromManifest(manifest);
|
||||
return {
|
||||
backupId: options.backupId,
|
||||
restoredFiles: restored.restoredFiles,
|
||||
deletedFiles: restored.deletedFiles,
|
||||
};
|
||||
}
|
||||
28
src/secrets/provider-env-vars.ts
Normal file
28
src/secrets/provider-env-vars.ts
Normal file
@@ -0,0 +1,28 @@
|
||||
export const PROVIDER_ENV_VARS: Record<string, readonly string[]> = {
|
||||
openai: ["OPENAI_API_KEY"],
|
||||
anthropic: ["ANTHROPIC_API_KEY"],
|
||||
google: ["GEMINI_API_KEY"],
|
||||
minimax: ["MINIMAX_API_KEY"],
|
||||
"minimax-cn": ["MINIMAX_API_KEY"],
|
||||
moonshot: ["MOONSHOT_API_KEY"],
|
||||
"kimi-coding": ["KIMI_API_KEY", "KIMICODE_API_KEY"],
|
||||
synthetic: ["SYNTHETIC_API_KEY"],
|
||||
venice: ["VENICE_API_KEY"],
|
||||
zai: ["ZAI_API_KEY", "Z_AI_API_KEY"],
|
||||
xiaomi: ["XIAOMI_API_KEY"],
|
||||
openrouter: ["OPENROUTER_API_KEY"],
|
||||
"cloudflare-ai-gateway": ["CLOUDFLARE_AI_GATEWAY_API_KEY"],
|
||||
litellm: ["LITELLM_API_KEY"],
|
||||
"vercel-ai-gateway": ["AI_GATEWAY_API_KEY"],
|
||||
opencode: ["OPENCODE_API_KEY", "OPENCODE_ZEN_API_KEY"],
|
||||
together: ["TOGETHER_API_KEY"],
|
||||
huggingface: ["HUGGINGFACE_HUB_TOKEN", "HF_TOKEN"],
|
||||
qianfan: ["QIANFAN_API_KEY"],
|
||||
xai: ["XAI_API_KEY"],
|
||||
volcengine: ["VOLCANO_ENGINE_API_KEY"],
|
||||
byteplus: ["BYTEPLUS_API_KEY"],
|
||||
};
|
||||
|
||||
export function listKnownSecretEnvVarNames(): string[] {
|
||||
return [...new Set(Object.values(PROVIDER_ENV_VARS).flatMap((keys) => keys))];
|
||||
}
|
||||
169
src/secrets/runtime.test.ts
Normal file
169
src/secrets/runtime.test.ts
Normal file
@@ -0,0 +1,169 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { ensureAuthProfileStore } from "../agents/auth-profiles.js";
|
||||
import { loadConfig, type OpenClawConfig } from "../config/config.js";
|
||||
import {
|
||||
activateSecretsRuntimeSnapshot,
|
||||
clearSecretsRuntimeSnapshot,
|
||||
prepareSecretsRuntimeSnapshot,
|
||||
} from "./runtime.js";
|
||||
|
||||
const runExecMock = vi.hoisted(() => vi.fn());
|
||||
|
||||
vi.mock("../process/exec.js", () => ({
|
||||
runExec: runExecMock,
|
||||
}));
|
||||
|
||||
describe("secrets runtime snapshot", () => {
|
||||
afterEach(() => {
|
||||
runExecMock.mockReset();
|
||||
clearSecretsRuntimeSnapshot();
|
||||
});
|
||||
|
||||
it("resolves env refs for config and auth profiles", async () => {
|
||||
const config: OpenClawConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "env", id: "OPENAI_API_KEY" },
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
skills: {
|
||||
entries: {
|
||||
"review-pr": {
|
||||
enabled: true,
|
||||
apiKey: { source: "env", id: "REVIEW_SKILL_API_KEY" },
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
const snapshot = await prepareSecretsRuntimeSnapshot({
|
||||
config,
|
||||
env: {
|
||||
OPENAI_API_KEY: "sk-env-openai",
|
||||
GITHUB_TOKEN: "ghp-env-token",
|
||||
REVIEW_SKILL_API_KEY: "sk-skill-ref",
|
||||
},
|
||||
agentDirs: ["/tmp/openclaw-agent-main"],
|
||||
loadAuthStore: () => ({
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openai:default": {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
key: "old-openai",
|
||||
keyRef: { source: "env", id: "OPENAI_API_KEY" },
|
||||
},
|
||||
"github-copilot:default": {
|
||||
type: "token",
|
||||
provider: "github-copilot",
|
||||
token: "old-gh",
|
||||
tokenRef: { source: "env", id: "GITHUB_TOKEN" },
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
expect(snapshot.config.models?.providers?.openai?.apiKey).toBe("sk-env-openai");
|
||||
expect(snapshot.config.skills?.entries?.["review-pr"]?.apiKey).toBe("sk-skill-ref");
|
||||
expect(snapshot.warnings).toHaveLength(2);
|
||||
expect(snapshot.authStores[0]?.store.profiles["openai:default"]).toMatchObject({
|
||||
type: "api_key",
|
||||
key: "sk-env-openai",
|
||||
});
|
||||
expect(snapshot.authStores[0]?.store.profiles["github-copilot:default"]).toMatchObject({
|
||||
type: "token",
|
||||
token: "ghp-env-token",
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves file refs via sops json payload", async () => {
|
||||
runExecMock.mockResolvedValueOnce({
|
||||
stdout: JSON.stringify({
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: "sk-from-sops",
|
||||
},
|
||||
},
|
||||
}),
|
||||
stderr: "",
|
||||
});
|
||||
|
||||
const config: OpenClawConfig = {
|
||||
secrets: {
|
||||
sources: {
|
||||
file: {
|
||||
type: "sops",
|
||||
path: "~/.openclaw/secrets.enc.json",
|
||||
timeoutMs: 7000,
|
||||
},
|
||||
},
|
||||
},
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "file", id: "/providers/openai/apiKey" },
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
const snapshot = await prepareSecretsRuntimeSnapshot({
|
||||
config,
|
||||
agentDirs: ["/tmp/openclaw-agent-main"],
|
||||
loadAuthStore: () => ({ version: 1, profiles: {} }),
|
||||
});
|
||||
|
||||
expect(snapshot.config.models?.providers?.openai?.apiKey).toBe("sk-from-sops");
|
||||
expect(runExecMock).toHaveBeenCalledWith(
|
||||
"sops",
|
||||
["--decrypt", "--output-type", "json", expect.stringContaining("secrets.enc.json")],
|
||||
{
|
||||
timeoutMs: 7000,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("activates runtime snapshots for loadConfig and ensureAuthProfileStore", async () => {
|
||||
const prepared = await prepareSecretsRuntimeSnapshot({
|
||||
config: {
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
apiKey: { source: "env", id: "OPENAI_API_KEY" },
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
env: { OPENAI_API_KEY: "sk-runtime" },
|
||||
agentDirs: ["/tmp/openclaw-agent-main"],
|
||||
loadAuthStore: () => ({
|
||||
version: 1,
|
||||
profiles: {
|
||||
"openai:default": {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
keyRef: { source: "env", id: "OPENAI_API_KEY" },
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
activateSecretsRuntimeSnapshot(prepared);
|
||||
|
||||
expect(loadConfig().models?.providers?.openai?.apiKey).toBe("sk-runtime");
|
||||
const store = ensureAuthProfileStore("/tmp/openclaw-agent-main");
|
||||
expect(store.profiles["openai:default"]).toMatchObject({
|
||||
type: "api_key",
|
||||
key: "sk-runtime",
|
||||
});
|
||||
});
|
||||
});
|
||||
337
src/secrets/runtime.ts
Normal file
337
src/secrets/runtime.ts
Normal file
@@ -0,0 +1,337 @@
|
||||
import { resolveOpenClawAgentDir } from "../agents/agent-paths.js";
|
||||
import { listAgentIds, resolveAgentDir } from "../agents/agent-scope.js";
|
||||
import type { AuthProfileCredential, AuthProfileStore } from "../agents/auth-profiles.js";
|
||||
import {
|
||||
clearRuntimeAuthProfileStoreSnapshots,
|
||||
loadAuthProfileStoreForRuntime,
|
||||
replaceRuntimeAuthProfileStoreSnapshots,
|
||||
} from "../agents/auth-profiles.js";
|
||||
import {
|
||||
clearRuntimeConfigSnapshot,
|
||||
setRuntimeConfigSnapshot,
|
||||
type OpenClawConfig,
|
||||
} from "../config/config.js";
|
||||
import { isSecretRef, type SecretRef } from "../config/types.secrets.js";
|
||||
import { resolveUserPath } from "../utils.js";
|
||||
import { readJsonPointer } from "./json-pointer.js";
|
||||
import { decryptSopsJsonFile, DEFAULT_SOPS_TIMEOUT_MS } from "./sops.js";
|
||||
|
||||
type SecretResolverWarningCode = "SECRETS_REF_OVERRIDES_PLAINTEXT";
|
||||
|
||||
export type SecretResolverWarning = {
|
||||
code: SecretResolverWarningCode;
|
||||
path: string;
|
||||
message: string;
|
||||
};
|
||||
|
||||
export type PreparedSecretsRuntimeSnapshot = {
|
||||
sourceConfig: OpenClawConfig;
|
||||
config: OpenClawConfig;
|
||||
authStores: Array<{ agentDir: string; store: AuthProfileStore }>;
|
||||
warnings: SecretResolverWarning[];
|
||||
};
|
||||
|
||||
type ResolverContext = {
|
||||
config: OpenClawConfig;
|
||||
env: NodeJS.ProcessEnv;
|
||||
fileSecretsPromise: Promise<unknown> | null;
|
||||
};
|
||||
|
||||
type ProviderLike = {
|
||||
apiKey?: unknown;
|
||||
};
|
||||
|
||||
type SkillEntryLike = {
|
||||
apiKey?: unknown;
|
||||
};
|
||||
|
||||
type GoogleChatAccountLike = {
|
||||
serviceAccount?: unknown;
|
||||
serviceAccountRef?: unknown;
|
||||
accounts?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
type ApiKeyCredentialLike = AuthProfileCredential & {
|
||||
type: "api_key";
|
||||
key?: string;
|
||||
keyRef?: unknown;
|
||||
};
|
||||
|
||||
type TokenCredentialLike = AuthProfileCredential & {
|
||||
type: "token";
|
||||
token?: string;
|
||||
tokenRef?: unknown;
|
||||
};
|
||||
|
||||
let activeSnapshot: PreparedSecretsRuntimeSnapshot | null = null;
|
||||
|
||||
function cloneSnapshot(snapshot: PreparedSecretsRuntimeSnapshot): PreparedSecretsRuntimeSnapshot {
|
||||
return {
|
||||
sourceConfig: structuredClone(snapshot.sourceConfig),
|
||||
config: structuredClone(snapshot.config),
|
||||
authStores: snapshot.authStores.map((entry) => ({
|
||||
agentDir: entry.agentDir,
|
||||
store: structuredClone(entry.store),
|
||||
})),
|
||||
warnings: snapshot.warnings.map((warning) => ({ ...warning })),
|
||||
};
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function isNonEmptyString(value: unknown): value is string {
|
||||
return typeof value === "string" && value.trim().length > 0;
|
||||
}
|
||||
|
||||
async function decryptSopsFile(config: OpenClawConfig): Promise<unknown> {
|
||||
const fileSource = config.secrets?.sources?.file;
|
||||
if (!fileSource) {
|
||||
throw new Error(
|
||||
`Secret reference source "file" is not configured. Configure secrets.sources.file first.`,
|
||||
);
|
||||
}
|
||||
if (fileSource.type !== "sops") {
|
||||
throw new Error(`Unsupported secrets.sources.file.type "${String(fileSource.type)}".`);
|
||||
}
|
||||
|
||||
const resolvedPath = resolveUserPath(fileSource.path);
|
||||
const timeoutMs =
|
||||
typeof fileSource.timeoutMs === "number" && Number.isFinite(fileSource.timeoutMs)
|
||||
? Math.max(1, Math.floor(fileSource.timeoutMs))
|
||||
: DEFAULT_SOPS_TIMEOUT_MS;
|
||||
return await decryptSopsJsonFile({
|
||||
path: resolvedPath,
|
||||
timeoutMs,
|
||||
missingBinaryMessage:
|
||||
"sops binary not found in PATH. Install sops >= 3.9.0 or disable secrets.sources.file.",
|
||||
});
|
||||
}
|
||||
|
||||
async function resolveSecretRefValue(ref: SecretRef, context: ResolverContext): Promise<unknown> {
|
||||
const id = ref.id.trim();
|
||||
if (!id) {
|
||||
throw new Error(`Secret reference id is empty.`);
|
||||
}
|
||||
|
||||
if (ref.source === "env") {
|
||||
const envValue = context.env[id];
|
||||
if (!isNonEmptyString(envValue)) {
|
||||
throw new Error(`Environment variable "${id}" is missing or empty.`);
|
||||
}
|
||||
return envValue;
|
||||
}
|
||||
|
||||
if (ref.source === "file") {
|
||||
context.fileSecretsPromise ??= decryptSopsFile(context.config);
|
||||
const payload = await context.fileSecretsPromise;
|
||||
return readJsonPointer(payload, id, { onMissing: "throw" });
|
||||
}
|
||||
|
||||
throw new Error(`Unsupported secret source "${String((ref as { source?: unknown }).source)}".`);
|
||||
}
|
||||
|
||||
async function resolveGoogleChatServiceAccount(
|
||||
target: GoogleChatAccountLike,
|
||||
path: string,
|
||||
context: ResolverContext,
|
||||
warnings: SecretResolverWarning[],
|
||||
): Promise<void> {
|
||||
const explicitRef = isSecretRef(target.serviceAccountRef) ? target.serviceAccountRef : null;
|
||||
const inlineRef = isSecretRef(target.serviceAccount) ? target.serviceAccount : null;
|
||||
const ref = explicitRef ?? inlineRef;
|
||||
if (!ref) {
|
||||
return;
|
||||
}
|
||||
if (explicitRef && target.serviceAccount !== undefined && !isSecretRef(target.serviceAccount)) {
|
||||
warnings.push({
|
||||
code: "SECRETS_REF_OVERRIDES_PLAINTEXT",
|
||||
path,
|
||||
message: `${path}: serviceAccountRef is set; runtime will ignore plaintext serviceAccount.`,
|
||||
});
|
||||
}
|
||||
target.serviceAccount = await resolveSecretRefValue(ref, context);
|
||||
}
|
||||
|
||||
async function resolveConfigSecretRefs(params: {
|
||||
config: OpenClawConfig;
|
||||
context: ResolverContext;
|
||||
warnings: SecretResolverWarning[];
|
||||
}): Promise<OpenClawConfig> {
|
||||
const resolved = structuredClone(params.config);
|
||||
const providers = resolved.models?.providers as Record<string, ProviderLike> | undefined;
|
||||
if (providers) {
|
||||
for (const [providerId, provider] of Object.entries(providers)) {
|
||||
if (!isSecretRef(provider.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
const resolvedValue = await resolveSecretRefValue(provider.apiKey, params.context);
|
||||
if (!isNonEmptyString(resolvedValue)) {
|
||||
throw new Error(
|
||||
`models.providers.${providerId}.apiKey resolved to a non-string or empty value.`,
|
||||
);
|
||||
}
|
||||
provider.apiKey = resolvedValue;
|
||||
}
|
||||
}
|
||||
|
||||
const skillEntries = resolved.skills?.entries as Record<string, SkillEntryLike> | undefined;
|
||||
if (skillEntries) {
|
||||
for (const [skillKey, entry] of Object.entries(skillEntries)) {
|
||||
if (!isSecretRef(entry.apiKey)) {
|
||||
continue;
|
||||
}
|
||||
const resolvedValue = await resolveSecretRefValue(entry.apiKey, params.context);
|
||||
if (!isNonEmptyString(resolvedValue)) {
|
||||
throw new Error(
|
||||
`skills.entries.${skillKey}.apiKey resolved to a non-string or empty value.`,
|
||||
);
|
||||
}
|
||||
entry.apiKey = resolvedValue;
|
||||
}
|
||||
}
|
||||
|
||||
const googleChat = resolved.channels?.googlechat as GoogleChatAccountLike | undefined;
|
||||
if (googleChat) {
|
||||
await resolveGoogleChatServiceAccount(
|
||||
googleChat,
|
||||
"channels.googlechat",
|
||||
params.context,
|
||||
params.warnings,
|
||||
);
|
||||
if (isRecord(googleChat.accounts)) {
|
||||
for (const [accountId, account] of Object.entries(googleChat.accounts)) {
|
||||
if (!isRecord(account)) {
|
||||
continue;
|
||||
}
|
||||
await resolveGoogleChatServiceAccount(
|
||||
account as GoogleChatAccountLike,
|
||||
`channels.googlechat.accounts.${accountId}`,
|
||||
params.context,
|
||||
params.warnings,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return resolved;
|
||||
}
|
||||
|
||||
async function resolveAuthStoreSecretRefs(params: {
|
||||
store: AuthProfileStore;
|
||||
context: ResolverContext;
|
||||
warnings: SecretResolverWarning[];
|
||||
agentDir: string;
|
||||
}): Promise<AuthProfileStore> {
|
||||
const resolvedStore = structuredClone(params.store);
|
||||
for (const [profileId, profile] of Object.entries(resolvedStore.profiles)) {
|
||||
if (profile.type === "api_key") {
|
||||
const apiProfile = profile as ApiKeyCredentialLike;
|
||||
const keyRef = isSecretRef(apiProfile.keyRef) ? apiProfile.keyRef : null;
|
||||
if (keyRef && isNonEmptyString(apiProfile.key)) {
|
||||
params.warnings.push({
|
||||
code: "SECRETS_REF_OVERRIDES_PLAINTEXT",
|
||||
path: `${params.agentDir}.auth-profiles.${profileId}.key`,
|
||||
message: `auth-profiles ${profileId}: keyRef is set; runtime will ignore plaintext key.`,
|
||||
});
|
||||
}
|
||||
if (keyRef) {
|
||||
const resolvedValue = await resolveSecretRefValue(keyRef, params.context);
|
||||
if (!isNonEmptyString(resolvedValue)) {
|
||||
throw new Error(`auth profile "${profileId}" keyRef resolved to an empty value.`);
|
||||
}
|
||||
apiProfile.key = resolvedValue;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (profile.type === "token") {
|
||||
const tokenProfile = profile as TokenCredentialLike;
|
||||
const tokenRef = isSecretRef(tokenProfile.tokenRef) ? tokenProfile.tokenRef : null;
|
||||
if (tokenRef && isNonEmptyString(tokenProfile.token)) {
|
||||
params.warnings.push({
|
||||
code: "SECRETS_REF_OVERRIDES_PLAINTEXT",
|
||||
path: `${params.agentDir}.auth-profiles.${profileId}.token`,
|
||||
message: `auth-profiles ${profileId}: tokenRef is set; runtime will ignore plaintext token.`,
|
||||
});
|
||||
}
|
||||
if (tokenRef) {
|
||||
const resolvedValue = await resolveSecretRefValue(tokenRef, params.context);
|
||||
if (!isNonEmptyString(resolvedValue)) {
|
||||
throw new Error(`auth profile "${profileId}" tokenRef resolved to an empty value.`);
|
||||
}
|
||||
tokenProfile.token = resolvedValue;
|
||||
}
|
||||
}
|
||||
}
|
||||
return resolvedStore;
|
||||
}
|
||||
|
||||
function collectCandidateAgentDirs(config: OpenClawConfig): string[] {
|
||||
const dirs = new Set<string>();
|
||||
dirs.add(resolveUserPath(resolveOpenClawAgentDir()));
|
||||
for (const agentId of listAgentIds(config)) {
|
||||
dirs.add(resolveUserPath(resolveAgentDir(config, agentId)));
|
||||
}
|
||||
return [...dirs];
|
||||
}
|
||||
|
||||
export async function prepareSecretsRuntimeSnapshot(params: {
|
||||
config: OpenClawConfig;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
agentDirs?: string[];
|
||||
loadAuthStore?: (agentDir?: string) => AuthProfileStore;
|
||||
}): Promise<PreparedSecretsRuntimeSnapshot> {
|
||||
const warnings: SecretResolverWarning[] = [];
|
||||
const context: ResolverContext = {
|
||||
config: params.config,
|
||||
env: params.env ?? process.env,
|
||||
fileSecretsPromise: null,
|
||||
};
|
||||
const resolvedConfig = await resolveConfigSecretRefs({
|
||||
config: params.config,
|
||||
context,
|
||||
warnings,
|
||||
});
|
||||
|
||||
const loadAuthStore = params.loadAuthStore ?? loadAuthProfileStoreForRuntime;
|
||||
const candidateDirs = params.agentDirs?.length
|
||||
? [...new Set(params.agentDirs.map((entry) => resolveUserPath(entry)))]
|
||||
: collectCandidateAgentDirs(resolvedConfig);
|
||||
const authStores: Array<{ agentDir: string; store: AuthProfileStore }> = [];
|
||||
for (const agentDir of candidateDirs) {
|
||||
const rawStore = loadAuthStore(agentDir);
|
||||
const resolvedStore = await resolveAuthStoreSecretRefs({
|
||||
store: rawStore,
|
||||
context,
|
||||
warnings,
|
||||
agentDir,
|
||||
});
|
||||
authStores.push({ agentDir, store: resolvedStore });
|
||||
}
|
||||
|
||||
return {
|
||||
sourceConfig: structuredClone(params.config),
|
||||
config: resolvedConfig,
|
||||
authStores,
|
||||
warnings,
|
||||
};
|
||||
}
|
||||
|
||||
export function activateSecretsRuntimeSnapshot(snapshot: PreparedSecretsRuntimeSnapshot): void {
|
||||
const next = cloneSnapshot(snapshot);
|
||||
setRuntimeConfigSnapshot(next.config, next.sourceConfig);
|
||||
replaceRuntimeAuthProfileStoreSnapshots(next.authStores);
|
||||
activeSnapshot = next;
|
||||
}
|
||||
|
||||
export function getActiveSecretsRuntimeSnapshot(): PreparedSecretsRuntimeSnapshot | null {
|
||||
return activeSnapshot ? cloneSnapshot(activeSnapshot) : null;
|
||||
}
|
||||
|
||||
export function clearSecretsRuntimeSnapshot(): void {
|
||||
activeSnapshot = null;
|
||||
clearRuntimeConfigSnapshot();
|
||||
clearRuntimeAuthProfileStoreSnapshots();
|
||||
}
|
||||
120
src/secrets/sops.ts
Normal file
120
src/secrets/sops.ts
Normal file
@@ -0,0 +1,120 @@
|
||||
import crypto from "node:crypto";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { runExec } from "../process/exec.js";
|
||||
|
||||
export const DEFAULT_SOPS_TIMEOUT_MS = 5_000;
|
||||
const MAX_SOPS_OUTPUT_BYTES = 10 * 1024 * 1024;
|
||||
|
||||
function normalizeTimeoutMs(value: number | undefined): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) {
|
||||
return Math.max(1, Math.floor(value));
|
||||
}
|
||||
return DEFAULT_SOPS_TIMEOUT_MS;
|
||||
}
|
||||
|
||||
function isTimeoutError(message: string | undefined): boolean {
|
||||
return typeof message === "string" && message.toLowerCase().includes("timed out");
|
||||
}
|
||||
|
||||
type SopsErrorContext = {
|
||||
missingBinaryMessage: string;
|
||||
operationLabel: string;
|
||||
};
|
||||
|
||||
function toSopsError(err: unknown, params: SopsErrorContext): Error {
|
||||
const error = err as NodeJS.ErrnoException & { message?: string };
|
||||
if (error.code === "ENOENT") {
|
||||
return new Error(params.missingBinaryMessage, { cause: err });
|
||||
}
|
||||
return new Error(`${params.operationLabel}: ${String(error.message ?? err)}`, {
|
||||
cause: err,
|
||||
});
|
||||
}
|
||||
|
||||
function ensureDirForFile(filePath: string): void {
|
||||
fs.mkdirSync(path.dirname(filePath), { recursive: true, mode: 0o700 });
|
||||
}
|
||||
|
||||
export async function decryptSopsJsonFile(params: {
|
||||
path: string;
|
||||
timeoutMs?: number;
|
||||
missingBinaryMessage: string;
|
||||
}): Promise<unknown> {
|
||||
const timeoutMs = normalizeTimeoutMs(params.timeoutMs);
|
||||
try {
|
||||
const { stdout } = await runExec("sops", ["--decrypt", "--output-type", "json", params.path], {
|
||||
timeoutMs,
|
||||
maxBuffer: MAX_SOPS_OUTPUT_BYTES,
|
||||
});
|
||||
return JSON.parse(stdout) as unknown;
|
||||
} catch (err) {
|
||||
const error = err as NodeJS.ErrnoException & { message?: string };
|
||||
if (isTimeoutError(error.message)) {
|
||||
throw new Error(`sops decrypt timed out after ${timeoutMs}ms for ${params.path}.`, {
|
||||
cause: err,
|
||||
});
|
||||
}
|
||||
throw toSopsError(err, {
|
||||
missingBinaryMessage: params.missingBinaryMessage,
|
||||
operationLabel: `sops decrypt failed for ${params.path}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export async function encryptSopsJsonFile(params: {
|
||||
path: string;
|
||||
payload: Record<string, unknown>;
|
||||
timeoutMs?: number;
|
||||
missingBinaryMessage: string;
|
||||
}): Promise<void> {
|
||||
ensureDirForFile(params.path);
|
||||
const timeoutMs = normalizeTimeoutMs(params.timeoutMs);
|
||||
const tmpPlain = path.join(
|
||||
path.dirname(params.path),
|
||||
`${path.basename(params.path)}.${process.pid}.${crypto.randomUUID()}.plain.tmp`,
|
||||
);
|
||||
const tmpEncrypted = path.join(
|
||||
path.dirname(params.path),
|
||||
`${path.basename(params.path)}.${process.pid}.${crypto.randomUUID()}.enc.tmp`,
|
||||
);
|
||||
|
||||
fs.writeFileSync(tmpPlain, `${JSON.stringify(params.payload, null, 2)}\n`, "utf8");
|
||||
fs.chmodSync(tmpPlain, 0o600);
|
||||
|
||||
try {
|
||||
await runExec(
|
||||
"sops",
|
||||
[
|
||||
"--encrypt",
|
||||
"--input-type",
|
||||
"json",
|
||||
"--output-type",
|
||||
"json",
|
||||
"--output",
|
||||
tmpEncrypted,
|
||||
tmpPlain,
|
||||
],
|
||||
{
|
||||
timeoutMs,
|
||||
maxBuffer: MAX_SOPS_OUTPUT_BYTES,
|
||||
},
|
||||
);
|
||||
fs.renameSync(tmpEncrypted, params.path);
|
||||
fs.chmodSync(params.path, 0o600);
|
||||
} catch (err) {
|
||||
const error = err as NodeJS.ErrnoException & { message?: string };
|
||||
if (isTimeoutError(error.message)) {
|
||||
throw new Error(`sops encrypt timed out after ${timeoutMs}ms for ${params.path}.`, {
|
||||
cause: err,
|
||||
});
|
||||
}
|
||||
throw toSopsError(err, {
|
||||
missingBinaryMessage: params.missingBinaryMessage,
|
||||
operationLabel: `sops encrypt failed for ${params.path}`,
|
||||
});
|
||||
} finally {
|
||||
fs.rmSync(tmpPlain, { force: true });
|
||||
fs.rmSync(tmpEncrypted, { force: true });
|
||||
}
|
||||
}
|
||||
@@ -26,45 +26,11 @@ describe("roleScopesAllow", () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("treats operator.write as satisfied by write/admin scopes", () => {
|
||||
it("keeps non-read operator scopes explicit", () => {
|
||||
expect(
|
||||
roleScopesAllow({
|
||||
role: "operator",
|
||||
requestedScopes: ["operator.write"],
|
||||
allowedScopes: ["operator.write"],
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
roleScopesAllow({
|
||||
role: "operator",
|
||||
requestedScopes: ["operator.write"],
|
||||
allowedScopes: ["operator.admin"],
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("treats operator.approvals/operator.pairing as satisfied by operator.admin", () => {
|
||||
expect(
|
||||
roleScopesAllow({
|
||||
role: "operator",
|
||||
requestedScopes: ["operator.approvals"],
|
||||
allowedScopes: ["operator.admin"],
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
roleScopesAllow({
|
||||
role: "operator",
|
||||
requestedScopes: ["operator.pairing"],
|
||||
allowedScopes: ["operator.admin"],
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("does not treat operator.admin as satisfying non-operator scopes", () => {
|
||||
expect(
|
||||
roleScopesAllow({
|
||||
role: "operator",
|
||||
requestedScopes: ["system.run"],
|
||||
allowedScopes: ["operator.admin"],
|
||||
}),
|
||||
).toBe(false);
|
||||
|
||||
@@ -2,7 +2,6 @@ const OPERATOR_ROLE = "operator";
|
||||
const OPERATOR_ADMIN_SCOPE = "operator.admin";
|
||||
const OPERATOR_READ_SCOPE = "operator.read";
|
||||
const OPERATOR_WRITE_SCOPE = "operator.write";
|
||||
const OPERATOR_SCOPE_PREFIX = "operator.";
|
||||
|
||||
function normalizeScopeList(scopes: readonly string[]): string[] {
|
||||
const out = new Set<string>();
|
||||
@@ -16,14 +15,12 @@ function normalizeScopeList(scopes: readonly string[]): string[] {
|
||||
}
|
||||
|
||||
function operatorScopeSatisfied(requestedScope: string, granted: Set<string>): boolean {
|
||||
if (granted.has(OPERATOR_ADMIN_SCOPE) && requestedScope.startsWith(OPERATOR_SCOPE_PREFIX)) {
|
||||
return true;
|
||||
}
|
||||
if (requestedScope === OPERATOR_READ_SCOPE) {
|
||||
return granted.has(OPERATOR_READ_SCOPE) || granted.has(OPERATOR_WRITE_SCOPE);
|
||||
}
|
||||
if (requestedScope === OPERATOR_WRITE_SCOPE) {
|
||||
return granted.has(OPERATOR_WRITE_SCOPE);
|
||||
return (
|
||||
granted.has(OPERATOR_READ_SCOPE) ||
|
||||
granted.has(OPERATOR_WRITE_SCOPE) ||
|
||||
granted.has(OPERATOR_ADMIN_SCOPE)
|
||||
);
|
||||
}
|
||||
return granted.has(requestedScope);
|
||||
}
|
||||
|
||||
@@ -370,62 +370,6 @@ describe("Slack native command argument menus", () => {
|
||||
harness.postEphemeral.mockClear();
|
||||
});
|
||||
|
||||
it("registers options handlers without losing app receiver binding", async () => {
|
||||
const commands = new Map<string, (args: unknown) => Promise<void>>();
|
||||
const actions = new Map<string, (args: unknown) => Promise<void>>();
|
||||
const options = new Map<string, (args: unknown) => Promise<void>>();
|
||||
const postEphemeral = vi.fn().mockResolvedValue({ ok: true });
|
||||
const app = {
|
||||
client: { chat: { postEphemeral } },
|
||||
command: (name: string, handler: (args: unknown) => Promise<void>) => {
|
||||
commands.set(name, handler);
|
||||
},
|
||||
action: (id: string, handler: (args: unknown) => Promise<void>) => {
|
||||
actions.set(id, handler);
|
||||
},
|
||||
options: function (this: unknown, id: string, handler: (args: unknown) => Promise<void>) {
|
||||
expect(this).toBe(app);
|
||||
options.set(id, handler);
|
||||
},
|
||||
};
|
||||
const ctx = {
|
||||
cfg: { commands: { native: true, nativeSkills: false } },
|
||||
runtime: {},
|
||||
botToken: "bot-token",
|
||||
botUserId: "bot",
|
||||
teamId: "T1",
|
||||
allowFrom: ["*"],
|
||||
dmEnabled: true,
|
||||
dmPolicy: "open",
|
||||
groupDmEnabled: false,
|
||||
groupDmChannels: [],
|
||||
defaultRequireMention: true,
|
||||
groupPolicy: "open",
|
||||
useAccessGroups: false,
|
||||
channelsConfig: undefined,
|
||||
slashCommand: {
|
||||
enabled: true,
|
||||
name: "openclaw",
|
||||
ephemeral: true,
|
||||
sessionPrefix: "slack:slash",
|
||||
},
|
||||
textLimit: 4000,
|
||||
app,
|
||||
isChannelAllowed: () => true,
|
||||
resolveChannelName: async () => ({ name: "dm", type: "im" }),
|
||||
resolveUserName: async () => ({ name: "Ada" }),
|
||||
} as unknown;
|
||||
const account = {
|
||||
accountId: "acct",
|
||||
config: { commands: { native: true, nativeSkills: false } },
|
||||
} as unknown;
|
||||
|
||||
await registerCommands(ctx, account);
|
||||
expect(commands.size).toBeGreaterThan(0);
|
||||
expect(actions.has("openclaw_cmdarg")).toBe(true);
|
||||
expect(options.has("openclaw_cmdarg")).toBe(true);
|
||||
});
|
||||
|
||||
it("shows a button menu when required args are omitted", async () => {
|
||||
const { respond } = await runCommandHandler(usageHandler);
|
||||
const actions = expectArgMenuLayout(respond);
|
||||
|
||||
@@ -734,19 +734,21 @@ export async function registerSlackMonitorSlashCommands(params: {
|
||||
}
|
||||
|
||||
const registerArgOptions = () => {
|
||||
const appWithOptions = ctx.app as unknown as {
|
||||
options?: (
|
||||
actionId: string,
|
||||
handler: (args: {
|
||||
ack: (payload: { options: unknown[] }) => Promise<void>;
|
||||
body: unknown;
|
||||
}) => Promise<void>,
|
||||
) => void;
|
||||
};
|
||||
if (typeof appWithOptions.options !== "function") {
|
||||
const optionsHandler = (
|
||||
ctx.app as unknown as {
|
||||
options?: (
|
||||
actionId: string,
|
||||
handler: (args: {
|
||||
ack: (payload: { options: unknown[] }) => Promise<void>;
|
||||
body: unknown;
|
||||
}) => Promise<void>,
|
||||
) => void;
|
||||
}
|
||||
).options;
|
||||
if (typeof optionsHandler !== "function") {
|
||||
return;
|
||||
}
|
||||
appWithOptions.options(SLACK_COMMAND_ARG_ACTION_ID, async ({ ack, body }) => {
|
||||
optionsHandler(SLACK_COMMAND_ARG_ACTION_ID, async ({ ack, body }) => {
|
||||
const typedBody = body as {
|
||||
value?: string;
|
||||
user?: { id?: string };
|
||||
|
||||
@@ -2,58 +2,6 @@ import { describe, expect, it, vi } from "vitest";
|
||||
import { createCommandHandlers } from "./tui-command-handlers.js";
|
||||
|
||||
describe("tui command handlers", () => {
|
||||
it("renders the sending indicator before chat.send resolves", async () => {
|
||||
let resolveSend: ((value: { runId: string }) => void) | null = null;
|
||||
const sendChat = vi.fn(
|
||||
() =>
|
||||
new Promise<{ runId: string }>((resolve) => {
|
||||
resolveSend = resolve;
|
||||
}),
|
||||
);
|
||||
const addUser = vi.fn();
|
||||
const requestRender = vi.fn();
|
||||
const setActivityStatus = vi.fn();
|
||||
|
||||
const { handleCommand } = createCommandHandlers({
|
||||
client: { sendChat } as never,
|
||||
chatLog: { addUser, addSystem: vi.fn() } as never,
|
||||
tui: { requestRender } as never,
|
||||
opts: {},
|
||||
state: {
|
||||
currentSessionKey: "agent:main:main",
|
||||
activeChatRunId: null,
|
||||
sessionInfo: {},
|
||||
} as never,
|
||||
deliverDefault: false,
|
||||
openOverlay: vi.fn(),
|
||||
closeOverlay: vi.fn(),
|
||||
refreshSessionInfo: vi.fn(),
|
||||
loadHistory: vi.fn(),
|
||||
setSession: vi.fn(),
|
||||
refreshAgents: vi.fn(),
|
||||
abortActive: vi.fn(),
|
||||
setActivityStatus,
|
||||
formatSessionKey: vi.fn(),
|
||||
applySessionInfoFromPatch: vi.fn(),
|
||||
noteLocalRunId: vi.fn(),
|
||||
});
|
||||
|
||||
const pending = handleCommand("/context");
|
||||
await Promise.resolve();
|
||||
|
||||
expect(setActivityStatus).toHaveBeenCalledWith("sending");
|
||||
const sendingOrder = setActivityStatus.mock.invocationCallOrder[0] ?? 0;
|
||||
const renderOrders = requestRender.mock.invocationCallOrder;
|
||||
expect(renderOrders.some((order) => order > sendingOrder)).toBe(true);
|
||||
|
||||
if (typeof resolveSend !== "function") {
|
||||
throw new Error("expected sendChat to be pending");
|
||||
}
|
||||
(resolveSend as (value: { runId: string }) => void)({ runId: "r1" });
|
||||
await pending;
|
||||
expect(setActivityStatus).toHaveBeenCalledWith("waiting");
|
||||
});
|
||||
|
||||
it("forwards unknown slash commands to the gateway", async () => {
|
||||
const sendChat = vi.fn().mockResolvedValue({ runId: "r1" });
|
||||
const addUser = vi.fn();
|
||||
|
||||
@@ -470,7 +470,6 @@ export function createCommandHandlers(context: CommandHandlerContext) {
|
||||
noteLocalRunId(runId);
|
||||
state.activeChatRunId = runId;
|
||||
setActivityStatus("sending");
|
||||
tui.requestRender();
|
||||
await client.sendChat({
|
||||
sessionKey: state.currentSessionKey,
|
||||
message: text,
|
||||
@@ -480,7 +479,6 @@ export function createCommandHandlers(context: CommandHandlerContext) {
|
||||
runId,
|
||||
});
|
||||
setActivityStatus("waiting");
|
||||
tui.requestRender();
|
||||
} catch (err) {
|
||||
if (state.activeChatRunId) {
|
||||
forgetLocalRunId?.(state.activeChatRunId);
|
||||
@@ -488,8 +486,8 @@ export function createCommandHandlers(context: CommandHandlerContext) {
|
||||
state.activeChatRunId = null;
|
||||
chatLog.addSystem(`send failed: ${String(err)}`);
|
||||
setActivityStatus("error");
|
||||
tui.requestRender();
|
||||
}
|
||||
tui.requestRender();
|
||||
};
|
||||
|
||||
return {
|
||||
|
||||
@@ -130,32 +130,10 @@ describe("shouldEnableWindowsGitBashPasteFallback", () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("enables fallback on macOS iTerm", () => {
|
||||
expect(
|
||||
shouldEnableWindowsGitBashPasteFallback({
|
||||
platform: "darwin",
|
||||
env: {
|
||||
TERM_PROGRAM: "iTerm.app",
|
||||
} as NodeJS.ProcessEnv,
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("enables fallback on macOS Terminal.app", () => {
|
||||
expect(
|
||||
shouldEnableWindowsGitBashPasteFallback({
|
||||
platform: "darwin",
|
||||
env: {
|
||||
TERM_PROGRAM: "Apple_Terminal",
|
||||
} as NodeJS.ProcessEnv,
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("disables fallback outside Windows", () => {
|
||||
expect(
|
||||
shouldEnableWindowsGitBashPasteFallback({
|
||||
platform: "linux",
|
||||
platform: "darwin",
|
||||
env: {
|
||||
MSYSTEM: "MINGW64",
|
||||
} as NodeJS.ProcessEnv,
|
||||
|
||||
@@ -84,24 +84,13 @@ export function shouldEnableWindowsGitBashPasteFallback(params?: {
|
||||
env?: NodeJS.ProcessEnv;
|
||||
}): boolean {
|
||||
const platform = params?.platform ?? process.platform;
|
||||
const env = params?.env ?? process.env;
|
||||
const termProgram = (env.TERM_PROGRAM ?? "").toLowerCase();
|
||||
|
||||
// Some macOS terminals emit multiline paste as rapid single-line submits.
|
||||
// Enable burst coalescing so pasted blocks stay as one user message.
|
||||
if (platform === "darwin") {
|
||||
if (termProgram.includes("iterm") || termProgram.includes("apple_terminal")) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (platform !== "win32") {
|
||||
return false;
|
||||
}
|
||||
|
||||
const env = params?.env ?? process.env;
|
||||
const msystem = (env.MSYSTEM ?? "").toUpperCase();
|
||||
const shell = env.SHELL ?? "";
|
||||
const termProgram = (env.TERM_PROGRAM ?? "").toLowerCase();
|
||||
if (msystem.startsWith("MINGW") || msystem.startsWith("MSYS")) {
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -372,13 +372,6 @@
|
||||
border-color: rgba(255, 255, 255, 0.2);
|
||||
}
|
||||
|
||||
/* Ensure chat toolbar toggles have a clearly visible active state. */
|
||||
.chat-controls .btn--icon.active {
|
||||
border-color: var(--accent);
|
||||
background: var(--accent-subtle);
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
/* Light theme icon button overrides */
|
||||
:root[data-theme="light"] .btn--icon {
|
||||
background: #ffffff;
|
||||
@@ -393,13 +386,6 @@
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
:root[data-theme="light"] .chat-controls .btn--icon.active {
|
||||
border-color: var(--accent);
|
||||
background: var(--accent-subtle);
|
||||
color: var(--accent);
|
||||
box-shadow: 0 0 0 1px var(--accent-subtle);
|
||||
}
|
||||
|
||||
.btn--icon svg {
|
||||
display: block;
|
||||
width: 18px;
|
||||
|
||||
@@ -542,12 +542,6 @@
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
|
||||
:root[data-theme="light"] .btn.active {
|
||||
border-color: var(--accent);
|
||||
background: var(--accent-subtle);
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
:root[data-theme="light"] .btn.primary {
|
||||
background: var(--accent);
|
||||
border-color: var(--accent);
|
||||
|
||||
Reference in New Issue
Block a user