* feat(secrets): expand secret target coverage and gateway tooling * docs(secrets): align gateway and CLI secret docs * chore(protocol): regenerate swift gateway models for secrets methods * fix(config): restore talk apiKey fallback and stabilize runner test * ci(windows): reduce test worker count for shard stability * ci(windows): raise node heap for test shard stability * test(feishu): make proxy env precedence assertion windows-safe * fix(gateway): resolve auth password SecretInput refs for clients * fix(gateway): resolve remote SecretInput credentials for clients * fix(secrets): skip inactive refs in command snapshot assignments * fix(secrets): scope gateway.remote refs to effective auth surfaces * fix(secrets): ignore memory defaults when enabled agents disable search * fix(secrets): honor Google Chat serviceAccountRef inheritance * fix(secrets): address tsgo errors in command and gateway collectors * fix(secrets): avoid auth-store load in providers-only configure * fix(gateway): defer local password ref resolution by precedence * fix(secrets): gate telegram webhook secret refs by webhook mode * fix(secrets): gate slack signing secret refs to http mode * fix(secrets): skip telegram botToken refs when tokenFile is set * fix(secrets): gate discord pluralkit refs by enabled flag * fix(secrets): gate discord voice tts refs by voice enabled * test(secrets): make runtime fixture modes explicit * fix(cli): resolve local qr password secret refs * fix(cli): fail when gateway leaves command refs unresolved * fix(gateway): fail when local password SecretRef is unresolved * fix(gateway): fail when required remote SecretRefs are unresolved * fix(gateway): resolve local password refs only when password can win * fix(cli): skip local password SecretRef resolution on qr token override * test(gateway): cast SecretRef fixtures to OpenClawConfig * test(secrets): activate mode-gated targets in runtime coverage fixture * fix(cron): support SecretInput webhook tokens safely * fix(bluebubbles): support SecretInput passwords across config paths * fix(msteams): make appPassword SecretInput-safe in onboarding/token paths * fix(bluebubbles): align SecretInput schema helper typing * fix(cli): clarify secrets.resolve version-skew errors * refactor(secrets): return structured inactive paths from secrets.resolve * refactor(gateway): type onboarding secret writes as SecretInput * chore(protocol): regenerate swift models for secrets.resolve * feat(secrets): expand extension credential secretref support * fix(secrets): gate web-search refs by active provider * fix(onboarding): detect SecretRef credentials in extension status * fix(onboarding): allow keeping existing ref in secret prompt * fix(onboarding): resolve gateway password SecretRefs for probe and tui * fix(onboarding): honor secret-input-mode for local gateway auth * fix(acp): resolve gateway SecretInput credentials * fix(secrets): gate gateway.remote refs to remote surfaces * test(secrets): cover pattern matching and inactive array refs * docs(secrets): clarify secrets.resolve and remote active surfaces * fix(bluebubbles): keep existing SecretRef during onboarding * fix(tests): resolve CI type errors in new SecretRef coverage * fix(extensions): replace raw fetch with SSRF-guarded fetch * test(secrets): mark gateway remote targets active in runtime coverage * test(infra): normalize home-prefix expectation across platforms * fix(cli): only resolve local qr password refs in password mode * test(cli): cover local qr token mode with unresolved password ref * docs(cli): clarify local qr password ref resolution behavior * refactor(extensions): reuse sdk SecretInput helpers * fix(wizard): resolve onboarding env-template secrets before plaintext * fix(cli): surface secrets.resolve diagnostics in memory and qr * test(secrets): repair post-rebase runtime and fixtures * fix(gateway): skip remote password ref resolution when token wins * fix(secrets): treat tailscale remote gateway refs as active * fix(gateway): allow remote password fallback when token ref is unresolved * fix(gateway): ignore stale local password refs for none and trusted-proxy * fix(gateway): skip remote secret ref resolution on local call paths * test(cli): cover qr remote tailscale secret ref resolution * fix(secrets): align gateway password active-surface with auth inference * fix(cli): resolve inferred local gateway password refs in qr * fix(gateway): prefer resolvable remote password over token ref pre-resolution * test(gateway): cover none and trusted-proxy stale password refs * docs(secrets): sync qr and gateway active-surface behavior * fix: restore stability blockers from pre-release audit * Secrets: fix collector/runtime precedence contradictions * docs: align secrets and web credential docs * fix(rebase): resolve integration regressions after main rebase * fix(node-host): resolve gateway secret refs for auth * fix(secrets): harden secretinput runtime readers * gateway: skip inactive auth secretref resolution * cli: avoid gateway preflight for inactive secret refs * extensions: allow unresolved refs in onboarding status * tests: fix qr-cli module mock hoist ordering * Security: align audit checks with SecretInput resolution * Gateway: resolve local-mode remote fallback secret refs * Node host: avoid resolving inactive password secret refs * Secrets runtime: mark Slack appToken inactive for HTTP mode * secrets: keep inactive gateway remote refs non-blocking * cli: include agent memory secret targets in runtime resolution * docs(secrets): sync docs with active-surface and web search behavior * fix(secrets): keep telegram top-level token refs active for blank account tokens * fix(daemon): resolve gateway password secret refs for probe auth * fix(secrets): skip IRC NickServ ref resolution when NickServ is disabled * fix(secrets): align token inheritance and exec timeout defaults * docs(secrets): clarify active-surface notes in cli docs * cli: require secrets.resolve gateway capability * gateway: log auth secret surface diagnostics * secrets: remove dead provider resolver module * fix(secrets): restore gateway auth precedence and fallback resolution * fix(tests): align plugin runtime mock typings --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>
318 lines
9.7 KiB
TypeScript
318 lines
9.7 KiB
TypeScript
import type { OpenClawConfig } from "../config/config.js";
|
|
import { resolveSecretInputRef } from "../config/types.secrets.js";
|
|
import { callGateway } from "../gateway/call.js";
|
|
import { validateSecretsResolveResult } from "../gateway/protocol/index.js";
|
|
import { collectCommandSecretAssignmentsFromSnapshot } from "../secrets/command-config.js";
|
|
import { setPathExistingStrict } from "../secrets/path-utils.js";
|
|
import { resolveSecretRefValues } from "../secrets/resolve.js";
|
|
import { collectConfigAssignments } from "../secrets/runtime-config-collectors.js";
|
|
import { applyResolvedAssignments, createResolverContext } from "../secrets/runtime-shared.js";
|
|
import { describeUnknownError } from "../secrets/shared.js";
|
|
import { discoverConfigSecretTargetsByIds } from "../secrets/target-registry.js";
|
|
import { GATEWAY_CLIENT_MODES, GATEWAY_CLIENT_NAMES } from "../utils/message-channel.js";
|
|
|
|
type ResolveCommandSecretsResult = {
|
|
resolvedConfig: OpenClawConfig;
|
|
diagnostics: string[];
|
|
};
|
|
|
|
type GatewaySecretsResolveResult = {
|
|
ok?: boolean;
|
|
assignments?: Array<{
|
|
path?: string;
|
|
pathSegments: string[];
|
|
value: unknown;
|
|
}>;
|
|
diagnostics?: string[];
|
|
inactiveRefPaths?: string[];
|
|
};
|
|
|
|
function dedupeDiagnostics(entries: readonly string[]): string[] {
|
|
const seen = new Set<string>();
|
|
const ordered: string[] = [];
|
|
for (const entry of entries) {
|
|
const trimmed = entry.trim();
|
|
if (!trimmed || seen.has(trimmed)) {
|
|
continue;
|
|
}
|
|
seen.add(trimmed);
|
|
ordered.push(trimmed);
|
|
}
|
|
return ordered;
|
|
}
|
|
|
|
function collectConfiguredTargetRefPaths(params: {
|
|
config: OpenClawConfig;
|
|
targetIds: Set<string>;
|
|
}): Set<string> {
|
|
const defaults = params.config.secrets?.defaults;
|
|
const configuredTargetRefPaths = new Set<string>();
|
|
for (const target of discoverConfigSecretTargetsByIds(params.config, params.targetIds)) {
|
|
const { ref } = resolveSecretInputRef({
|
|
value: target.value,
|
|
refValue: target.refValue,
|
|
defaults,
|
|
});
|
|
if (ref) {
|
|
configuredTargetRefPaths.add(target.path);
|
|
}
|
|
}
|
|
return configuredTargetRefPaths;
|
|
}
|
|
|
|
function classifyConfiguredTargetRefs(params: {
|
|
config: OpenClawConfig;
|
|
configuredTargetRefPaths: Set<string>;
|
|
}): {
|
|
hasActiveConfiguredRef: boolean;
|
|
hasUnknownConfiguredRef: boolean;
|
|
diagnostics: string[];
|
|
} {
|
|
if (params.configuredTargetRefPaths.size === 0) {
|
|
return {
|
|
hasActiveConfiguredRef: false,
|
|
hasUnknownConfiguredRef: false,
|
|
diagnostics: [],
|
|
};
|
|
}
|
|
const context = createResolverContext({
|
|
sourceConfig: params.config,
|
|
env: process.env,
|
|
});
|
|
collectConfigAssignments({
|
|
config: structuredClone(params.config),
|
|
context,
|
|
});
|
|
|
|
const activePaths = new Set(context.assignments.map((assignment) => assignment.path));
|
|
const inactiveWarningsByPath = new Map<string, string>();
|
|
for (const warning of context.warnings) {
|
|
if (warning.code !== "SECRETS_REF_IGNORED_INACTIVE_SURFACE") {
|
|
continue;
|
|
}
|
|
inactiveWarningsByPath.set(warning.path, warning.message);
|
|
}
|
|
|
|
const diagnostics = new Set<string>();
|
|
let hasActiveConfiguredRef = false;
|
|
let hasUnknownConfiguredRef = false;
|
|
|
|
for (const path of params.configuredTargetRefPaths) {
|
|
if (activePaths.has(path)) {
|
|
hasActiveConfiguredRef = true;
|
|
continue;
|
|
}
|
|
const inactiveWarning = inactiveWarningsByPath.get(path);
|
|
if (inactiveWarning) {
|
|
diagnostics.add(inactiveWarning);
|
|
continue;
|
|
}
|
|
hasUnknownConfiguredRef = true;
|
|
}
|
|
|
|
return {
|
|
hasActiveConfiguredRef,
|
|
hasUnknownConfiguredRef,
|
|
diagnostics: [...diagnostics],
|
|
};
|
|
}
|
|
|
|
function parseGatewaySecretsResolveResult(payload: unknown): {
|
|
assignments: Array<{ path?: string; pathSegments: string[]; value: unknown }>;
|
|
diagnostics: string[];
|
|
inactiveRefPaths: string[];
|
|
} {
|
|
if (!validateSecretsResolveResult(payload)) {
|
|
throw new Error("gateway returned invalid secrets.resolve payload.");
|
|
}
|
|
const parsed = payload as GatewaySecretsResolveResult;
|
|
return {
|
|
assignments: parsed.assignments ?? [],
|
|
diagnostics: (parsed.diagnostics ?? []).filter((entry) => entry.trim().length > 0),
|
|
inactiveRefPaths: (parsed.inactiveRefPaths ?? []).filter((entry) => entry.trim().length > 0),
|
|
};
|
|
}
|
|
|
|
function collectInactiveSurfacePathsFromDiagnostics(diagnostics: string[]): Set<string> {
|
|
const paths = new Set<string>();
|
|
for (const entry of diagnostics) {
|
|
const marker = ": secret ref is configured on an inactive surface;";
|
|
const markerIndex = entry.indexOf(marker);
|
|
if (markerIndex <= 0) {
|
|
continue;
|
|
}
|
|
const path = entry.slice(0, markerIndex).trim();
|
|
if (path.length > 0) {
|
|
paths.add(path);
|
|
}
|
|
}
|
|
return paths;
|
|
}
|
|
|
|
function isUnsupportedSecretsResolveError(err: unknown): boolean {
|
|
const message = describeUnknownError(err).toLowerCase();
|
|
if (!message.includes("secrets.resolve")) {
|
|
return false;
|
|
}
|
|
return (
|
|
message.includes("does not support required method") ||
|
|
message.includes("unknown method") ||
|
|
message.includes("method not found") ||
|
|
message.includes("invalid request")
|
|
);
|
|
}
|
|
|
|
async function resolveCommandSecretRefsLocally(params: {
|
|
config: OpenClawConfig;
|
|
commandName: string;
|
|
targetIds: Set<string>;
|
|
preflightDiagnostics: string[];
|
|
}): Promise<ResolveCommandSecretsResult> {
|
|
const sourceConfig = params.config;
|
|
const resolvedConfig = structuredClone(params.config);
|
|
const context = createResolverContext({
|
|
sourceConfig,
|
|
env: process.env,
|
|
});
|
|
collectConfigAssignments({
|
|
config: resolvedConfig,
|
|
context,
|
|
});
|
|
if (context.assignments.length > 0) {
|
|
const resolved = await resolveSecretRefValues(
|
|
context.assignments.map((assignment) => assignment.ref),
|
|
{
|
|
config: sourceConfig,
|
|
env: context.env,
|
|
cache: context.cache,
|
|
},
|
|
);
|
|
applyResolvedAssignments({
|
|
assignments: context.assignments,
|
|
resolved,
|
|
});
|
|
}
|
|
|
|
const inactiveRefPaths = new Set(
|
|
context.warnings
|
|
.filter((warning) => warning.code === "SECRETS_REF_IGNORED_INACTIVE_SURFACE")
|
|
.map((warning) => warning.path),
|
|
);
|
|
const commandAssignments = collectCommandSecretAssignmentsFromSnapshot({
|
|
sourceConfig,
|
|
resolvedConfig,
|
|
commandName: params.commandName,
|
|
targetIds: params.targetIds,
|
|
inactiveRefPaths,
|
|
});
|
|
|
|
return {
|
|
resolvedConfig,
|
|
diagnostics: dedupeDiagnostics([
|
|
...params.preflightDiagnostics,
|
|
...commandAssignments.diagnostics,
|
|
]),
|
|
};
|
|
}
|
|
|
|
export async function resolveCommandSecretRefsViaGateway(params: {
|
|
config: OpenClawConfig;
|
|
commandName: string;
|
|
targetIds: Set<string>;
|
|
}): Promise<ResolveCommandSecretsResult> {
|
|
const configuredTargetRefPaths = collectConfiguredTargetRefPaths({
|
|
config: params.config,
|
|
targetIds: params.targetIds,
|
|
});
|
|
if (configuredTargetRefPaths.size === 0) {
|
|
return { resolvedConfig: params.config, diagnostics: [] };
|
|
}
|
|
const preflight = classifyConfiguredTargetRefs({
|
|
config: params.config,
|
|
configuredTargetRefPaths,
|
|
});
|
|
if (!preflight.hasActiveConfiguredRef && !preflight.hasUnknownConfiguredRef) {
|
|
return {
|
|
resolvedConfig: params.config,
|
|
diagnostics: preflight.diagnostics,
|
|
};
|
|
}
|
|
|
|
let payload: GatewaySecretsResolveResult;
|
|
try {
|
|
payload = await callGateway<GatewaySecretsResolveResult>({
|
|
method: "secrets.resolve",
|
|
requiredMethods: ["secrets.resolve"],
|
|
params: {
|
|
commandName: params.commandName,
|
|
targetIds: [...params.targetIds],
|
|
},
|
|
timeoutMs: 30_000,
|
|
clientName: GATEWAY_CLIENT_NAMES.CLI,
|
|
mode: GATEWAY_CLIENT_MODES.CLI,
|
|
});
|
|
} catch (err) {
|
|
try {
|
|
const fallback = await resolveCommandSecretRefsLocally({
|
|
config: params.config,
|
|
commandName: params.commandName,
|
|
targetIds: params.targetIds,
|
|
preflightDiagnostics: preflight.diagnostics,
|
|
});
|
|
return {
|
|
resolvedConfig: fallback.resolvedConfig,
|
|
diagnostics: dedupeDiagnostics([
|
|
...fallback.diagnostics,
|
|
`${params.commandName}: gateway secrets.resolve unavailable (${describeUnknownError(err)}); resolved command secrets locally.`,
|
|
]),
|
|
};
|
|
} catch {
|
|
// Fall through to original gateway-specific error reporting.
|
|
}
|
|
if (isUnsupportedSecretsResolveError(err)) {
|
|
throw new Error(
|
|
`${params.commandName}: active gateway does not support secrets.resolve (${describeUnknownError(err)}). Update the gateway or run without SecretRefs.`,
|
|
{ cause: err },
|
|
);
|
|
}
|
|
throw new Error(
|
|
`${params.commandName}: failed to resolve secrets from the active gateway snapshot (${describeUnknownError(err)}). Start the gateway and retry.`,
|
|
{ cause: err },
|
|
);
|
|
}
|
|
|
|
const parsed = parseGatewaySecretsResolveResult(payload);
|
|
const resolvedConfig = structuredClone(params.config);
|
|
for (const assignment of parsed.assignments) {
|
|
const pathSegments = assignment.pathSegments.filter((segment) => segment.length > 0);
|
|
if (pathSegments.length === 0) {
|
|
continue;
|
|
}
|
|
try {
|
|
setPathExistingStrict(resolvedConfig, pathSegments, assignment.value);
|
|
} catch (err) {
|
|
const path = pathSegments.join(".");
|
|
throw new Error(
|
|
`${params.commandName}: failed to apply resolved secret assignment at ${path} (${describeUnknownError(err)}).`,
|
|
{ cause: err },
|
|
);
|
|
}
|
|
}
|
|
const inactiveRefPaths =
|
|
parsed.inactiveRefPaths.length > 0
|
|
? new Set(parsed.inactiveRefPaths)
|
|
: collectInactiveSurfacePathsFromDiagnostics(parsed.diagnostics);
|
|
collectCommandSecretAssignmentsFromSnapshot({
|
|
sourceConfig: params.config,
|
|
resolvedConfig,
|
|
commandName: params.commandName,
|
|
targetIds: params.targetIds,
|
|
inactiveRefPaths,
|
|
});
|
|
|
|
return {
|
|
resolvedConfig,
|
|
diagnostics: dedupeDiagnostics(parsed.diagnostics),
|
|
};
|
|
}
|