Fixes#15692
The previous fix was too broad — it removed the relay for ALL isolated jobs.
This broke backwards compatibility for jobs without explicit delivery config.
The correct behavior is:
- If job.delivery exists → isolated runner handles it via runSubagentAnnounceFlow
- If only legacy payload.deliver fields → relay to main if requested (original behavior)
This addresses Greptile's review feedback about runIsolatedAgentJob being an
injected dependency that might not call runSubagentAnnounceFlow.
Uses resolveCronDeliveryPlan().source to distinguish between explicit delivery
config and legacy payload-only jobs.
When an isolated cron job delivers its output via deliverOutboundPayloads
or the subagent announce flow, the finish handler in executeJobCore
unconditionally posts a summary to the main agent session and wakes it
via requestHeartbeatNow. The main agent then generates a second response
that is also delivered to the target channel, resulting in duplicate
messages with different content.
Add a `delivered` flag to RunCronAgentTurnResult that is set to true
when the isolated run successfully delivers its output. In executeJobCore,
skip the enqueueSystemEvent + requestHeartbeatNow call when the flag is
set, preventing the main agent from waking up and double-posting.
Fixes#15692
Verified:
- CI checks for commit 86a7ecb45ebf0be61dce9261398000524fd9fab6
- Rebase conflict resolution for compatibility with latest main
Co-authored-by: vpesh <9496634+vpesh@users.noreply.github.com>
* fix: exclude maxTokens and token-count fields from config redaction
The /token/i regex in SENSITIVE_KEY_PATTERNS falsely matched fields like
maxTokens, maxOutputTokens, maxCompletionTokens etc. These are numeric
config fields for token counts, not sensitive credentials.
Added a whitelist (SENSITIVE_KEY_WHITELIST) that explicitly excludes
known token-count field names from redaction. This prevents config
corruption when maxTokens gets replaced with __OPENCLAW_REDACTED__
during config round-trips.
Fixes#13236
* fix: honor deleteAfterRun for one-shot 'at' jobs with 'skipped' status
Previously, deleteAfterRun only triggered when result.status was 'ok'.
For one-shot 'at' jobs, a 'skipped' status (e.g. empty heartbeat file)
would leave the job in state but disabled, never getting cleaned up.
Now deleteAfterRun also triggers on 'skipped' status for 'at' jobs,
since a skipped one-shot job has no meaningful retry path.
Fixes#13249
* Cron: format timer.ts
---------
Co-authored-by: nice03 <niceyslee@gmail.com>
Co-authored-by: Tak Hoffman <781889+Takhoffman@users.noreply.github.com>
* fix(cron): pass agentId to runHeartbeatOnce for main-session jobs
Main-session cron jobs with agentId always ran the heartbeat under
the default agent, ignoring the job's agent binding. enqueueSystemEvent
correctly routed the system event to the bound agent's session, but
runHeartbeatOnce was called without agentId, so the heartbeat ran under
the default agent and never picked up the event.
Thread agentId from job.agentId through the CronServiceDeps type,
timer execution, and the gateway wrapper so heartbeat-runner uses the
correct agent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* cron: add heartbeat agentId propagation regression test (#14140) (thanks @ishikawa-pro)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Tak Hoffman <781889+Takhoffman@users.noreply.github.com>
Previously, if one cron job had a malformed schedule expression (e.g. invalid cron syntax),
the error would propagate up and break the entire scheduler loop. This meant one misconfigured
job could prevent ALL cron jobs from running.
Changes:
- Wrap per-job schedule computation in try/catch in recomputeNextRuns()
- Track consecutive schedule errors via new scheduleErrorCount field
- Log warnings for schedule errors with job ID and name
- Auto-disable jobs after 3 consecutive schedule errors (with error-level log)
- Clear error count when schedule computation succeeds
- Continue processing other jobs even when one fails
This ensures the scheduler is resilient to individual job misconfigurations while still
providing visibility into problems through logging.
Co-authored-by: Marvin <numegilagent@gmail.com>
* fix(cron): re-arm timer when onTimer fires during active job execution
When a cron job takes longer than MAX_TIMER_DELAY_MS (60s), the clamped
timer fires while state.running is still true. The early return in
onTimer() previously exited without re-arming the timer, leaving no
setTimeout scheduled. This silently kills the cron scheduler until the
next gateway restart.
The fix calls armTimer(state) before the early return so the scheduler
continues ticking even when a job is in progress.
This is the likely root cause of recurring cron jobs silently skipping,
as reported in #12025. One-shot (kind: 'at') jobs were unaffected
because they typically complete within a single timer cycle.
Includes a regression test that simulates a slow job exceeding the
timer clamp period and verifies the next occurrence still fires.
* fix: update tests for timer re-arm behavior
- Update existing regression test to expect timer re-arm with non-zero
delay instead of no timer at all
- Simplify new test to directly verify state.timer is set after onTimer
returns early due to running guard
* fix: use fixed 60s delay for re-arm to prevent zero-delay hot-loop
When the running guard re-arms the timer, use MAX_TIMER_DELAY_MS
directly instead of calling armTimer() which can compute a zero delay
for past-due jobs. This prevents a tight spin while still keeping the
scheduler alive.
* style: add curly braces to satisfy eslint(curly) rule
The `computeNextRunAtMs` function used `nowSecondMs - 1` as the
reference time for croner's `nextRun()`, which caused it to return the
current second as a valid next-run time. When a job fired at e.g.
11:00:00.500, computing the next run still yielded 11:00:00.000 (same
second, already elapsed), causing the scheduler to immediately re-fire
the job in a tight loop (15-21x observed in the wild).
Fix: use `nowSecondMs` directly (no `-1` lookback) and change the
return guard from `>=` to `>` so next-run is always strictly after
the current second.
Fixes#14164
* Heartbeat: inject cron-style current time into prompts
* Tests: fix type for web heartbeat timestamp test
* Infra: inline heartbeat current-time injection
* fix: prune stale session entries, cap entry count, and rotate sessions.json
The sessions.json file grows unbounded over time. Every heartbeat tick (default: 30m)
triggers multiple full rewrites, and session keys from groups, threads, and DMs
accumulate indefinitely with large embedded objects (skillsSnapshot,
systemPromptReport). At >50MB the synchronous JSON parse blocks the event loop,
causing Telegram webhook timeouts and effectively taking the bot down.
Three mitigations, all running inside saveSessionStoreUnlocked() on every write:
1. Prune stale entries: remove entries with updatedAt older than 30 days
(configurable via session.maintenance.pruneDays in openclaw.json)
2. Cap entry count: keep only the 500 most recently updated entries
(configurable via session.maintenance.maxEntries). Entries without updatedAt
are evicted first.
3. File rotation: if the existing sessions.json exceeds 10MB before a write,
rename it to sessions.json.bak.{timestamp} and keep only the 3 most recent
backups (configurable via session.maintenance.rotateBytes).
All three thresholds are configurable under session.maintenance in openclaw.json
with Zod validation. No env vars.
Existing tests updated to use Date.now() instead of epoch-relative timestamps
(1, 2, 3) that would be incorrectly pruned as stale.
27 new tests covering pruning, capping, rotation, and integration scenarios.
* feat: auto-prune expired cron run sessions (#12289)
Add TTL-based reaper for isolated cron run sessions that accumulate
indefinitely in sessions.json.
New config option:
cron.sessionRetention: string | false (default: '24h')
The reaper runs piggy-backed on the cron timer tick, self-throttled
to sweep at most every 5 minutes. It removes session entries matching
the pattern cron:<jobId>:run:<uuid> whose updatedAt + retention < now.
Design follows the Kubernetes ttlSecondsAfterFinished pattern:
- Sessions are persisted normally (observability/debugging)
- A periodic reaper prunes expired entries
- Configurable retention with sensible default
- Set to false to disable pruning entirely
Files changed:
- src/config/types.cron.ts: Add sessionRetention to CronConfig
- src/config/zod-schema.ts: Add Zod validation for sessionRetention
- src/cron/session-reaper.ts: New reaper module (sweepCronRunSessions)
- src/cron/session-reaper.test.ts: 12 tests covering all paths
- src/cron/service/state.ts: Add cronConfig/sessionStorePath to deps
- src/cron/service/timer.ts: Wire reaper into onTimer tick
- src/gateway/server-cron.ts: Pass config and session store path to deps
Closes#12289
* fix: sweep cron session stores per agent
* docs: add changelog for session maintenance (#13083) (thanks @skyfallsin, @Glucksberg)
* fix: add warn-only session maintenance mode
* fix: warn-only maintenance defaults to active session
* fix: deliver maintenance warnings to active session
* docs: add session maintenance examples
* fix: accept duration and size maintenance thresholds
* refactor: share cron run session key check
* fix: format issues and replace defaultRuntime.warn with console.warn
---------
Co-authored-by: Pradeep Elankumaran <pradeepe@gmail.com>
Co-authored-by: Glucksberg <markuscontasul@gmail.com>
Co-authored-by: max <40643627+quotentiroler@users.noreply.github.com>
Co-authored-by: quotentiroler <max.nussbaumer@maxhealth.tech>
* fix(cron): recover flat params when LLM omits job wrapper (#11310)
Non-frontier models (e.g. Grok) flatten job properties to the top level
alongside `action` instead of nesting them inside the `job` parameter.
The opaque schema (`Type.Object({}, { additionalProperties: true })`)
gives these models no structural hint, so they put name, schedule,
payload, etc. as siblings of action.
Add a flat-params recovery step in the cron add handler: when
`params.job` is missing or an empty object, scan for recognised job
property names on params and construct a synthetic job object before
passing to `normalizeCronJobCreate`. Recovery requires at least one
meaningful signal field (schedule, payload, message, or text) to avoid
false positives.
Added tests:
- Flat params with no job wrapper → recovered
- Empty job object + flat params → recovered
- Message shorthand at top level → inferred as agentTurn
- No meaningful fields → still throws 'job required'
- Non-empty job takes precedence over flat params
* fix(cron): floor nowMs to second boundary before croner lookback
Cron expressions operate at second granularity. When nowMs falls
mid-second (e.g. 12:00:00.500) and the pattern targets that exact
second (like '0 0 12 * * *'), a 1ms lookback still lands inside the
matching second. Croner interprets this as 'already past' and skips
to the next occurrence (e.g. the following day).
Fix: floor nowMs to the start of the current second before applying
the 1ms lookback. This ensures the reference always falls in the
*previous* second, so croner correctly identifies the current match.
Also compare the result against the floored nowSecondMs (not raw nowMs)
so that a match at the start of the current second is not rejected by
the >= guard when nowMs has sub-second offset.
Adds regression tests for 6-field cron patterns with specific seconds.
* fix: add changelog entries for cron fixes (#12124) (thanks @tyler6204)
* test: stabilize warning filter emit assertion (#12124) (thanks @tyler6204)
* initial commit
* feat: implement deriveSessionTotalTokens function and update usage tests
* Added deriveSessionTotalTokens function to calculate total tokens based on usage and context tokens.
* Updated usage tests to include cases for derived session total tokens.
* Refactored session usage calculations in multiple files to utilize the new function for improved accuracy.
* fix: restore overflow truncation fallback + changelog/test hardening (#11551) (thanks @tyler6204)
* fix(cron): comprehensive cron scheduling and delivery fixes
- Fix delivery target resolution for isolated agent cron jobs
- Improve schedule parsing and validation
- Add job retry logic and error handling
- Enhance cron ops with better state management
- Add timer improvements for more reliable cron execution
- Add cron event type to protocol schema
- Support cron events in heartbeat runner (skip empty-heartbeat check,
use dedicated CRON_EVENT_PROMPT for relay)
* fix: remove cron debug test and add changelog/docs notes (#11641) (thanks @tyler6204)
* refactor: update cron job wake mode and run mode handling
- Changed default wake mode from 'next-heartbeat' to 'now' in CronJobEditor and related CLI commands.
- Updated cron-tool tests to reflect changes in run mode, introducing 'due' and 'force' options.
- Enhanced cron-tool logic to handle new run modes and ensure compatibility with existing job structures.
- Added new tests for delivery plan consistency and job execution behavior under various conditions.
- Improved normalization functions to handle wake mode and session target casing.
This refactor aims to streamline cron job configurations and enhance the overall user experience with clearer defaults and improved functionality.
* test: enhance cron job functionality and UI
- Added tests to ensure the isolated agent correctly announces the final payload text when delivering messages via Telegram.
- Implemented a new function to pick the last deliverable payload from a list of delivery payloads.
- Enhanced the cron service to maintain legacy "every" jobs while minute cron jobs recompute schedules.
- Updated the cron store migration tests to verify the addition of anchorMs to legacy every schedules.
- Improved the UI for displaying cron job details, including job state and delivery information, with new styles and layout adjustments.
These changes aim to improve the reliability and user experience of the cron job system.
* test: enhance sessions thinking level handling
- Added tests to verify that the correct thinking levels are applied during session spawning.
- Updated the sessions-spawn-tool to include a new parameter for overriding thinking levels.
- Enhanced the UI to support additional thinking levels, including "xhigh" and "full", and improved the handling of current options in dropdowns.
These changes aim to improve the flexibility and accuracy of thinking level configurations in session management.
* feat: enhance session management and cron job functionality
- Introduced passthrough arguments in the test-parallel script to allow for flexible command-line options.
- Updated session handling to hide cron run alias session keys from the sessions list, improving clarity.
- Enhanced the cron service to accurately record job start times and durations, ensuring better tracking of job execution.
- Added tests to verify the correct behavior of the cron service under various conditions, including zero-delay timers.
These changes aim to improve the usability and reliability of session and cron job management.
* feat: implement job running state checks in cron service
- Added functionality to prevent manual job runs if a job is already in progress, enhancing job management.
- Updated the `isJobDue` function to include checks for running jobs, ensuring accurate scheduling.
- Enhanced the `run` function to return a specific reason when a job is already running.
- Introduced a new test case to verify the behavior of forced manual runs during active job execution.
These changes aim to improve the reliability and clarity of cron job execution and management.
* feat: add session ID and key to CronRunLogEntry model
- Introduced `sessionid` and `sessionkey` properties to the `CronRunLogEntry` struct for enhanced tracking of session-related information.
- Updated the initializer and Codable conformance to accommodate the new properties, ensuring proper serialization and deserialization.
These changes aim to improve the granularity of logging and session management within the cron job system.
* fix: improve session display name resolution
- Updated the `resolveSessionDisplayName` function to ensure that both label and displayName are trimmed and default to an empty string if not present.
- Enhanced the logic to prevent returning the key if it matches the label or displayName, improving clarity in session naming.
These changes aim to enhance the accuracy and usability of session display names in the UI.
* perf: skip cron store persist when idle timer tick produces no changes
recomputeNextRuns now returns a boolean indicating whether any job
state was mutated. The idle path in onTimer only persists when the
return value is true, eliminating unnecessary file writes every 60s
for far-future or idle schedules.
* fix: prep for merge - explicit delivery mode migration, docs + changelog (#10776) (thanks @tyler6204)
* fix(cron): handle legacy atMs field in schedule when computing next run
The cron scheduler only checked for `schedule.at` (string) but legacy jobs
may have `schedule.atMs` (number) from before the schema migration.
This caused nextRunAtMs to stay null because:
1. Store migration runs on load but may not persist immediately
2. Race conditions or file mtime issues can skip migration
3. computeJobNextRunAtMs/computeNextRunAtMs only checked `at`, not `atMs`
Fix: Make both functions defensive by checking `atMs` first (number),
then `atMs` (string, for edge cases), then falling back to `at` (string).
This ensures jobs fire correctly even if:
- Migration hasn't run yet
- Old data was written by a previous version
- The store was manually edited
Fixes#9930
* fix: validate numeric atMs to prevent NaN/Infinity propagation
Addresses review feedback - numeric atMs values are now validated with
Number.isFinite() && atMs > 0 before use. This prevents corrupted or
manually edited stores from causing hot timer loops via setTimeout(..., NaN).
