Skip to content

Log a task failure in the daemon loop instead of stopping the daemon - #6

Merged
KalimeroMK merged 7 commits into
masterfrom
fix-daemon-stopping-on-task-failure
Oct 12, 2026
Merged

KalimeroMK merged 7 commits into
masterfrom
fix-daemon-stopping-on-task-failure

Conversation

@samdark

@samdark samdark commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Problem

One failing task stops the whole schedule:work daemon and sends SIGTERM to every other task that is running.

An unignored failure was thrown out of Scheduler::run(), from tick() in sequential mode and from reapChildren() in forked mode, and the finally of run() then terminated the remaining children. The SIGCHLD handler of WorkCommand did the same through stop(). No listener ignores a failure by default, so any task exiting with an error brought the schedule down. This contradicted the reapChildren() docblock ("neither kills its siblings").

How to reproduce

Save as repro.php in the repository root and run php repro.php. It uses the test doubles from tests/Support.

repro.php
<?php

require 'vendor/autoload.php';

use Yiisoft\Scheduler\Handler\CallableTaskHandler;
use Yiisoft\Scheduler\RecurringTask;
use Yiisoft\Scheduler\Schedule;
use Yiisoft\Scheduler\Scheduler;
use Yiisoft\Scheduler\Tests\Support\TestClock;

$clock = new TestClock();
$scheduler = null;
$healthy = 0;

$schedule = (new Schedule())->task(
    RecurringTask::cron('* * * * *', static function (): void {
        throw new RuntimeException('Task failed.');
    }),
    RecurringTask::cron('* * * * *', static function () use (&$healthy, &$scheduler): void {
        ++$healthy;
        $scheduler->stop(); // ends the loop, so the script returns
    }),
);

$scheduler = new Scheduler([$schedule], new CallableTaskHandler(), $clock);
$scheduler->tick();           // the first tick only records the position
$clock->advance('+1 minute'); // now both tasks are due

try {
    $scheduler->run(0.01, false); // what schedule:work --sequential calls
    echo "the daemon loop returned normally\n";
} catch (RuntimeException $error) {
    echo 'the daemon loop was ended by: ', $error->getMessage(), "\n";
}

echo "healthy task ran: $healthy times\n";

On master:

the daemon loop was ended by: Task failed.
healthy task ran: 0 times

With this change:

the daemon loop returned normally
healthy task ran: 1 times

The script uses the sequential loop so that it needs no ext-pcntl; the forked loop ends the same way on master, with "The task process exited with code 1."

Fix

While run() is driving the ticks, a task failure that no listener ignored is logged as an error instead of thrown. The failure event and the schedule's failure listeners are still notified as usual.

Outside of run() nothing changes: tick() and reapChildren() still throw, so schedule:run still exits with a failure.

Why this way

  • Why not catch the error in WorkCommand and start the loop again. Leaving run() already terminates every running child, so the damage is done before the command sees the error; and it would fix the command only, not run() for anyone calling it directly.
  • Why not make a failure ignored by default. tick() would stop throwing as well, and schedule:run would exit with 0 after a failed task. A cron entry relies on that exit code.
  • Why not an argument of run() to choose between the two. A daemon that dies on the first failing task is not something to opt out of; there is no use for the other value.
  • Why log. A failure that is neither thrown nor ignored would otherwise be visible to FailureEvent listeners only. An ignored failure was already logged as an error, so this is the same record.

Things to know

  • The loop stops handling failures this way only once its children are stopped, so a child failing during the shutdown is logged too rather than thrown at whoever stopped the daemon.
  • The note for listeners is on FailureEvent::ignore(), not on the class docblock, which Keep a failing task from blocking the rest of the schedule #4 rewrites.
  • With no PSR-3 logger and no FailureEvent listener, a failure in the daemon is now silent rather than fatal. The README says so.
  • Only task failures are tolerated. An error of the schedule itself (checkpoint, trigger) or one thrown by a listener still ends the loop.
  • In forked mode the parent still only knows "exited with code 1"; the real error is logged by the child.

Tests

  • A sequential loop where the task behind a failing one stops the loop, and the error reaches the logger.
  • A forked loop where the failure listener stops the loop without ignoring the failure.
  • tick() still throws after run() has returned.

The first two fail on master.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 11, 2026 21:36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new documentation incorrectly claims ignored failures are not logged, contradicting the implementation.

2 open findings
What changed in this PR

Keeps the scheduler daemon running after task failures while preserving throwing behavior outside run().

Changes:

  • Tracks daemon-loop execution and logs task failures instead of rethrowing them.
  • Documents daemon failure handling.
  • Adds sequential and forked-mode regression tests.
File Description
src/​Scheduler.php Implements daemon failure tolerance.
src/​Event/​FailureEvent.php Documents daemon-specific behavior.
tests/​SchedulerTest.php Adds failure-handling tests.
README.md Documents operational behavior.
CHANGELOG.md Records the fix.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread README.md Resolved
Comment thread src/Scheduler.php Resolved
samdark and others added 2 commits October 12, 2026 01:17
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A child failing between the end of the loop and the shutdown of the others
was thrown at whoever stopped the daemon. The flag is now dropped by
terminateChildren(), which also leaves the end of run() as it is on master.

The note on the daemon moves from the FailureEvent class docblock to
ignore(), and the new tests further down, so the pull request no longer
conflicts with #4 and #9. The comments of the work command said its signal
handler catches a task failure, which it no longer does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/Scheduler.php
$this->stopChildren();
} finally {
$this->terminating = false;
$this->looping = false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this reset has no test: removing it keeps the suite green, and a later tick() would then swallow failures. A forked-mode test that stops run() while a child is still running and then expects a later tick() to throw would pin it.

@KalimeroMK
KalimeroMK merged commit dde4091 into master Oct 12, 2026
25 checks passed
@KalimeroMK
KalimeroMK deleted the fix-daemon-stopping-on-task-failure branch October 12, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants