Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
C
Core
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 913
    • Issues 913
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Incidents
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • Development
  • Core
  • Issues
  • #1799

Closed
Open
Opened Jun 05, 2020 by JonGold@JonGoldDeveloper

Daily (and other) scheduled jobs slip forward each day until they're eventually missed

If this subject line looks familiar, there's a reason.

This issue was originally raised and fixed in April 2015. However, a code simplification in December 2015 removed the fix.

Many of us are familiar with this issue. Cron runs at 10:00:00, scheduled jobs are executed in order, and one job might not fire until 10:00:03. If the job fires hourly, we would expect the job to fire at 11:00:00, but since an hour hasn't elapsed since 10:00:00, the job doesn't fire until the next cron, maybe at 11:15:00. This has repercussions for a number of scheduled job use cases.

After some thought, I realized the correct solution is to record the time the cron job fires, not the time the job actually is started. I believe that when the last_run field was created, there was no sense that these times might be different, hence the bug.

I wrote a patch (I'm leaving it in production for a few days before submitting a PR) that does two things:

  • last_run now refers to the time the cron job started, not the scheduled job.
  • To avoid race-y conditions on very slow shared hosts, I force the last_run time to always have "00" as its seconds. This is a belt-and-suspenders (or belt-and-braces in non-'Merican) approach that reflects that a) shared hosting could be slow enough that this would otherwise be a problem, b) cron doesn't allow scheduled at the seconds level, so this shouldn't impact anyone's scheduling.
To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: dev/core#1799