Scheduling/workflow for security updates of dependencies
Synopsis
The workflow for first-party/in-house security updates and the workflow for third-party/upstream security updates are qualitatively different
The question of this issue is: Do we keep one general policy/schedule for both kinds of security issues, or do we have a more nuanced policy that distinguishes between them?
Background
CiviCRM's policy for scheduling/workflow on security updates has a few key elements:
- Report and discuss vulnerabilities privately
- Release updates on a designated release window (the first/third Wed of each month)
- Make an effort to pre-announce (often 1-4 weeks in advance)
Those bullets are based on the premise that we control the process for disclosure/development/etc. This is true and appropriate for the common case where the security vulnerability originates in code maintained directly by CiviCRM.
But there is another common case: dependencies ("libraries", "packages", "subpackages", etc) used by CiviCRM and maintained by another group. These break the bullet-points from above:
- The purpose of upstream's public advisory is to notify people like us. The issue is necessarily public when we get the information.
- There are several different upstreams. Their release scheduling is (on the whole) fluid - some have release-windows; some don't; some make pre-announcements; some don't; each of those policies may change over time.
- The vulnerability is public. Delaying the release (in service of a pre-announcement/spin-up period) exacerbates the risk exposure. We don't want our scheduling to add extra exposure.
The security updates of a dependency affect CiviCRM in a few ways. Anyone reading this probably has some understanding already. But just to be complete, those effects include:
- Dependency-updates require some correlated change in how we use that dependency. In the best case, that just means metadata (eg
composer.json
,composer.lock
) - but it can also be much more involved. It varies case-by-case. - Several artifacts need to be republished when a dependency changes (notably the tarballs/zipballs for WP/D7/BD/J - but also any images published via docker, etc).
Proposal
Security fixes that have been previously published by an upstream vendor should be assimilated through CiviCRM's public development channels (Gitlab/Github/Mattermost/etc). The process should closely match the process for patch-releases that fix recent-regressions:
- Like a regular patch-release...
- Any patches/PRs should be submitted to the RC's public queue.
- After approving the RC PR, then backport to stable/ESR. (Only backport if we believe it to be "likely" exploitable.)
- Discussion about testing,
r-run
, compatibility, etc can happen in the public PR. - We do not need to assign a CIVI-SA-* identifier or write an "advisory" record.
- In addition, there are extra bits...
- We'll send a mailblast when the stable/ESR updates are published.
- Release notes should highlight the "Security" issue as distinct from any other "Bug fix" issues. They should link to upstream's advisory (in addition to the usual Github/Gitlab links for Civi).
- In the public media, don't discuss how to specifically exploit the vulnerability. If that requires discussion, go to private Mattermost (
security
and/or direct-message). The public PR may have general claims (eg "I have successfully exploited this on my local system"; or "Alice, Bob, and Carol discussed on security channel - and all felt it is probably exploitable.") - Backports for stable and ESR will be done in parallel. (They may be done by different people).
All other security issues (ie first-party vulnerabilities; unpublished third-party vulnerabilities; uninvestigated vulnerabilities) should continue through the current (private) workflows.
We should update https://civicrm.org/security to indicate this distinction.
Rationale
- If a black hat is motivated enough to monitor CiviCRM's issue/PR queue for heads-up about CiviCRM vulnerabilities, then they can just as easily monitor the official release feeds for
dompdf
,ckeditor
, etc. - Github's "dependabot" is already likely to post public PRs when there's a published vulnerability affecting a CiviCRM dependency.
- Pro-active contributors will find it natural to raise these issues publicly (because they're already public).
- This change should reduce typical turn-around-time / duration-of-exposure for this type of issue. (Compare: 2 weeks vs 0-3 days)
- Routing dependency-updates through the private security medium adds noise to the private tracker without adding much security benefit.
Other thoughts
Microsoft made "Patch Tuesday" famous. But they generally own all their dependencies.
Drupal has landed on "third Wed" as their release-window. However, they appear to make an exception when a third-party library publishes outside their preferred schedule (ex: https://www.drupal.org/sa-core-2022-006).
If we relax the scheduling on dependency updates, then we don't need to keep 1st Wed on the books. CiviCRM-specific fixes could be like Drupal -- strictly third Wed.
Anecdotally, I feel upstream announcements land on a weekday (esp Tue/Wed/Thu) -- and this lines up the interest of deployers. We could lean into this (eg dependency updates only happen on weekdays).
Note: Backdrop's release-window is any Wed. AFAICS, WordPress, Joomla, and PHP don't have formal release-windows. Based on skimming advisories, Joomla has a strong Tue bias. WP+PHP float around. (Between them, I skimmed ~20 prior releases, and there was only one that landed on a weekend.)