Implement private distribution channel for ESR
(This note is mostly written for myself.) I've been debating a bit about how to do this ("Design 1", "Design 2") and wrote down both for my own sanity.
After re-reading these, "Design 1" seems to make more sense than "Design 2". Why? In my imagination, I hoped "Design 2" would leave us with a simpler overall arrangement (e.g. we'd use the same steps for publishing tarballs in any version); however, this doesn't work out because there are multiple differences in the publication process, and they don't all evaporate just because we shuffle the gs buckets a bit differently. So if life is going to be a little complicated, then we might as well go with "Design 1" and keep some of its merits.
Design 1 (Accepted)
Merits:
- Less change to existing downloads
- Normal public downloads/redirects can be serviced faster (by nginx instead of php/symfony)
General layout:
-
gsutil://civicrm
remains a public bucket; it only gets public files -
gsutil://civicrm-build
becomes a private bucket; buthttps://download.civicrm.org/latest/*
redirects will include auth code -
gsutil://civicrm-private
is a new private bucket;https://download.civicrm.org/esr/*
redirects will include auth code
Tasks:
- Part 1: Make existing channels more secure
-
distmgr: the /latest
list of branches should be filtered to show$(http://latest.civicrm.org/stable.php)
and newer -
distmgr: the download redirects should include auth code -
distmgr: deploy changes -
gcloud: make gs://civicrm-build
private; re-test public links; verify auth required -
gcloud: create private gs://civicrm-private
-
- Part 2: Publish ESR tarballs securely
-
releaser: implement --esr-publish (which submits to gs://civicrm-private
instead ofgs://civicrm
; and it skips sourceforge) -
(If we want git tags) -
lab: create private projects (/esr/civicrm-core, /esr/civicrm-wordpress, /esr/civicrm-drupal, etc) -
releaser: implement --esr-tag
-
-
(bonus; since we're in there) releaser: generate sha256 checksums -
release-mgmt: write 5.x-esr.md
instructions
-
- Part 3: Provide accessible end-point for downloading ESR
-
distmgr: add an authenticated listing '/esr', in which URL's provide authentication -
nginx: requests for download.civicrm.org/esr should require http authentication -
FIXME: define process to manage htaccess list (Idea: setup an extension/module/endpoint for servicing https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/ ) -
monitoring: verify that download.civicrm.org/esr is up and require http auth
-
Design 2 (Rejected)
Merits:
- More consistent download mechanics (among various builds/versions)
General layout:
-
gsutil://civicrm
andgsutil://civicrm-build
both become private. -
https://download.civicrm.org/*
redirects include auth code; this includes- links currently handled by symfony redirect --
/latest/*
- links currently handled by nginx redirect (these need to change to go through symfony)
- new links provided by
/esr
- links currently handled by symfony redirect --
Tasks:
- Part 1: Make existing channels more secure
-
distmgr: the /latest
list of branches should be filtered to show$(http://latest.civicrm.org/stable.php)
and newer -
distmgr: the download redirects should include auth code -
distmgr: remove nginx redirects -
distmgr: add routes which redirect the simple cases of gs://civicrm/*
subdirs -
distmgr: deploy changes -
gcloud: make gs://civicrm-build
private; re-test public links; verify auth required -
gcloud: make gs://civicrm
private; re-test public links; verify auth required
-
- Part 2: Publish ESR tarballs securely
-
releaser: implement --esr-publish (which skips sourceforge) -
(If we want git tags) -
lab: create private projects (/esr/civicrm-core, /esr/civicrm-wordpress, /esr/civicrm-drupal, etc) -
releaser: implement --esr-tag
-
-
(bonus; since we're in there) releaser: generate sha256 checksums -
release-mgmt: write 5.x-esr.md
instructions
-
- Part 3: Provide accessible end-point for downloading
-
distmgr: add an authenticated listing '/esr', in which URL's provide authentication -
nginx: requests for download.civicrm.org/esr should require http authentication -
monitoring: verify that download.civicrm.org/esr is up and require http auth
-