- Blog posts / updates
- Design philosophy
- Tasks / Status
- Custom code
- Content-type stats
- Regular resyncs (disabled since 2020-04-05)
- Extension directory
- Time tracking
We don't know yet which will be the next CMS, but we know that no matter what the solution, some pages/features will need to be worked on.
We chose Drupal8 (see #141 and #130) as the next CMS for civicrm.org. We are avoiding the word "upgrade", because it's a big change that requires a fair amount of planning, changes to features, and the upgrade will happen over a few months, with iterations.
This page defines the upgrade roadmap and tracks changes at a high level. This page is updated regularly as work progresses.
Blog posts / updates
- 2020-04-05 : https://civicrm.org/blog/bgm/progress-civicrmorg-drupal8-upgrade
- 2019-09-05 : https://civicrm.org/blog/bgm/upgrade-and-translation-plan-for-the-civicrmorg-website
- Keep using a Bootstrap-based solution (v3 or v4) - because everyone knows it, and it's CMS-neutral.
- Avoid CMS-specific plugins
- Promote the CiviCRM Contact dashboard, instead of a CMS user dashboard
- All forms must be managed by CiviCRM or CiviCRM extensions (ex: formbuilder, but not webform).
- Avoid creating CMS-specific plugins for custom code. Migrate them to CiviCRM extensions as much as possible. Reduce the quantity of custom code as much as possible.
- Multi-lingual - it should be possible to easily translate all content.
- Adding content should be easy (not require knowledge of HTML, easy to upload images).
- Document features - main features must have a short wiki page describing how it was built, for the next contributor.
- Keep it simple - minimal use of plugins.
- Make it easy to add blog, comments, civicamp content, but strong gate-keeping of content (request updates via the website issue queue).
- Identify a page/layout builder that offers fixed layouts (define a layout for content "a", "b" and "c", enforces some consistency)
Tasks / Status
- Decide what sections to prioritize, migration workflow based on the technical constraints (user-generated content first, or static content first; see section below)
- #152 Theme port (ex: https://d8.civicrm.org/explore-civicrm)
- Migrate users (and association of user-content)
- #167 Header (menus, logo)
- Blog (content, taxonomy #153, authors, comments, theming; categories block, popular posts block) https://d8.civicrm.org/blog
- #170 Fix missing menu items
- #169 Migrate over most of the simple static pages (nginx proxy) - nodes where type=page and status=published. Pages that are not relevant anymore today have been unpublished.
- Migrate over the 'explore civicrm' page (more complex layout)
- Migrate the Spark page (more complex layout)
- Deprecate https://civicrm.org/newsletter-signup (redirect to 'my mailing prefs')
- #168 Convert 'my mailing pref' to a standard profile (or redirect to the existing one)
- Test if we can do regular automatic resync users from d7->d8, would make it easier to connect d8 on the same CiviCRM instance, for Views.
Have a regular cron to sync users and blog posts (
crontab -e -u aegir)
- Main /blog page (live on D8, thanks to the sync)
- #179 Download page (port code over, send live traffic to it)
- local-sites#4 Enable and test multi-lingual on a small scale
- #171 Decide which languages to support in the short term (most popular are: German, French, Spanish, Dutch)
- #139 User Dashboard
- CSS/i18n of CiviCRM public form, Ex: https://civicrm.org/fr/civicrm/contribute/transact?reset=1&id=89
- User account creation
- #185 Fix civicrm_org_newsfeed or find an alternative (required for json newsfeed/dashlet)
- #186 Make users login to d8.c.o by default (via proxying), so that blog posts, jobs are on d8 only (case studies will have to wait?).
- CiviCRM forms go live
- Jobs - TODO: fix the location field (add a plain text field? and add to the Job view)
- CSS of CiviCRM public pages (contributions, events).
- Fix the pseudo-ldap logins with Gitlab
- Partner listings
- #200 Security-related views
- #205 Member listing
- Migrate the location of the CiviCRM database, so that it lives in Drupal8 only (after most of the CiviCRM-based views have been migrated over to d8).
- #219 Review URL redirects from Drupal7 (we can't use the redirect module in d8, civicrm bug, #177)
- #199 Extensions (and see the "Extensions" section below)
Migrate over all the rest of the static pages
- CMS pages: wordpress, drupal, etc.
- Features pages, ex: https://civicrm.org/features/configurable-customizable
- #175 Connect CiviCRM to d8
- Case Studies
- #233 Make it happen campaigns
- #218 Demo sites
- Front page (see also #143, but the page was moved to Gutenberg pending revision, so that we can work on translation mechanisms)
- ccrm_extensionvalidation: Mostly removed by removing the possibility to release manually. See also: extensions/extensions-directory#45
- civicrm_org_members: sumfields hook for member program? (only a few lines of code)
- #185 civicrm_org_newsfeed: does a Views hack to generate RSS feeds. Incorporated into the civiorg8 custom module.
- civicrm_org_providers: adds badges to parters (in a custom field)
- civicrm_org_stats: d7 tokens for content - deprecated / not ported
- civicrm_org_stats: extensions/extensions-directory#14 updates extension usage stats
- cividownload: download page, and a fundraising campaign that has long been disabled. It's now a blog that is incorporated in a Gutenberg page. Code is in civiorg8.
extdir/git-urls.jsonand other services (medium-sized) - most of it should be replaced by Comex, but we may need to do more review.
This was to help evaluate where to focus, and track which content-types were regularly re-synchronized, and which were one-off migrations.
> select type, count(*) from node group by type; +---------------------------+----------+ | type | count(*) | +---------------------------+----------+ | advisory | 94 | (done) | blog | 2447 | (done) | ccrm_case_study | 188 | <---- Important | civicamp | 19 | | community_highlight | 9 | | conference | 5 | | demo_site | 12 | | download | 4 | ? | event_detail_page | 22 | | extension | 407 | (done) | extension_release_civicrm | 1506 | (done) | extension_release_cms | 54 | (deleted, was not used, & we do not allow manual releases anymore, extdir does not support non-civi modules/plugins) | faq | 2 | | feature | 11 | | groups | 10 | <--- deprecate? (lists WGs, but we can point to Gitlab) | homepage_slide | 17 | (done, deleted) | homepage_stat | 4 | (done, deleted) | job | 71 | (done) | lang_reg | 2 | ? | make_it_happen | 38 | | member_benefit | 15 | | option | 9 | <--- simple nodes, probably embedded in some panels..? | page | 158 | (done) | sectioned_page | 5 | | sponsor | 68 | | story | 4 | (nodes 1147 to 1150 .. delete?) | strategic_initiative | 12 | | team | 4 | | webform | 42 | (delete?) | working_group | 22 | +---------------------------+----------+ 30 rows in set (0.00 sec)
- CRM for non-profits, what drives uptake?, 2019-04 survey
- Beyond English: improving internationalization in CiviCRM, last responses in 2019-08, survey
- #209 Register a site, very active
- Keep in touch with CiviCRM, newsletter/security signup form, still used. - Now redirects to /civicrm/mailing/subscribe.
Queued for deletion, deleted, or redirected:
- Update my mailing preferences, lots of spam, linked from the site footer. Redirects to civicrm/mailing/subscribe.
- Newsletter Signup, used, but lots of spam. Redirects to civicrm/mailing/subscribe.
- Receive Security Advisories - deleted in favour of the "update my mailing preferences" (already done on d7)
- Sign up to our newsletter, last subscription in 2016
- Update my details, last submission in 2016, now unpublished
- Where are you at with CiviCRM?, last submission in 2016, unpublished
- Paid issue queue old, according to the URL, unpublished
- Service provider information, 2013-2018, few responses
- Training feedback, 2014-2017, few responses
- CiviCRM user survey, 2013, few responses
- Submit Session Proposal to St Louis, we had many identical variants of these pages, most were spammed and now deleted, but keeping one of them for future reference.
- Voter registration challenge, EC-CC form, mostly spammed
- Voter Registration for the Community Council process, EC-CC form
- Nomination form for the Community Council, EC-CC form
- CiviCamp London Feedback, 2018, should use profile for participant info.
- Your Feedback on CiviCamp SF Bay Area 2018, 2018, should use profile for participant info.
- Session Proposal: CiviCamp San Francisco Bay Area 2018, 2018
- CiviCamp San Francisco Bay Area: What do you want to get out of it?, 2018, should be participant profile
- CiviCamp Feedback, Hartford 2018? should use participant profile update.
- Register page, spammed/closed, was never really used, should be a profile or membership page instead.
- Join the Marketing Team, 2013-2014
- CiviCRM Release Sign Up - not sure what the point was.
- Update a Contact and their Organization - zero submissions, seems like a test.
- CiviCRM Survey 2016 - a lot of responses
- Signup a member - zero submissions, created by Coleman.
- Record your contributions to CiviCRM! - moved to Gitlab/Timetrack
- CiviDay London Feedback - had one submission.
- CMS Info - 4 responses.. what was it for? created by Josh.
- Work with the core team, zero submissions, created by Michael
- Win a ticket to CiviCon Denver, worth noting that it received over 100 submissions.
- Ambassador renewal, one test submission, by Michael
- CiviCRM User Summit Feedback, should use participant profile
- Pledge to Test - No experience neccessary
- Pledge Support for 4.5, had over 100 submissions
- Poll - March 2014 Newsletter, had over 100 submissions
- London Meetup Feedback, 2013
- Newsletter signup - partner bar, 2015
- Configuring, Customizing and Extending CiviCRM - Toronto Feedback, 2-3 submissions, old
- Contact an Ambassador
- CiviCRM UK Sprint - Butcombe 2012 feedback
- CiviCRM Euro Sprint Apeldoorn 2012 feedback
- CiviCon feedback, 2017
- Sprint feedback, should use participant profile update, last used in 2019-02, but mostly 2017.
- Sprint feedback (old)
- Conference feedback (old)
- Training feedback (old)
- CiviCRM Release Sign Up
Regular resyncs (disabled since 2020-04-05)
This was happening every 2h so that we could sync the blog/users to Drupal8. It was disabled on 2020-04-05, now that the Drupal8 site is the canonical version for content.
drush migrate-import upgrade_d7_user --update drush migrate-import upgrade_d7_node_blog --update drush migrate-import upgrade_d7_url_alias --update
NB: there is a node using the alias /blog and it's breaking the blog view. We should delete it on the d7 site once /blog is served from d8, so that the resync does not break the d8 view.
Drupal7 setup to port to Drupal8:
- Extension node
- Extension release
- CMS-specific views
- RSS feeds
- Releases for a specific extension
Jenkins job that runs a drush script: https://test.civicrm.org/view/All/job/Extdir%20Scan%20(civicrm.org)/
Relies on a Drush command in 'extdir' (
sudo -u aegir -H drush @civicrm.org extdir-scan)
- Relies on a Drush command in 'extdir' (
- [ x] Fix field permissions on the Extension node (i.e. do not allow changing the Extension Review Status?)
- Stats extraction (for stats.c.o)
- JSON feeds for the in-app extension browser / Symfony app
- Views for generating the data used by the Symfony app
Now that Drupal8 is the main front-facing CMS on civicrm.org (handling user management), extensions are running from d7.civicrm.org and some requests are proxied. It's not possible to create new extensions or new extension releases, but json feeds continue to work. Need to either fix d7, or quickly port over to d8.
Things to consider:
- The most important bit is the drush script that Jenkins run.
- There is a fair amount of code for allowing manual creation of releases, and we could probably disable that option (only allow it for admins, for now, without validation, since it's rarely used and clunky).
- Porting the views that produces the json feeds should be fairly straight forward. They had not been very much customized?
In mid-2020, as we shift our focus on Comex:
- Comex would generate the feed required for Composer
- Comex could feed itself off c.o, or it could feed itself from a repo with json files. Adding an extension would require doing a merge-request on the repo to add the json file.
- If it helps, adding an extension node on c.o might auto-create the Gitlab project under /extensions. It would ensure that the namespace is reserved there too, and avoid dependency on ext-wg leads for creating those repos.
- Remove CMS-specific releases. They haven't been used in over a year, with the exception on one item identified as a Drupal module but is actually an extension that has a WordPress bug (donation receipts).
- Remove CMS-specific RSS feeds (ex: /extensions/(drupal|wordpress|backdrop|joomla)/rss) - they have zero traffic, not worth the hassle re-creating/testing them.
civi/w8 (website drupal 8 upgrade)