CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support". End Users on shared hosting platforms (CPanel, Plesk etc.) cannot modify my.cnf

changed title from CiviCRM 5.13.4 - CiviCRM alert cannot be actioned: "Future versions of CiviCRM may require MySQL utf8mb4 support" by CiviCRM users on shared hosting platforms (CPanel, Plesk etc.) since they cannot modify my.cnf to CiviCRM 5.13.4 - CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support" by CiviCRM users on shared hosting platforms (CPanel, Plesk etc.) since they cannot modify my.cnf

changed the description

changed title from CiviCRM 5.13.4 - CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support" by CiviCRM users on shared hosting platforms (CPanel, Plesk etc.) since they cannot modify my.cnf to CiviCRM 5.13.4 - CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support". End Users on shared hosting platforms (CPanel, Plesk etc.) cannot modify my.cnf

This seems like the same thing as php versions - ie not everyone can upgrade php either - but in the future we will require later php versions & mysql utfmb8 support - at some point

@mfb

@eileen Perhaps we lower the severity for now? Perhaps a notice for now and elevate to warning before we are ready to make this mandatory?

I do fear that this will cause issues for users on shared hosting as I have asked a couple of hosts to set the required INNODB options and have been told they won't do it.

For now I'd like to lower the severity unless we are planning to make this required soon (before 5.20.x)

I think it would be a good idea to reword the message to make sense to non-technical folks.

mentioned in issue #978 (closed)

Here's the current text of the alert:

Future versions of CiviCRM may require MySQL utf8mb4 support. It is recommended, though not yet required, to configure your MySQL server for utf8mb4 support. You will need the following MySQL server configuration: innodb_large_prefix=true innodb_file_format=barracuda innodb_file_per_table=true

There is no mention of the MySQL utf8mb4 requirement in the Installation Requirements docs, https://docs.civicrm.org/sysadmin/en/latest/requirements/

A section detailing this requirement with instructions of how to apply would be a helpful reference for both CiviCRM hosting providers and self-hosted CiviCRM sites. Then provide a link to this page in the alert message.

Slight tangent, but hear me out:

There is a direct correlation between future and existing CiviCRM users and the complexity of the "Installation Requirements" - the more non-standard and unique, the smaller the CiviCRM user base because the barrier for entry is too high.

Ensuring that CiviCRM is Web hosting control panel "friendly" should be goal.

This would allow Web hosting control panels to bundle CiviCRM as a one-click installable application and support hosting CiviCRM. You will note that CiviCRM is currently absent from the two most popular app directories for CPanel: https://installatron.com/apps http://www.softaculous.com/softaculous/php_requirements

I made a PR for adding this to the sysadmin guide: https://github.com/civicrm/civicrm-sysadmin-guide/pull/165

I wouldn't say the configs are unique to CiviCRM, since they are also needed by other apps to support utf8mb4, and are the default in MySQL ≥ 5.7

As far as why have a plan to migrate to utf8mb4: basically MySQL's utf8 is a misnomer and broken historical accident, so it's time to finally phase it out :) I realize this will require a little "leaning" on service providers and sysadmins to make it happen, thus this system status check.

Thanks @mfb

MySQL 5.7 is very new so I would caution on assuming widespread adoption. I checked the CPanel release notes and they only added support for MySQL 5.7 on Feb 20, 2019 - https://documentation.cpanel.net/display/70Docs/70+Release+Notes#id-70ReleaseNotes-MySQL%C2%AE5.7support

Even with the support, CPanel still recommends MariaDB 10.2 as the database server.

Looking at compatibility between MySQL 5.7 and MariaDB, looks like MariaDB 10.3 is required. https://mariadb.com/kb/en/library/mariadb-vs-mysql-compatibility/

FYI, we have found MariaDB 10.3 to be unstable and dropped using it as a result.

Mysql 5.7 was released in 2015

personally, I think it's fine for CiviCRM to continue to support MySQL 5.6 and 5.5, but I also think it's fine to recommend certain non-default configurations for those versions, and eventually (hard to say when, maybe six months from now? it depends on feedback like this issue) require those configurations.

Fast forward to Feb 20, 2019: MySQL® 5.7 support - In cPanel & WHM version 70 we added support for MySQL 5.7 https://documentation.cpanel.net/display/70Docs/70+Release+Notes#id-70ReleaseNotes-MySQL%C2%AE5.7support

I dont see any answer to the question in this thread. To me I agree @justinfreeman that end users don't have a clue about those messages. And as long as CiviCRM doesn't get upgraded everything keeps working as usual. So why those messages? Which only confuses end users and will probably scare them off of civicrm in the long run. I would say that those messages belong to the release notes/newsletters/dev mailing list etc.

added needs:concept approval ~196 triaged labels

@jaapjansma thanks

I think a warning makes sense, because the idea is this is checking for the recommended configuration, which could be required in the future - see e.g. other warning messages re: mysqli, recommended PHP version, etc.

Are we giving messages other than system checks? We definitely use system checks to warn of various sysadmin things - including php versions & such like that are nearing their end of support (we should also be warning people off old versions of mysql I expect).

It might be that we should have a permission to see system checks so that site administrators can prevent end users seeing them if appropriate - since basically ALL system checks are targetting sysadmins & site admins not data entry people

@justinfreeman OK - cpanel finally put mysql support in in 2019 - but that doesn't make it "very new" - it only makes cpanel very behind!

From our stats I can see the top versions are mysql 5.7 and 'other' - my best guess is it's MariaB 10.3 & there needs to be some hard-coding to make it show, with a little percona thrown in

A similar discussion on #1184 (closed)

Linking in a PR to change the wording for this alert, https://github.com/civicrm/civicrm-core/pull/15946

changed the description

changed title from CiviCRM 5.13.4 - CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support". End Users on shared hosting platforms (CPanel, Plesk etc.) cannot modify my.cnf to CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support". End Users on shared hosting platforms (CPanel, Plesk etc.) cannot modify my.cnf

I understand the pushback, but I agree with Eileen and mfb.

To rehash:

We want users to upgrade regularly (it may not be every month, but hopefully at least when there are security releases, every 3-6 months).
We want CiviCRM to keep improving and fix bugs. Without utf8mb4, you will have issues with:
- Sending emails/newsletters with some types of alphabets or some types of emoji
- Inbound emails (idem).
We know that many CiviCRM users are accidental techies, not professional sysadmins, and some types of hosting do not make it easy to upgrade.
At the same time, we want to keep our users safe (PHP upgrades) and avoid bugs (utf8mb4). We also want to avoid unpleasant situations where their schema might get converted while they upgrade for a security release.
utf8mb4 should already be supported by most hosts, we're not exactly early adopters here.

From what I hear of some reactions pushing back against utf8mb4, is that it's not a problem for you. You're in the lucky ones, but you're basically pulling the blanket on your side. For other people, it means that we lost users or potential users, because they hit those bugs. It's also a bug that people are going to increasingly hit, as those utf8 characters are increasingly used.

WordPress migrated to utf8mb4 somewhere in 2017. Drupal also migrated some time ago (and that's Drupal, sorry for my lack of kindness, but they're not exactly early adopters).

I would add that we are a relatively small community, and doing an upgrade like this is likely to be an upgrade we will want to do once, not have to support two types of installations.

If the problem is that your end users have "administer CiviCRM" permissions, and they should not see system warnings, I think that's a better framing of the issue, but it should not stop us from adding other system checks meanwhile. Someone has to see those messages.

(I know some SaaS civicrm providers patch to hide those messages)

@bgm the issue was two-fold when I reported it:

The alert message was shown to CiviCRM end users - which they do not understand and cannot action
CiviCRM sites on shared hosting platforms cannot modify their MySQL configuration - they will not be able to upgrade CiviCRM at all. Anecdotally: "I guess that can mean for me the end of updates or a very tedious task of migrating CiviCRM elsewhere." - https://civicrm.stackexchange.com/questions/30903/when-will-mysql-utf8mb4-support-be-mandatory

That's it.

Rollout support for utf8mb4 in CiviCRM.

closed

CiviCRM alert cannot be actioned by End Users: "Future versions of CiviCRM may require MySQL utf8mb4 support". End Users on shared hosting platforms (CPanel, Plesk etc.) cannot modify my.cnf

Child items 0

Activity