Translation issueshttps://lab.civicrm.org/dev/translation/-/issues2021-08-12T19:25:30Zhttps://lab.civicrm.org/dev/translation/-/issues/67Canonize API for storing translated data2021-08-12T19:25:30ZtottenCanonize API for storing translated data# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is ...# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is most immediately motivated by https://lab.civicrm.org/dev/mail/-/issues/83, which aims to improve the process+experience of drafting+testing workflow templates. For this case, the string that is being edited (ie `civicrm_msg_template.msg_html`) is a relatively rich piece of content (with HTML tags, tokens, Smarty expressions - which in turn may vary based on the context for which the template will be used). The richness of the text implies that one should have more features available (token-pickers, syntax-highlighting, ad nauseum). Editing a translation of this content in a generic textbox (as with multilingual UI, Transifex UI, or POEdit) would be difficult and error-prone.
* This is intended as a step in support of https://lab.civicrm.org/community/feature-request/-/issues/26, which is a broad effort (initiated by @ayduns @BjoernE) to re-conceive how the multilingual subsystem works. TLDR: Current multilingual requires significant MySQL schema manipulation. This works for 1-3 languages but does not scale to 10 languages. Resolving it requires changes in the storage/lifecycle of translated data.
* Inspired by this discussion, Eileen wrote a proof-of-concept extension https://github.com/eileenmcnaughton/civi-data-translate. The scope of `civi-data-translate` mostly matches the scope of this filing, but not quite perfectly. It matches insofar as it introduces an APIv4 interface and a MySQL table for strings. It diverges insofar as it specifically touches on `MessageTemplate`. (The work for `MessageTemplate` is left as a separate matter.) Its biggest obstacle is dependency-hell: it requires a skilled administrator to maintain a deployment, which disincentivizes development and usage.
# Approaches
Working within the limits of available code and capacity, it appears feasible to adapt `civi-data-translate` to this purpose. Either:
1. Move its APIv4 interface and data-storage to core-proper, or...
2. Move its APIv4 interface and data-storage to core-extension.
# Comments
* Having an API to edit the strings would be meaningless if we did not have a data-store.
* There is a performance question about using MySQL for a string table. (Most FOSS applications use `gettext` MO files which are optimized for fast lookup of static strings. This is how Civi handles translation of its numerous app-strings.) In prior discussions with @BjoernE @ayduns etal, we identified this balance:
* There is a difference between *administration* (browsing/editing strings) and *runtime lookup* (substituting 1000 strings during a page-load).
* For administration, there is no question about whether the performance of a MySQL string-table would be acceptable. It would be. In fact, many different tools/workflows/stores can be acceptable.
* The performance question is relevant to *runtime lookup of heavily used strings*. The performance question is not necessarily closed, and it depends on other variables (*the #data-strings, the use-case, the hardware, etc*).
* If one does need to optimize lookup, the best known approach is to compile to gettext. To wit: Read strings from whatever source is handy, aggregate them, and [write them](https://github.com/pear/File_Gettext/blob/master/File/Gettext/MO.php) to a cache folder in `*.mo` format. (You can see [de.systopia.l10nmo](https://github.com/systopia/de.systopia.l10nmo) as a foray into this approach of blending/merging string sources.)
* I was worried about proposing this - specifically, worried that it might conflict with a more optimized dataflow. However, on reflection, I think it is complementary progress. Suppose you wanted to patch `l10nmo` to include a feed of strings provided by web-based administrators. If each web UI stored strings differently, then you'd probably give up. But if they use the same (shared/documented) string API, then it's easier to pull from there.
* (*I mention this as a hypothetical. In practice, some things like dev/mail#83 can be achieved without this level of optimization. The upshot is that we can bite off a chunk of work here on the API/storage side and make some incremental progress.*)https://lab.civicrm.org/dev/translation/-/issues/56Membership Types are accidentally translated on Find Membership2023-05-18T23:33:29ZbgmMembership Types are accidentally translated on Find Membership* Switch the locale to French
* Create a Membership Type called "Premium"
* Go to Find Membership screen
Result: "Premium" is displayed as "Cadeau" instead of the intended label.
cc @alainb* Switch the locale to French
* Create a Membership Type called "Premium"
* Go to Find Membership screen
Result: "Premium" is displayed as "Cadeau" instead of the intended label.
cc @alainbhttps://lab.civicrm.org/dev/translation/-/issues/44When you have multiple available languages using the language switcher on an ...2020-04-14T14:46:25ZDaveDWhen you have multiple available languages using the language switcher on an angular page gives page not foundI thought I'd seen this reported already but can't find it. Maybe I was thinking of https://lab.civicrm.org/dev/translation/-/issues/23 which is maybe related but seems a bit different. The issue here is the resulting url.
1. On the loc...I thought I'd seen this reported already but can't find it. Maybe I was thinking of https://lab.civicrm.org/dev/translation/-/issues/23 which is maybe related but seems a bit different. The issue here is the resulting url.
1. On the localization page add another language in the "Available languages" field.
1. Visit an angular page like Admin - CiviCase - Case Types.
1. Using the language switcher (on drupal 7 it's in the left sidebar), change the language.
1. It will change the language, but you get a page that just says "Unknown path".https://lab.civicrm.org/dev/translation/-/issues/41Invoice for recurring contribution should be sent in the preferred language o...2022-11-25T16:01:21ZbgmInvoice for recurring contribution should be sent in the preferred language of the contactThe contact record has a "preferred language" field, which can be automatically set when people fill in Contribution or Event forms.
That field can be used in certain circumstances, such as in Mailings and in Scheduled Reminders.
Howev...The contact record has a "preferred language" field, which can be automatically set when people fill in Contribution or Event forms.
That field can be used in certain circumstances, such as in Mailings and in Scheduled Reminders.
However, it is not used in the following circumstances:
* In Administration > CiviContribute > Component settings, enable "tax and invoicing", and enable PDF invoices.
* In Administration > Localization > Languages, enable multilingual, and enable a second locale.
* Setup a form that supports recurring contributions, enable email receipts
* As an anonymous user, donate on that form in the second language (not the default global language), selecting a recurring (ex: daily) donation.
Double-check that the contact record that was created has the second-language. Change if it necessary (I forget where the setting is, for defining the pref. language).
Then, as time passes, notice that the recurring email receipts are in the default site language. This is because the cron or IPNs will usually run in the default site language.
`CRM_Core_BAO_ActionSchedule::setCommunicationLanguage()` would be a good implementation reference. We should probably move it to `CRM_Core_I18n` for clarity.https://lab.civicrm.org/dev/translation/-/issues/27Translation option missing for "on behalf of label"2021-06-02T18:01:34ZMartinTranslation option missing for "on behalf of label"I might be misunderstanding how this is supposed to work, but here's the situation that I'm seeing. I have multilingual for civi turned on, and have a contribution page. For the text/longtext fields there is a translation icon next to th...I might be misunderstanding how this is supposed to work, but here's the situation that I'm seeing. I have multilingual for civi turned on, and have a contribution page. For the text/longtext fields there is a translation icon next to the field label in the admin when configuring the page. But when I enable the "allow on behalf of" option, the "on behalf of label" does not have this translation option showing (see screenshot). So it seems that we're not able to provide the right text to the end user in this case.
In addition, when displaying the translated page the following Drupal error shows:
> Notice: Undefined index: for_organization in CRM_Contribute_Form_ContributionBase->buildComponentForm() (line 976 of /home/mysite/public_html/sites/all/modules/civicrm/CRM/Contribute/Form/ContributionBase.php).
Running civicrm 5.12.4 on drupal 7.66 with php 5.6 (I know, I know). Issue was also seen previously on civi 4.7.31.
[civi_trans_issue_1](/uploads/a300619c6ab7cbcabb43e521f7bd46ed/civi_trans_issue_1.jpg)https://lab.civicrm.org/dev/translation/-/issues/20DB Error: no such table when executing schedule jobs2022-11-07T18:05:35ZYepaDB Error: no such table when executing schedule jobsWe've a single language installation with CiviCRM 5.6.1 & Drupal 7
After we've enabled the "Multiple Language" option under "Multiple Languages Support" section in the config page:
https://www.domain.org/civicrm/admin/setting/localiz...We've a single language installation with CiviCRM 5.6.1 & Drupal 7
After we've enabled the "Multiple Language" option under "Multiple Languages Support" section in the config page:
https://www.domain.org/civicrm/admin/setting/localization?reset=1
If we prepare a mailing with a language different from the language interface, when we execute the schedule jobs, we have the following error:
```
Feb 14 18:17:32 [info] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => handle
)
[code] => -18
[message] => DB Error: no such table
[mode] => 16
[debug_info] => select id, extends, extends_entity_column_value, style from civicrm_custom_group__en_US where is_active = 1 [nativecode=1146 ** Table 'drupal.civicrm_custom_group__en_US' doesn't exist]
[type] => DB_Error
[user_info] => select id, extends, extends_entity_column_value, style from civicrm_custom_group__en_US where is_active = 1 [nativecode=1146 ** Table 'drupal.civicrm_custom_group__en_US' doesn't exist]
[to_string] => [db_error: message="DB Error: no such table" code=-18 mode=callback callback=CRM_Core_Error::handle prefix="" info="select id, extends, extends_entity_column_value, style from civicrm_custom_group__en_US where is_activ$
)
Feb 14 18:17:32 [info] $backTrace = #0 /var/www/drupal/public_html/sites/all/modules/civicrm/CRM/Core/Error.php(232): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 [internal function](): CRM_Core_Error::handle(Object(DB_Error))
#2 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/PEAR.php(921): call_user_func((Array:2), Object(DB_Error))
#3 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: no such table", -18, 16, (Array:2), "select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#4 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/PEAR.php(575): DB_Error->__construct(-18, 16, (Array:2), "select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#5 [internal function](): PEAR->_raiseError(Object(DB_mysqli), NULL, -18, NULL, NULL, "select id, extends, extends_entity_column_value, style from civicrm_custom_gr...", "DB_Error", TRUE)
#6 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/PEAR.php(224): call_user_func_array((Array:2), (Array:8))
#7 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#8 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->raiseError(NULL, -18, NULL, NULL, "select id, extends, extends_entity_column_value, style from civicrm_custom_gr...", "DB_Error", TRUE)
#9 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-18, NULL, NULL, NULL, "1146 ** Table 'drupal.civicrm_custom_group__en_US' doesn't exist")
#10 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#11 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#12 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/DataObject.php(2415): DB_common->query("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#13 /var/www/drupal/public_html/sites/all/modules/civicrm/packages/DB/DataObject.php(1607): DB_DataObject->_query("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#14 /var/www/drupal/public_html/sites/all/modules/civicrm/CRM/Core/DAO.php(438): DB_DataObject->query("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#15 /var/www/drupal/public_html/sites/all/modules/civicrm/CRM/Core/DAO.php(1411): CRM_Core_DAO->query("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...", TRUE)
#16 /var/www/drupal/public_html/sites/all/modules/civicrm/drupal/modules/views/components/civicrm.core.inc(3119): CRM_Core_DAO::executeQuery("select id, extends, extends_entity_column_value, style from civicrm_custom_gr...")
#17 /var/www/drupal/public_html/sites/all/modules/civicrm/drupal/modules/views/civicrm.views.inc(87): _civicrm_core_data((Array:301), (Array:8))
#18 /var/www/drupal/public_html/sites/all/modules/contrib/views/includes/cache.inc(93): civicrm_views_data_alter((Array:301))
#19 /var/www/drupal/public_html/sites/all/modules/contrib/views/includes/cache.inc(37): _views_fetch_data_build()
#20 /var/www/drupal/public_html/sites/all/modules/contrib/views/views.module(1319): _views_fetch_data("twitter", FALSE, FALSE)
#21 /var/www/drupal/public_html/sites/all/modules/contrib/views/views.module(1655): views_fetch_data("twitter", FALSE)
#22 /var/www/drupal/public_html/sites/all/modules/contrib/views/includes/view.inc(277): views_move_table("twitter")
#23 /var/www/drupal/public_html/sites/all/modules/contrib/views/views.module(1643): view->update()
#24 /var/www/drupal/public_html/sites/all/modules/contrib/views/views.module(762): views_get_view("tweets")
#25 [internal function](): views_block_view("tweets-block")
#26 /var/www/drupal/public_html/includes/module.inc(934): call_user_func_array("views_block_view", (Array:1))
#27 /var/www/drupal/public_html/modules/block/block.module(911): module_invoke("views", "block_view", "tweets-block")
#28 /var/www/drupal/public_html/modules/block/block.module(690): _block_render_blocks((Array:1))
#29 /var/www/drupal/public_html/modules/block/block.module(319): block_list("footer3")
#30 /var/www/drupal/public_html/modules/block/block.module(270): block_get_blocks_by_region("footer3")
#31 /var/www/drupal/public_html/includes/common.inc(5914): block_page_build((Array:11))
#32 /var/www/drupal/public_html/includes/common.inc(2761): drupal_render_page("\n<div id=\"crm-container\" class=\"crm-container\" lang=\"en\" xml:lang=\"en...")
#33 /var/www/drupal/public_html/includes/common.inc(2634): drupal_deliver_html_page("\n<div id=\"crm-container\" class=\"crm-container\" lang=\"en\" xml:lang=\"en...")
#34 /var/www/drupal/public_html/includes/menu.inc(542): drupal_deliver_page("\n<div id=\"crm-container\" class=\"crm-container\" lang=\"en\" xml:lang=\"en...", "")
#35 /var/www/drupal/public_html/index.php(21): menu_execute_active_handler()
#36 {main}
```
The global $dbLocale is equal to '__en_US' because it's not correctly set in the public function setLocale($locale) of i18n.php. The $locale is equal to '_en_US' when the fatal error occurred.
```
/**
* Change the processing language without changing the current user language
*
* @param $locale
* Locale (for example 'en_US', or 'fr_CA').
* True if the domain was changed for an extension.
*/
public function setLocale($locale) {
// Change the language of the CMS as well, for URLs.
CRM_Utils_System::setUFLocale($locale);
// change the gettext ressources
if ($this->_nativegettext) {
$this->setNativeGettextLocale($locale);
}
else {
// phpgettext
$this->setPhpGettextLocale($locale);
}
// for sql queries
global $dbLocale;
$dbLocale = "_{$locale}";
}
```
"inherit locale" option is enabled and no cache error.
Here is the backtrace from setLocale function: [locale_not_well_set_error.txt](/uploads/29d6263e244abc1d0973ff4679463615/locale_not_well_set_error.txt)
I tried without sendgrid extension, the error was the same.