Translation issueshttps://lab.civicrm.org/dev/translation/-/issues2021-08-12T19:25:30Zhttps://lab.civicrm.org/dev/translation/-/issues/67Canonize API for storing translated data2021-08-12T19:25:30ZtottenCanonize API for storing translated data# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is ...# Goal
Enable richer user experiences which incorporate data-translation. Specifically, provide a CRUD API for administrative applications that need to read/write alternate versions of a string in the database.
# Background
* This is most immediately motivated by https://lab.civicrm.org/dev/mail/-/issues/83, which aims to improve the process+experience of drafting+testing workflow templates. For this case, the string that is being edited (ie `civicrm_msg_template.msg_html`) is a relatively rich piece of content (with HTML tags, tokens, Smarty expressions - which in turn may vary based on the context for which the template will be used). The richness of the text implies that one should have more features available (token-pickers, syntax-highlighting, ad nauseum). Editing a translation of this content in a generic textbox (as with multilingual UI, Transifex UI, or POEdit) would be difficult and error-prone.
* This is intended as a step in support of https://lab.civicrm.org/community/feature-request/-/issues/26, which is a broad effort (initiated by @ayduns @BjoernE) to re-conceive how the multilingual subsystem works. TLDR: Current multilingual requires significant MySQL schema manipulation. This works for 1-3 languages but does not scale to 10 languages. Resolving it requires changes in the storage/lifecycle of translated data.
* Inspired by this discussion, Eileen wrote a proof-of-concept extension https://github.com/eileenmcnaughton/civi-data-translate. The scope of `civi-data-translate` mostly matches the scope of this filing, but not quite perfectly. It matches insofar as it introduces an APIv4 interface and a MySQL table for strings. It diverges insofar as it specifically touches on `MessageTemplate`. (The work for `MessageTemplate` is left as a separate matter.) Its biggest obstacle is dependency-hell: it requires a skilled administrator to maintain a deployment, which disincentivizes development and usage.
# Approaches
Working within the limits of available code and capacity, it appears feasible to adapt `civi-data-translate` to this purpose. Either:
1. Move its APIv4 interface and data-storage to core-proper, or...
2. Move its APIv4 interface and data-storage to core-extension.
# Comments
* Having an API to edit the strings would be meaningless if we did not have a data-store.
* There is a performance question about using MySQL for a string table. (Most FOSS applications use `gettext` MO files which are optimized for fast lookup of static strings. This is how Civi handles translation of its numerous app-strings.) In prior discussions with @BjoernE @ayduns etal, we identified this balance:
* There is a difference between *administration* (browsing/editing strings) and *runtime lookup* (substituting 1000 strings during a page-load).
* For administration, there is no question about whether the performance of a MySQL string-table would be acceptable. It would be. In fact, many different tools/workflows/stores can be acceptable.
* The performance question is relevant to *runtime lookup of heavily used strings*. The performance question is not necessarily closed, and it depends on other variables (*the #data-strings, the use-case, the hardware, etc*).
* If one does need to optimize lookup, the best known approach is to compile to gettext. To wit: Read strings from whatever source is handy, aggregate them, and [write them](https://github.com/pear/File_Gettext/blob/master/File/Gettext/MO.php) to a cache folder in `*.mo` format. (You can see [de.systopia.l10nmo](https://github.com/systopia/de.systopia.l10nmo) as a foray into this approach of blending/merging string sources.)
* I was worried about proposing this - specifically, worried that it might conflict with a more optimized dataflow. However, on reflection, I think it is complementary progress. Suppose you wanted to patch `l10nmo` to include a feed of strings provided by web-based administrators. If each web UI stored strings differently, then you'd probably give up. But if they use the same (shared/documented) string API, then it's easier to pull from there.
* (*I mention this as a hypothetical. In practice, some things like dev/mail#83 can be achieved without this level of optimization. The upshot is that we can bite off a chunk of work here on the API/storage side and make some incremental progress.*)https://lab.civicrm.org/dev/translation/-/issues/47Investigate javascript currency library2023-06-02T19:33:39ZeileenInvestigate javascript currency libraryI think that we should use an approach for money fields similar to date fields where formatting is at the js layer and the values are always submitted in unrounded US decimal format - ie. always submitted like 4,000.0123 (or preferably 4...I think that we should use an approach for money fields similar to date fields where formatting is at the js layer and the values are always submitted in unrounded US decimal format - ie. always submitted like 4,000.0123 (or preferably 4000.123) and the php can expect just to 'use what it gets'
There are a number of possible js libraries. https://bashooka.com/coding/javascript-libraries-for-formatting-number-currency-time-date/
I feel like the requirements are
1) Format currencies appropriately. If I have a page that accepts donations in EUR it should display as 2.534.234,00 €
2) Format currencies by locale if known. If I 'know' - probably from a drupal url or similar that my page is for French Euro users then I want 2 534 234,00 €
3) Use the site numeric separator on backend screens. If I'm used to using a US decimal format I want all donations displayed with the same 'meaning' of the decimal
4) Format on input.
5) Package is appropriately licenced
The first 3 seem comfortably addressed by https://osrec.github.io/currencyFormatter.js/ which also offers the very nice
```
<div class='money' data-ccy='EUR'> 1234564.58 </div>
<div class='money' data-ccy='GBP'> 8798583.85 </div>
<div class='money' data-ccy='CHF'> 0.9754 </div>
```
syntax - which we could extend for scenarios 2 & 3 if necessary. Is MIT license OK?
The package seems pretty static - and they have not merged a PR to add Romanian which might be not great. I'm just looking at a couple of others as well
I REALLY like the jquery syntax above :-)
See https://lab.civicrm.org/dev/translation/-/issues/47 for same topic but php layer
Also Format on input - http://autonumeric.org/ looks promising - also MIT
Dinero doesn't really seem to do much formatting https://dinerojs.com/module-dinero
http://numbrojs.com/format.html also offers pretty reasonable formatting. I like the 'unformat' function but I wonder if it would work for Euro currencies
Note the screenshot demonstrates where full locale-based formatting would make no sense. The rows all need to use the decimal point the same way
![Screen_Shot_2020-05-18_at_4.35.13_PM](/uploads/58e7cfce6e88ccae2e19ba1b711d9dc4/Screen_Shot_2020-05-18_at_4.35.13_PM.png)
https://lab.civicrm.org/dev/translation/-/issues/46Proposal - languages from urls2020-05-26T20:15:41ZeileenProposal - languages from urlsI want to propose an addition to our basic edit forms
1) in CRM_Core_Form -add a property language and in generic preProcess look for this property in the url & set it on the form
2) in any APIv4 calls at the form layer use this value t...I want to propose an addition to our basic edit forms
1) in CRM_Core_Form -add a property language and in generic preProcess look for this property in the url & set it on the form
2) in any APIv4 calls at the form layer use this value to setLanguage, if exists, on the api call (this is mostly a principle of intent at this stage)
What does this achieve? Basically it allows me to take action in an extension. Currently the MessageTemplate form uses apiv4 to retrieve and save message templates. I can listen to these calls in civi_data_translate_civicrm_apiWrappers and intervene - retrieving the values relevant to the specific language, if appropriate. This is a pretty tough intervention to do without any core support, and fairly simple with a small change to core + an agreement as to how we prefer to use the api going forwards
https://github.com/eileenmcnaughton/civi-data-translate/blob/master/civi_data_translate.php#L187
Next steps - I would look to convert the EntityFormTrait to use apiv4 to create a generic template / model for how this should be done. (I would need to set api version as a parameter on the form as I realise extensions may use this trait for entities with no v4 api as yet.)
@bgm @colemanw @seamuslee @tottenhttps://lab.civicrm.org/dev/translation/-/issues/42Document guidelines for situations where trivial technical changes do more ha...2020-04-02T13:28:24ZDaveDDocument guidelines for situations where trivial technical changes do more harm than goodMoved to gitlab from https://github.com/civicrm/civicrm-dev-docs/pull/754
The [end-to-end process](https://lab.civicrm.org/dev/translation/-/wikis/Pushing-new-strings-to-transifex) from the time you make a change in the code to when the...Moved to gitlab from https://github.com/civicrm/civicrm-dev-docs/pull/754
The [end-to-end process](https://lab.civicrm.org/dev/translation/-/wikis/Pushing-new-strings-to-transifex) from the time you make a change in the code to when the translation of it ends up in the tarball of a release is lengthy, depends on human translators' time, and currently includes a [manual tedious step](https://lab.civicrm.org/infra/ops/-/issues/931) for administrators (which means it doesn't even get to transifex to be made available to translators for a few months). In the meantime the string you just changed has now potentially become untranslated immediately in all languages in both master and likely still untranslated when it becomes a release candidate. And possibly for several releases/months after.
So if the wording is not preventing users from completing tasks and the technical functionality is working, then is having it be EXACTLY right worth invalidating an existing translation and generating another end-to-end cycle of the translation process *if that is the only change being made*. It seems like it is possible to write out some guidelines and examples.
The PR proposed these as a starting point:
#### Don't change
```smarty
{ts 1="https://www.example.org"}This is a block of text where the wording is fine and the link works but the <a href="%1" target="_blank">link attributes</a> should be a parameter to the ts. But it's not preventing anyone completing their work tasks and isn't confusing to users.{/ts}
```
#### Do change
* If it would normally break translation but the new string is already used somewhere else, and so it's already translated.
* Spelling mistakes. While not preventing a user from completing a task, the positives outweigh the negatives, and this is visible to the user, as compared to the above "don't change" example where the user doesn't see a problem.https://lab.civicrm.org/dev/translation/-/issues/41Invoice for recurring contribution should be sent in the preferred language o...2022-11-25T16:01:21ZbgmInvoice for recurring contribution should be sent in the preferred language of the contactThe contact record has a "preferred language" field, which can be automatically set when people fill in Contribution or Event forms.
That field can be used in certain circumstances, such as in Mailings and in Scheduled Reminders.
Howev...The contact record has a "preferred language" field, which can be automatically set when people fill in Contribution or Event forms.
That field can be used in certain circumstances, such as in Mailings and in Scheduled Reminders.
However, it is not used in the following circumstances:
* In Administration > CiviContribute > Component settings, enable "tax and invoicing", and enable PDF invoices.
* In Administration > Localization > Languages, enable multilingual, and enable a second locale.
* Setup a form that supports recurring contributions, enable email receipts
* As an anonymous user, donate on that form in the second language (not the default global language), selecting a recurring (ex: daily) donation.
Double-check that the contact record that was created has the second-language. Change if it necessary (I forget where the setting is, for defining the pref. language).
Then, as time passes, notice that the recurring email receipts are in the default site language. This is because the cron or IPNs will usually run in the default site language.
`CRM_Core_BAO_ActionSchedule::setCommunicationLanguage()` would be a good implementation reference. We should probably move it to `CRM_Core_I18n` for clarity.https://lab.civicrm.org/dev/translation/-/issues/7Update Localizing Documentation (Drupal configurations)2019-01-14T15:52:50ZshaneonabikeUpdate Localizing Documentation (Drupal configurations)There is some special mapping configurations for multilingual Drupal sites to map the proper languages between Drupal and CiviCRM. Is there a way I could update the existing documentation?
https://docs.civicrm.org/user/en/latest/the-civ...There is some special mapping configurations for multilingual Drupal sites to map the proper languages between Drupal and CiviCRM. Is there a way I could update the existing documentation?
https://docs.civicrm.org/user/en/latest/the-civicrm-community/localising-civicrm/
`"Inherit CMS language" and regional translations (ex: fr_CA)
If you have a multi-lingual site and you are using the "inherit CMS language" configuration option, but wish to, for example, use fr_CA instead of the default fr_FR (for French), you can define a constant in your "civicrm.settings.php" to override the default behavior.
See CRM-9558 for more information.
define('CIVICRM_LANGUAGE_MAPPING_FR', 'fr_CA');
`
From https://wiki.civicrm.org/confluence/display/CRMDOC/i18n+Administrator%27s+Guide%3A+Using+CiviCRM+in+your+own+languagehttps://lab.civicrm.org/dev/translation/-/issues/1Distribution and management of l10n/xx_YY/settings.default.json files2020-08-11T11:18:57ZbgmDistribution and management of l10n/xx_YY/settings.default.json filesNow that CRM-16395 has been merged, we can provide localised setting files for supported languages.
Now we need to:
* [x] decide where to store (in the 'l10n' git repo) the settings files
* [x] bundle those json files in civicrm-5.x.xx...Now that CRM-16395 has been merged, we can provide localised setting files for supported languages.
Now we need to:
* [x] decide where to store (in the 'l10n' git repo) the settings files
* [x] bundle those json files in civicrm-5.x.xx-l10n.tar.gz
* [x] document the process by which people can send/update setting files
* [ ] update the documentation about non-English installation?
Example:
https://github.com/civicrm/civicrm-core/files/305643/settings.default.json.txt