Merge pull request #440 from mlutfy/mlutfy-php-l10n

Add Translation section

docs/standards/php.md

@@ -99,6 +99,24 @@ Classes and interfaces in Civi take one of two forms:
 
 Changing these can be quite difficult and can break interfaces consumed by downstream. For more discussion of `CRM_` and `Civi\`, see [The Codebase](/framework/filesystem.md).
 
+## Localization
+
+Any string that will be displayed to the user should be wrapped in `ts()` to translate the string:
+
+```php
+$string = ts("Hello, world!");
+```
+
+Translation strings can also include placeholders for variables:
+
+```php
+$string = ts("Membership for %1 has been updated. The membership End Date is %2.", array(
+  1 => $userDisplayName,
+  2 => $endDate,
+));
+```
+
+For more information on translation, see [Translation for Developers](/translation/index.md).
 
 ## Scope
 

docs/translation/database.md

+# Database localized fields and upgrades
+
+## PHP code related to database upgrades
+
+We try to reduce the number of strings that will be practically never seen by administrators. Very unlikely error messages should be left untranslated.
+
+For example, in `CRM_Upgrade_Incremental_php_FourSeven`, `addTask()` task names such as "Upgrade DB to ..." should be translated:
+
+```php
+$this->addTask(
+  ts('Upgrade DB to %1: SQL', array(1 => '4.3.5')),
+  'task_4_3_x_runSql',
+  $rev
+);
+```
+
+Very specific one-time tasks should not be translated (wrapped in `ts`). Administrators are very unlikely to see such strings. If they do, they will probably need the original English string in order to get support on the forums. They are also strings that are very hard to translate because of lack of context.
+
+For example, do not translate:
+
+```php
+$this->addTask(
+  'Update financial_account_id in financial_trxn table',
+  'updateFinancialTrxnData',
+  $rev
+);
+```
+
+## Localized fields
+
+Any value stored in the database that may depend on the language must be localizable. For example:
+
+* A contribution page title or description,
+* A group title or description,
+* A configurable string to display on forms (ex: a custom "submit" button label).
+
+However, since localizable fields add a certain technical complexity, the following type of fields are not localized:
+
+* Contact information, such as the first name, nickname, etc.
+* Address fields.
+
+While there are many cities where street names can officially be in multiple languages (or have official transliterations), users usually enter their address only in one language. It is rarely required to store the address translation (one exception: event locations, which is currently a known limitation).
+
+Similarly, the first and last name of an individual may be written in different alphabets (ex: Latin and Cyrillic), but this is not a frequent use-case worth the complexity. Administrators can workaround this by creating custom fields.
+
+In order to define a field as localizable, the [XML schema definition](/framework/database/schema-definition.md) for that field must have the following tag:
+
+```
+<localizable>true</localizable>
+```
+
+If a new field was not initially tagged as localizable, the upgrade must explicitly convert the field. See the section below on localised fields schema changes.
+
+## SQL upgrades
+
+SQL upgrades must account for two use-cases:
+
+* localized CiviCRM: the values in the database are in one language only, so no new database fields are created, however
+* multi-lingual CiviCRM: a typical `value` field will be removed, and replaced with `value_en_US`, `value_fr_FR`, and so on.
+
+There are two variables exposed to the sql templates when upgrading: `$multilingual` makes it possible to test if the database is multi-lingual, while `$locales` lists the enabled languages. For example:
+
+```smarty
+{if $multilingual}
+  {foreach from=$locales item=locale}
+    UPDATE civicrm_option_group
+    SET label_{$locale} = description_{$locale}
+    WHERE label_{$locale} IS NULL;
+  {/foreach}
+{else}
+  UPDATE civicrm_option_group
+  SET `label` = `description`
+  WHERE `label` IS NULL;
+{/if}
+```
+
+However the `{localize}` helper for SQL upgrades (e.g. statements in `CRM/Upgrade/Incremental/sql/*.mysql.tpl` files)  allows you do the same thing without explicitly looping on locales. This `UPDATE` statement handles both multi-lingual and non-multi-lingual cases.
+
+```smarty
+UPDATE `civicrm_premiums`
+SET
+  {localize field="premiums_nothankyou_label"}
+    premiums_nothankyou_label = '{ts escape="sql"}No thank-you{/ts}'
+  {/localize};
+```
+
+On a multi-lingual site with English and French enabled, this would evaluate to:
+
+```sql
+UPDATE `civicrm_premiums`
+SET
+  premiums_nothankyou_label_en_US = 'No thank-you',
+  premiums_nothankyou_label_fr_FR = 'Non merci';
+```
+
+The `{ts}` tag translates the string based on the default language that is set _when the upgrade is being run_. In the above example if the upgrade was run while the default language was French, that column would be set to "Merci non". It would be good to fix this so that the values for each enabled language are translated when a translated string is available.
+
+For an `INSERT` example, the following query:
+
+```smarty
+INSERT INTO civicrm_option_value (
+  option_group_id,
+  {localize field='label'}label{/localize},
+  value,
+  name,
+  filter,
+  weight,
+  is_active )
+VALUES (
+  @option_group_id_ere,
+  {localize}'{ts escape="sql"}Participant Role{/ts}'{/localize},
+  1,
+  'participant_role',
+  0,
+  1,
+  1 );
+```
+
+On a multi-lingual site with English and French enabled, the above would evaluate to:
+
+```sql
+INSERT INTO civicrm_option_value (
+  option_group_id,
+  label_en_US,
+  label_fr_FR,
+  value,
+  name,
+  filter,
+  weight,
+  is_active )
+VALUES (
+  @option_group_id_ere,
+  'Participant Role',
+  'Rôle du participant',
+  1,
+  'participant_role',
+  0,
+  1,
+  1 );
+```
+
+## Localised fields schema changes
+
+Two use-cases:
+
+1. An existing field in CiviCRM was not tagged in the [XML schema](/framework/database/schema-definition.md) as `<localizable>` (ex: the `title` in `civicrm_survey`, before CiviCRM 4.5). After adding the `<localize>` tag in the XML file, you must also add an upgrade snippet for existing databases. Example, from `sql/4.1.0.mysql.tpl`:
+
+    ```smarty
+    {if $multilingual}
+      {foreach from=$locales item=locale}
+          ALTER TABLE civicrm_pcp_block ADD link_text_{$locale} varchar(255);
+          UPDATE civicrm_pcp_block SET link_text_{$locale} = link_text;
+      {/foreach}
+      ALTER TABLE civicrm_pcp_block DROP link_text;
+    {/if}
+    ```
+
+2. If a localized field was removed or added, the schema does odd things during the upgrade to figure out which fields are mutli-lingual. Rebuilding the multi-lingual schema will check in `CRM/Core/I18n/SchemaStructure.php` for the fields used by the database views. If the schema is changed, copy the `SchemaStructure.php` from the master branch to, for example, `SchemaStructure_4_5_alpha1.php`. The 4.5 alpha1 will then read this file when rebuilding the schema, see `CRM/Core/I18n/Schema.php` for more information (`getLatestSchema`). i.e. during an upgrade, we may be upgrading from 4.0 to 4.5, and when rebuilding the views at each stage, we need to load the correct schema version. Since we do not have a schema file for each minor version, CiviCRM will attempt to load the most relevant schema version to the version of the upgrade step being run.

docs/translation/extensions.md

+# Extensions Translation
+
+For developing a CiviCRM extension in a way that can be translated, all the best practices described in the [Internationalisation for Developers](/translation/index.md) page apply. This page describes special considerations that need to be taken into account for extensions.
+
+## For translators: Translating strings on Transifex
+
+There is a separate project on Transifex to translate extensions. Each extension has its own "resource". Therefore, when a translator joins a translation team, they can translate all extensions. We didn't see a need to separate each extension in a separate project, because each extension should have only one translation (`.po`) file.
+
+See: <https://www.transifex.com/projects/p/civicrm_extensions/>
+
+## For administrators: Download translation files for extensions
+
+The easiest way to download translations for extensions is to use the [l10nupdate](https://github.com/cividesk/com.cividesk.l10n.update/) extension.
+
+## For developers: Correct usage of the `ts()` function
+
+In PHP, Smarty, and JS code, the convention is to perform translations using the `ts()` helper function. This is the same as in core code &mdash; with the additional requirement that one must specify the "domain" so that the translation engine can use the correct dictionary (`.mo` file) at run-time.
+
+PHP:
+
+```php
+$string = ts('Hello, %1', array(
+  1 => $display_name,
+  'domain' => 'org.example.myextension',
+));
+```
+ 
+Smarty templates:
+
+```smarty
+{crmScope extensionKey='org.example.myextension'}
+  <p>{ts 1=$display_name}Hello, %1{/ts}</p>
+{/crmScope}
+```
+
+Javascript:
+
+```js
+(function ($, ts){
+  $('.foo').click(function greet(display_name) {
+    window.alert(ts('Hello, %1', {1: display_name}));
+  });
+}(CRM.$, CRM.ts('org.example.myextension')));
+```
+
+Angular:
+
+```js
+$scope.ts = CRM.ts('org.example.myextension');
+```
+
+Angular HTML templates:
+
+```html
+<p>{{ts('Hello, %1', {1: display_name})}}</p>
+```

docs/translation/index.md

+# Translation for Developers
+
+When writing new application code, developers should organize their code in a way which is amenable to internationalization, so that it may be localized to various languages and regions of the world.
+
+If you are an extension developer, there is additional documentation in the [Extension translation](/translation/extensions.md) page.
+
+## PHP
+
+* The strings hard-coded into PHP should be wrapped in `ts()` function calls. For example:
+
+    ```php
+    $string = ts('Hello, World!');
+    
+    $group = array('' => ts('- any group -')) + $this->_group;
+    ```
+
+* You can also use placeholders for variables:
+
+    ```php
+    $string = ts("A new '%1' has been created.", array(1 => $contactType));
+    ```
+
+    Note that variables should themselves be translated by your code before passing in, if appropriate.
+
+* If the string might be singular or plural, use the following syntax:
+
+    ```php
+    $string = ts('%count item created',
+      array('count' => $total, 'plural' => '%count items created')
+    );
+    ```
+
+## Javascript
+
+When translating strings in an extension, ts scope needs to be declared. The `CRM.ts` function takes scope as an argument and returns a function that always applies that scope to ts calls:
+
+```js
+// This closure gets a local copy of jQuery, Lo-dash, and ts
+(function($, _, ts) {
+  CRM.alert(ts('Your foo has been barred.'));
+})(CRM.$, CRM._, CRM.ts('foo.bar.myextension'));
+```
+
+!!! note
+    `CRM.ts` is not the same as the global `ts` function. `CRM.ts` is a function that returns a function (javascript is wacky like that). Since your closure gives the local `ts` the same name as the global `ts`, it will be used instead.
+
+!!! important
+    Your local version of `ts` could be named anything, but strings in your javascript file cannot be accurately parsed unless you name it `ts`.
+
+## Smarty templates
+
+* The strings hard-coded into templates should be wrapped in `{ts}...{/ts}` tags. For example:
+
+    ```smarty
+    {ts}Full or partial name (last name, or first name, or organization name).{/ts}
+    ```
+
+* If you need to pass a variable to the localizable string, you should use the following pattern:
+
+    ```smarty
+    <div class="status">
+      {ts 1=$delName}Are you sure you want to delete <b>%1</b> Tag?{/ts}
+    </div>
+    ```
+
+## Best practices
+
+The general rules for avoiding errors may be summed up like this:
+
+* If the string needs to be parsed (i.e. is in double quotes) then there's probably an error there.
+* No string concatenation in the `ts()` calls.
+* The second parameter of the `ts()` call must be an array.
+* You must pass a literal string into `ts()`, not a variable.
+
+### Avoid variables inside strings
+
+!!! failure "Bad"
+    
+    ```php
+    $string = ts("The date type '$name' has been saved.");
+    ```
+
+!!! success "Good"
+
+    ```php
+    $string = ts("The date type '%1' has been saved.", array(1 => $name));
+    ```
+
+### Avoid tags inside strings 
+
+!!! failure "Bad"
+    
+    ```smarty
+    {ts}<p>Hello, world!</p>{/ts}
+    ```
+
+!!! success "Good"
+
+    ```smarty
+    <p>{ts}Hello, world!{/ts}</p>
+    ```
+    
+### Avoid multi-line strings
+
+Even if your code editor may not like it, long strings should be on a single line since a change in indentation might change where the line breaks are, which would then require re-translating the string.
+
+!!! failure "Bad"
+    
+    ```php
+    $string = ts("Lorem ipsum dolor sit amet, consectetur adipiscing elit.
+      Proin elementum, ex in pretium tincidunt, felis lorem facilisis 
+      lacus, vel iaculis ex orci vitae risus. Maecenas in sapien ut velit
+      scelerisque interdum.");
+    ```
+
+!!! success "Good"
+
+    ```php
+    $string = ts("Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin elementum, ex in pretium tincidunt, felis lorem facilisis lacus, vel iaculis ex orci vitae risus. Maecenas in sapien ut velit scelerisque interdum.");
+    ```
+
+### Avoid strings which begin or end with spaces
+
+!!! failure "Bad"
+    
+    ```php
+    $string = $labelFormat['label'] . ts(' has been created.'),
+    ```
+
+!!! success "Good"
+
+    ```php
+    $string = ts('%1 has been created.', array(1 => $labelFormat['label'])),
+    ```
+ 
+### Avoid escaped quotes
+
+!!! failure "Bad"
+    
+    ```php
+    $string = ts('A new \'%1\' has been created.', array(1 => $contactType));
+    ```
+
+!!! success "Good"
+
+    ```php
+    $string = ts("A new '%1' has been created.", array(1 => $contactType));
+    ```
+    
+### Use separate strings for plural items
+
+!!! failure "Bad"
+    
+    ```php
+    $string = ts('%1 item(s) created', array(1 => $count));
+    ```
+
+!!! success "Good"
+
+    ```php
+    $string = ts('%count item created',
+      array('count' => $total, 'plural' => '%count items created')
+    );
+    ```
+    
+### Ensure that strings have *some* words in them
+
+Another common error is to use `ts()` to aggregate strings or as a "clever" way of writing shorter code:
+
+!!! failure "Bad"
+
+    Incorrect aggregation. This will be extremely confusing to translations and might give some really bad results in some languages.
+    
+    ```php
+    $operation = empty($params['id']) ? ts('New') : ts('Edit'));
+    $string = ts("%1 %2", array(1 => $operation, 2 => $contactType));
+    ```
+
+!!! success "Less bad"
+
+    ```php
+    if (empty($params['id'])) {
+      $string = ts("New %1", array(1 => $contactType));
+    }
+    else {
+      $string = ts("Edit %1", array(1 => $contactType));
+    }
+    ```
+    
+    Note that this still makes it difficult to use the correct gender.
+
+### Include typography in strings
+
+Typography is different in different languages and thus must be translated along with the string. For example, in French, there must be a space before a colon. 
+
+!!! failure "Bad"
+    
+    ```smarty
+    {ts}Event Total{/ts}:
+    ```
+
+!!! success "Good"
+
+    ```smarty
+    {ts}Event Total:{/ts}
+    ```
+
+## Rationale for using Gettext
+
+In most projects, strings are typically translated by either:
+
+* using Gettext (which is what CiviCRM does),
+* using arrays of key/string dictionaries,
+* using database lookups of strings (which is what Drupal does).
+
+In order to be support Joomla!, WordPress, Backdrop and eventually other content management systems. Gettext is the standard way to translate strings in PHP, used by most projects.
+
+## Other guides/references
+
+Here are the guides to other popular projects:
+
+* Drupal: <https://www.drupal.org/node/322729>
+* Joomla!: <https://docs.joomla.org/Specification_of_language_files>
+* WordPress: <https://codex.wordpress.org/I18n_for_WordPress_Developers>

mkdocs.yml

@@ -239,6 +239,10 @@ pages:
     - Extending Smarty: framework/templates/extending-smarty.md
   - UI Reference: framework/ui.md
   - Upgrade Reference: framework/upgrade.md
+- Translation:
+  - Translation: translation/index.md
+  - Extensions Translation: translation/extensions.md
+  - Database localized fields and upgrades: translation/database.md
 - Standards:
   - Coding Standards: standards/index.md
   - PHP Standards: standards/php.md

redirects/wiki-crmdoc.txt

@@ -197,3 +197,4 @@ Permission+Reference security/permissions
 Develop index.html
 Customized+and+Custom+Tokens framework/civimail#extending-the-token-system
 Tokens framework/civimail/tokens
+Internationalisation+for+Developers translation