Skip to content

System Workflow Messages - Improve localization experience

Background

Several CiviCRM components - such as CiviContribute, CiviCase, and CiviEvent - send automated email notifications. Many organizations find it useful to customize/tune these emails, and the emails may be administered in the web UI ("Administer: Message Templates: System Workflow Messages").

In principle, this is a powerful tool in which an administrator may use Smarty templating to produce highly tuned messages -- and it requires only the ability to understand template notations (e.g. {$contact.first_name} or {$event.start_date}). However, in practice, this is often difficult -- real-world templates are long and complex, and they may be used with data of varying quality/character, and the workflow to test a template can be tedious.

Goals

This epic (overarching issue) aims to improve the experience of maintaining system workflow templates. We are specifically funded to work on it from the following perspective: You have an organization which maintains multiple templates for use in multiple countries/locales -- periodically, as campaigns/operations are revised, you may need to engage with a few people to update+retest a few templates. We have three main themes to improve upon:

  • Drafting/Workflow: Currently, Civi has exactly one variant/revision of a template, and it is always live. There is no way to develop a draft (new revisions) or to keep a record of past revisions.
  • Translation Localization: For any given message, you may need to compose different versions for different locales.
  • Previews/Samples: Many templates are sensitive to fine-grained details. This is true in multiple ways, e.g. (1) the template adjusts based on available data ("Do we have a mailing address for this person? Do we have a full-name, or just an email? Is this one-off contribution or a recurring contribution?"), and (2) the template includes precise codes that must be well-formed (e.g. matching tags, matching quotes, using an email-friendly dialect of HTML/CSS).

Related work

Subtopics / Brainstorming

(Some of the descriptions here are terse - but they may get more explanation in the wireframe.)

  • Sample data presentation: While composing a message, how do you show a preview based on sample data? Some ideas:

    • Edit-Preview Split-Panes
      • Pro: very amenable to live-update
      • Con: needs wide screen (prob dialog). agreeable with O(10) sample records - but disagreeable with O(1000) records
    • Edit-Preview Accordion
      • Pro: fairly amenable to live-update. full width.
      • Con: agreeable with O(10) sample records - but disagreeable with O(1000) records
    • Explore-Preview Panes in dialog
      • Pro: agreeable with O(1000) records
      • Con: requires more navigating.
      • Comment: live-update might be possible with separate popup - but more difficult
      • Comment: might mitigate navigation work with a hotkey

  • Sample data source: When viewing a preview based on some example record(s), where does the example come from?

    • Basic/prepackaged: Define some JSON files, which are linked to the workflows.
      • Comment: Requires metadata to say, "workflows a+b can use sample data c+d"
      • Pro: Some templates have very nuanced purposes and may have special vars. This allows you to mock data that's attuned to edge-cases, and you can use these samples in any environment (dev-site/public-demo-site/live-site). No mixing-up real records and sample records. Can simulate complex data (e.g. "tpl for use-case with 2 contribution records")
      • Con: More upfront work in making sample data. User cannot test against real data.
      • Comment: Should have unit-test to ensure sample-data-structure stays in sync with real-data-structure
    • Data search: Search for a live record (eg Contribution or Membership) to use as a sample.
      • Comment: Requires metadata to say, "workflows a+b need data from entity $c"
      • Pro: You can use real data, which can be subjectively attuned.
      • Con: You have to find real data that's useful. May be awkward if you're trying to test edge-cases, unless you're willing to add mock/synthetic records to the DB. All mail-merge data must be selectable from one record (eg "Contribution #123"; eg no other inputs besides DB content).
      • Variants: The UI for this can take on a few variations, eg
        • In "Edit Msg Tpl", show a searchable tree. Filter by contact name and select the target record.
        • In "Edit Msg Tpl", ask the user to enter a numeric ID.
        • In "View Contact", add links/actions for generating previews.
    • Flagged records: As an adminsitrator, find real records and save them in a list (e.g. "Record #$x is a sample for use in workflows $a, $b")
      • Comment: Requires somewhere to store the list (e.g. custom-field or new-field or new-table)
      • Pro: You can use real data, which can be subjectively attuned. The list of samples is curated/abbreviated.
      • Con: The individual composing the template may not know how to flag/unflag sample records. Requires configuration (and extant samples) before you can use email preview.

  • Grain of translation localization

    • Record level: For each (locale,workflow), create another civicrm_msg_template record.

    • Field level: Each (workflow) only has one civicrm_msg_template record. Attached to that record are multiple translations (ex: field=subject,locale=fr_FR,text="Bonjour").

    • String level: Each (workflow) only has one civicrm_msg_template record, and that record has one body_text. However, the body_text uses translatable strings (ex: 'body_text={cts id=greeting}Greetings,{/cts}`)

    • Trade-off

      Record-level Field-level String-level
      Each translation goes through a separate, well-delineated drafting workflow. Translation workflow is batched (per template). All translations must be ready to activate together. Orthogonal workflows for templates vs strings.
      Message structure (paras/sections) is localized. Message structure (paras/sections) is localized. Message structure is uniform across locales. (Realistic? What abouts <table> LTR/RTL?
      Cannot re-use translated snippets. Cannot re-use translated snippets. Re-use translated snippets.

Wireframes

There are a few different ways to approach this. To get a sense for the usability and work, we can post a few alternative wireframes. (This section may be updated as more are added.)

  • (0) Status quo: Listing Screen, Edit Screen

  • (1) Record-level translations w/previews and drafting workflow: Listing Screen, Edit Screen (Note: The blue-arcs indicate a couple variations on how to include the preview/sample functionality. This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.)

  • (2) Field-level translations: Listing Screen, Edit Screen (Note: This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.)

  • (3) String-level translations: Listing Screen, Edit Screen (Note: This tracks close to status-quo, and incremental changes are highlighted with yellow-dots.)

  • (4) Record-level translations w/browse-edit tree + accordion-preview: Browse-Edit Screen

  • (5) Field-level translations w/browse-edit tree + accordion-preview Browse-Edit Screen

  • (6) Field-level translations w/matrix, preview-dialog, and drafting workflow: Listing Screen, Edit Screen (Note: This assumes that there is a separate string-storage service like civi-data-translate, and it assumes that the String record has some workflow property like like is_draft or revision. So each string might have the properties entity,entity_id,field,locale,text,is_draft.)

  • (7) As with 6, but with various refinements: Listing screen, Edit screen

Edited by totten
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information