Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • CiviCRM Core CiviCRM Core
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Graph
    • Compare revisions
  • Issues 1k
    • Issues 1k
    • List
    • Boards
    • Service Desk
    • Milestones
  • Deployments
    • Deployments
    • Releases
  • Packages and registries
    • Packages and registries
    • Model experiments
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • DevelopmentDevelopment
  • CiviCRM CoreCiviCRM Core
  • Issues
  • #2898

URL Tokens - CiviMail BAO and FlexMailer disagree over duplicate `http://`

Overview

CiviMail BAO and Flexmailer provide two mechanisms to rendering/delivering mail-blasts. To phase out BAO-rendering, we should eventually migrate existing sites to Flexmailer. However, small discrepancies between them can be an obstacle. This issue asks how to deal with a discrepancy when handling certain URL tokens.

(The issue is an off-shoot from the review discussion on https://github.com/civicrm/civicrm-core/pull/21522. However, they are somewhat distinct as 21522 applies to new-sites and this problem is more pressing on upgrade-sites.)

Background

Suppose you have an email token which generates a URL (eg {action.forward} or {action.unsubscribeUrl}) . The most correct way to use this token in an HTML-email is to place it in a hyperlink (<a href="{action.forward}>).

However, if you use CKEditor to compose a message, the hyperlink dialog strongly encourages you to put http:// at the front of any hyperlink. Thus, you can organically produce expressions like <a href="http://{action.forward}">.

CkeditorHttpToken

CiviMail BAO has a feature which mitigates this - in effect, both notations give the same output. However, Flexmailer (TokenProcessor) does not have a similar mitigation. Thus:

HTML Email Expression BAO Output Flexmailer Output
<a href="{action.forward}> <a href="https://example.com/civicrm/mailing/forward?... <a href="https://example.com/civicrm/mailing/forward?...
<a href="http://{action.forward}> <a href="https://example.com/civicrm/mailing/forward?... <a href="http://https://example.com/civicrm/mailing/forward?...

Note the duplicate URL scheme (http://http://).

Scenarios: Good

There are several scenarios wherein this discrepancy does not matter:

  • You use CKEditor + BAO-renderer. (The BAO renderer mitigates extraneous http://.)
  • You use Mosaico + Flexmailer. (Mosaico's UI doesn't create extraneous http://.)

(I suspect these two cohorts are the largest.)

Scenarios: Bad

The potential for difficulty arises when mixing CKEditor with Flexmailer -- you take some HTML that was composed with CKEditor, run it through Flexmailer, and now you have invalid URLs with http://https://.

If we flip over more existing sites to Flexmailer, then it is foreseeable that more sites will be in this mixed scenario.

There's a secondary consideration in who may experience problems -- templates. Most users are not in the habit of typing {action.forward} or {action.unsubscribeUrl} regularly. Anecdotally, the typical practice is to put these kinds of tokens into a template, eg

  • (A) Add a "User-Defined Template" (edited w/CKEditor in web; stored in civicrm_msg_template), or...
  • (B) Reuse/copy a previous "Mailing" (edited w/CKEditor in web; stored in civicrm_mailing), or...
  • (C) Configure default header/footer (no CKEditor; stored in civicrm_mailing_component)

Practical recap: today, if you enable Flexmailer on a site that uses CKEditor for mailings, then either:

  • Your mailings and templates will have broken links -- because they use <a href="http://{action.unsubscribeUrl}">.
    • This seems more likely if you use (A)/(B) - because the web UI uses CKEditor.
  • Your mailings and templates will be cleanly interoperable -- because they use <a href="{action.unsubscribeUrl}">.
    • This seems more likely if you use (C) header/footer because the web UI uses textarea.

Approaches

  1. Add a mitigation to Flexmailer or TokenProcessor which removes extraneous schema during the rendering phase.
    • Strength: Best interoperability
    • Criticism: Adds quirky bits to rendering.
    • Variations: Patch Flexmailer vs patch TokenProcessor.
    • Variations: Apply cleanup before rendering (http://{action.*} => {action.*}) or after rendering (http://https?:// => https?://).
  2. Add an upgrade-task to remove extra http:// in front of tokens.
    • Strength: No quirky bits in rendering.
    • Criticism: CKEditor is still there. Can only upgrade core tables. Rewrites history (civicrm_mailing.body_html of past mailings).
  3. Do nothing - Downstream users should manually convert their mailings/templates.
    • Strength: Look, ma, no hands!
    • Criticism: We cannot measure how many users will be impacted by break. For users with a "Reuse/Copy" workflow, they cannot fix historical mailings in UI.
Edited Oct 08, 2021 by totten
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking