Commit 547a6808 authored by bgm's avatar bgm Committed by Aegir user
Browse files

Remove flexmailer, it's in the platform

parent 6e31c513
# Change Log
## v0.2-alpha1
* Override core's `Mailing.preview` API to support rendering via
Flexmailer events.
* (BC Break) In the class `DefaultComposer`, change the signature for
`createMessageTemplates()` and `applyClickTracking()` to provide full
access to the event context (`$e`).
This diff is collapsed.
# org.civicrm.flexmailer
FlexMailer (`org.civicrm.flexmailer`) is an email delivery engine for CiviCRM v4.7+. It replaces the internal guts of CiviMail. It is a
drop-in replacement which enables *other* extensions to provide richer email features.
* [Introduction](docs/index.md)
* [Installation](docs/install.md)
* [Development](docs/develop/index.md)
* [CheckSendableEvent](docs/develop/CheckSendableEvent.md)
* [WalkBatchesEvent](docs/develop/WalkBatchesEvent.md)
* [ComposeBatchEvent](docs/develop/ComposeBatchEvent.md)
* [SendBatchEvent](docs/develop/SendBatchEvent.md)
The `CheckSendableEvent` (`EVENT_CHECK_SENDABLE`) determines whether a draft mailing is fully specified for delivery.
For example, some jurisdictions require that email blasts provide contact
information for the organization (eg street address) and an opt-out link.
By default, the check-sendable event will verify that this information is
provided through a mail-merge token (eg `{action.unsubscribeUrl}`).
The token validation logic depends on how the message has been encoded. If
you provide a new template language, you can implement new enforcement logic,
e.g.
```php
<?php
function mustache_civicrm_container($container) {
$container->addResource(new \Symfony\Component\Config\Resource\FileResource(__FILE__));
$container->findDefinition('dispatcher')->addMethodCall('addListener',
array(\Civi\FlexMailer\Validator::EVENT_CHECK_SENDABLE, '_mustache_check_sendable')
);
}
function _mustache_check_sendable(\Civi\FlexMailer\Event\CheckSendableEvent $e) {
if ($e->getMailing()->template_type !== 'mustache') return;
if (strpos('{{unsubscribeUrl}}', $e->getMailing()->body_html) === FALSE) {
$e->setError('body_html:unsubscribeUrl', E::ts('Please include the token {{unsubscribeUrl}}'));
}
}
```
The `ComposeBatchEvent` builds the email messages. Each message is represented as a `FlexMailerTask` with a list of `MailParams`.
Some listeners are "under the hood" -- they define less visible parts of the message, e.g.
* `BasicHeaders` defines `Message-Id`, `Precedence`, `From`, `Reply-To`, and others.
* `BounceTracker` defines various headers for bounce-tracking.
* `OpenTracker` appends an HTML tracking code to any HTML messages.
The heavy-lifting of composing the message content is also handled by a listener, such as
`DefaultComposer`. `DefaultComposer` replicates traditional CiviMail functionality:
* Reads email content from `$mailing->body_text` and `$mailing->body_html`.
* Interprets tokens like `{contact.display_name}` and `{mailing.viewUrl}`.
* Loads data in batches.
* Post-processes the message with Smarty (if `CIVICRM_SMARTY` is enabled).
The traditional CiviMail semantics have some problems -- e.g. the Smarty post-processing is incompatible with Smarty's
template cache, and it is difficult to securely post-process the message with Smarty. However, changing the behavior
would break existing templates.
A major goal of FlexMailer is to facilitate a migration toward different template semantics. For example, an
extension might (naively) implement support for Mustache templates using:
```php
<?php
function mustache_civicrm_container($container) {
$container->addResource(new \Symfony\Component\Config\Resource\FileResource(__FILE__));
$container->findDefinition('dispatcher')->addMethodCall('addListener',
array(\Civi\FlexMailer\FlexMailer::EVENT_COMPOSE, '_mustache_compose_batch')
);
}
function _mustache_compose_batch(\Civi\FlexMailer\Event\ComposeBatchEvent $event) {
if ($event->getMailing()->template_type !== 'mustache') return;
$m = new Mustache_Engine();
foreach ($event->getTasks() as $task) {
if ($task->hasContent()) continue;
$contact = civicrm_api3('Contact', 'getsingle', array(
'id' => $task->getContactId(),
));
$task->setMailParam('text', $m->render($event->getMailing()->body_text, $contact));
$task->setMailParam('html', $m->render($event->getMailing()->body_html, $contact));
}
}
```
This implementation is naive in a few ways -- it performs separate SQL queries for each recipient; it doesn't optimize
the template compilation; it has a very limited range of tokens; and it doesn't handle click-through tracking. For
more ideas about these issues, review `DefaultComposer`.
> FIXME: Core's `TokenProcessor` is useful for batch-loading token data.
> However, you currently have to use `addMessage()` and `render()` to kick it
> off -- but those are based on CiviMail template notation. We should provide
> another function that doesn't depend on the template notation -- so that
> other templates can leverage our token library.
> **Tip**: When you register a listener for `EVENT_COMPOSE`, note the weight.
> The default weight puts your listener in the middle of pipeline -- right
> before the `DefaultComposer`. However, you might want to position
> relative to other places -- e.g. `WEIGHT_PREPARE`, `WEIGHT_MAIN`,
> `WEIGHT_ALTER`, or `WEIGHT_END`.
The `SendBatchEvent` (`EVENT_SEND`) takes a batch of recipients and messages, and it delivers the messages. For example, suppose you wanted to
replace the built-in delivery mechanism with a batch-oriented web-service:
```php
<?php
function example_civicrm_container($container) {
$container->addResource(new \Symfony\Component\Config\Resource\FileResource(__FILE__));
$container->findDefinition('dispatcher')->addMethodCall('addListener',
array(\Civi\FlexMailer\FlexMailer::EVENT_SEND, '_example_send_batch')
);
}
function _example_send_batch(\Civi\FlexMailer\Event\SendBatchEvent $event) {
$event->stopPropagation(); // Disable standard delivery
$context = stream_context_create(array(
'http' => array(
'method' => 'POST',
'header' => 'Content-type: application/vnd.php.serialize',
'content' => serialize($event->getTasks()),
),
));
return file_get_contents('https://example.org/batch-delivery', false, $context);
}
```
The `WalkBatchesEvent` examines the recipient list and pulls out a subset for whom you want to send email. This is useful if you need strategies for
chunking-out deliveries.
The basic formula for defining your own batch logic is:
```php
<?php
function example_civicrm_container($container) {
$container->addResource(new \Symfony\Component\Config\Resource\FileResource(__FILE__));
$container->findDefinition('dispatcher')->addMethodCall('addListener',
array(\Civi\FlexMailer\FlexMailer::EVENT_WALK, '_example_walk_batches')
);
}
function _example_walk_batches(\Civi\FlexMailer\Event\WalkBatchesEvent $event) {
$event->stopPropagation(); // Disable standard delivery
while (...) {
$tasks = array();
$task[] = new FlexMailerTask(...);
$task[] = new FlexMailerTask(...);
$task[] = new FlexMailerTask(...);
$event->visit($tasks);
}
}
```
## Unit tests
The [headless unit tests](https://docs.civicrm.org/dev/en/latest/testing/#headless) are based on `phpunit4` and `cv`. Simply run:
```
$ phpunit4
```
## Events
!!! tip "Symfony Events"
This documentation references the [Symfony EventDispatcher](http://symfony.com/components/EventDispatcher).
If this is unfamiliar, you can read [a general introduction to Symfony events](http://symfony.com/doc/2.7/components/event_dispatcher.html)
or [a specific introduction about CiviCRM and Symfony events](https://docs.civicrm.org/dev/en/latest/hooks/setup/symfony/).
FlexMailer is an *event* based delivery system. It defines a few events:
* [CheckSendableEvent](CheckSendableEvent.md): In this event, one examines a draft mailing to determine if it is complete enough to deliver.
* [WalkBatchesEvent](WalkBatchesEvent.md): In this event, one examines the recipient list and pulls out a subset for whom you want to send email.
* [ComposeBatchEvent](ComposeBatchEvent.md): In this event, one examines the mail content and the list of recipients -- then composes a batch of fully-formed email messages.
* [SendBatchEvent](SendBatchEvent.md): In this event, one takes a batch of fully-formed email messages and delivers the messages.
These events are not conceived in the same way as a typical *CiviCRM hook*; rather, they resemble *pipelines*. For each event, several listeners
have an opportunity to weigh-in, and the *order* of the listeners is important. As such, it helps to *inspect* the list of listeners. You can do
this with the CLI command, `cv`:
```
$ cv debug:event-dispatcher /flexmail/
[Event] civi.flexmailer.checkSendable
+-------+------------------------------------------------------------+
| Order | Callable |
+-------+------------------------------------------------------------+
| #1 | Civi\FlexMailer\Listener\Abdicator->onCheckSendable() |
| #2 | Civi\FlexMailer\Listener\RequiredFields->onCheckSendable() |
| #3 | Civi\FlexMailer\Listener\RequiredTokens->onCheckSendable() |
+-------+------------------------------------------------------------+
[Event] civi.flexmailer.walk
+-------+---------------------------------------------------+
| Order | Callable |
+-------+---------------------------------------------------+
| #1 | Civi\FlexMailer\Listener\DefaultBatcher->onWalk() |
+-------+---------------------------------------------------+
[Event] civi.flexmailer.compose
+-------+-------------------------------------------------------+
| Order | Callable |
+-------+-------------------------------------------------------+
| #1 | Civi\FlexMailer\Listener\BasicHeaders->onCompose() |
| #2 | Civi\FlexMailer\Listener\ToHeader->onCompose() |
| #3 | Civi\FlexMailer\Listener\BounceTracker->onCompose() |
| #4 | Civi\FlexMailer\Listener\DefaultComposer->onCompose() |
| #5 | Civi\FlexMailer\Listener\Attachments->onCompose() |
| #6 | Civi\FlexMailer\Listener\OpenTracker->onCompose() |
| #7 | Civi\FlexMailer\Listener\HookAdapter->onCompose() |
+-------+-------------------------------------------------------+
[Event] civi.flexmailer.send
+-------+--------------------------------------------------+
| Order | Callable |
+-------+--------------------------------------------------+
| #1 | Civi\FlexMailer\Listener\DefaultSender->onSend() |
+-------+--------------------------------------------------+
```
The above listing shows the default set of listeners at time of writing. (Run the command yourself to see how they appear on your system.)
The default listeners behave in basically the same way as CiviMail's traditional BAO-based delivery system (respecting `mailerJobSize`,
`mailThrottleTime`, `mailing_backend`, `hook_civicrm_alterMailParams`, etal).
There are a few tricks for manipulating the pipeline:
* __Register new listeners__. Each event has its own documentation which describes how to do this.
* __Manage the priority__. When registering a listener, the `addListener()` function accepts a `$priority` integer. Use this to move up or down the pipeline.
!!! note "Priority vs Order"
When writing code, you will set the *priority* of a listener. The default is `0`, and the usual range is `2000` (first) to `-2000` (last).
<!-- The default listeners have priorities based on the constants `FlexMailer::WEIGHT_PREPARE` (1000), `FlexMailer::WEIGHT_MAIN` (0),
`FlexMailer::WEIGHT_ALTER` (-1000), and `FlexMailer::WEIGHT_END` (-2000). -->
At runtime, the `EventDispatcher` will take all the listeners and sort them by priority. This produces the *order*, which simply counts up (`1`, `2`, `3`, ...).
* __Alter a listener__. Most listeners are *services*, and you can manipulate options on these services. For example, suppose you wanted to replace the default bounce-tracking mechanism.
Here's a simple way to disable the default `BounceTracker`:
```php
<?php
\Civi::service('civi_flexmailer_bounce_tracker')->setActive(FALSE);
```
Of course, this change needs to be made before the listener runs. You might use a global hook (like `hook_civicrm_config`), or you might
have your own listener which disables `civi_flexmailer_bounce_tracker` and adds its own bounce-tracking.
Most FlexMailer services support `setActive()`, which enables you to completely replace them.
Additionally, some services have their own richer methods. In this example, we modify the list of required tokens:
```php
<?php
$tokens = \Civi::service('civi_flexmailer_required_tokens')
->getRequiredTokens();
unset($tokens['domain.address']);
\Civi::service('civi_flexmailer_required_tokens')
->setRequiredTokens($tokens);
```
## Services
Most features in FlexMailer are implemented by *services*, and you can override or manipulate these features if you understand the corresponding service.
For more detailed information about how to manipulate a service, consult its docblocks.
* Listener services (`CheckSendableEvent`)
* `civi_flexmailer_required_fields` (`RequiredFields.php`): Check for fields like "Subject" and "From".
* `civi_flexmailer_required_tokens` (`RequiredTokens.php`): Check for tokens like `{action.unsubscribeUrl}` (in `traditional` mailings).
* Listener services (`WalkBatchesEvent`)
* `civi_flexmailer_default_batcher` (`DefaultBatcher.php`): Split the recipient list into smaller batches (per CiviMail settings)
* Listener services (`ComposeBatchEvent`)
* `civi_flexmailer_basic_headers` (`BasicHeaders.php`): Add `From:`, `Reply-To:`, etc
* `civi_flexmailer_to_header` (`ToHeader.php`): Add `To:` header
* `civi_flexmailer_bounce_tracker` (`BounceTracker.php`): Add bounce-tracking codes
* `civi_flexmailer_default_composer` (`DefaultComposer.php`): Read the email template and evaluate any tokens (based on CiviMail tokens)
* `civi_flexmailer_attachments` (`Attachments.php`): Add attachments
* `civi_flexmailer_open_tracker` (`OpenTracker.php`): Add open-tracking codes
* `civi_flexmailer_hooks` (`HookAdapter.php`): Backward compatibility with `hook_civicrm_alterMailParams`
* Listener services (`SendBatchEvent`)
* `civi_flexmailer_default_sender` (`DefaultSender.php`): Send the batch using CiviCRM's default delivery service
* Other services
* `civi_flexmailer_html_click_tracker` (`HtmlClickTracker.php`): Add click-tracking codes (for HTML messages)
* `civi_flexmailer_text_click_tracker` (`TextClickTracker.php`): Add click-tracking codes (for plain-text messages)
* `civi_flexmailer_api_overrides` (`Services.php.php`): Alter the `Mailing` APIs
FlexMailer (`org.civicrm.flexmailer`) is an email delivery engine for CiviCRM v4.7+. It replaces the internal guts of CiviMail. It is a
drop-in replacement which enables *other* extensions to provide richer email features.
By default, FlexMailer supports the same user interfaces, delivery algorithms, and use-cases as CiviMail. After activating FlexMailer, an
administrator does not need to take any special actions.
The distinguishing improvement here is under-the-hood: it provides better APIs and events for extension-developers. For example,
other extensions might:
* Change the template language
* Manipulate tracking codes
* Rework the delivery mechanism
* Redefine the batching algorithm
To download the latest alpha or beta version:
```bash
$ cv dl --dev flexmailer
```
To download the latest, bleeding-edge code:
```bash
$ cv dl org.civicrm.flexmailer@https://github.com/civicrm/org.civicrm.flexmailer/archive/master.zip
```
To download the latest, bleeding-edge code from git:
```bash
$ cd $(cv path -x .)
$ git clone https://github.com/civicrm/org.civicrm.flexmailer.git
$ cv en flexmailer
```
<?php
// AUTO-GENERATED FILE -- Civix may overwrite any changes made to this file
/**
* The ExtensionUtil class provides small stubs for accessing resources of this
* extension.
*/
class CRM_Flexmailer_ExtensionUtil {
const SHORT_NAME = "flexmailer";
const LONG_NAME = "org.civicrm.flexmailer";
const CLASS_PREFIX = "CRM_Flexmailer";
/**
* Translate a string using the extension's domain.
*
* If the extension doesn't have a specific translation
* for the string, fallback to the default translations.
*
* @param string $text
* Canonical message text (generally en_US).
* @param array $params
* @return string
* Translated text.
* @see ts
*/
public static function ts($text, $params = array()) {
if (!array_key_exists('domain', $params)) {
$params['domain'] = array(self::LONG_NAME, NULL);
}
return ts($text, $params);
}
/**
* Get the URL of a resource file (in this extension).
*
* @param string|NULL $file
* Ex: NULL.
* Ex: 'css/foo.css'.
* @return string
* Ex: 'http://example.org/sites/default/ext/org.example.foo'.
* Ex: 'http://example.org/sites/default/ext/org.example.foo/css/foo.css'.
*/
public static function url($file = NULL) {
if ($file === NULL) {
return rtrim(CRM_Core_Resources::singleton()->getUrl(self::LONG_NAME), '/');
}
return CRM_Core_Resources::singleton()->getUrl(self::LONG_NAME, $file);
}
/**
* Get the path of a resource file (in this extension).
*
* @param string|NULL $file
* Ex: NULL.
* Ex: 'css/foo.css'.
* @return string
* Ex: '/var/www/example.org/sites/default/ext/org.example.foo'.
* Ex: '/var/www/example.org/sites/default/ext/org.example.foo/css/foo.css'.
*/
public static function path($file = NULL) {
// return CRM_Core_Resources::singleton()->getPath(self::LONG_NAME, $file);
return __DIR__ . ($file === NULL ? '' : (DIRECTORY_SEPARATOR . $file));
}
/**
* Get the name of a class within this extension.
*
* @param string $suffix
* Ex: 'Page_HelloWorld' or 'Page\\HelloWorld'.
* @return string
* Ex: 'CRM_Foo_Page_HelloWorld'.
*/
public static function findClass($suffix) {
return self::CLASS_PREFIX . '_' . str_replace('\\', '_', $suffix);
}
}
use CRM_Flexmailer_ExtensionUtil as E;
/**
* (Delegated) Implements hook_civicrm_config().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_config
*/
function _flexmailer_civix_civicrm_config(&$config = NULL) {
static $configured = FALSE;
if ($configured) {
return;
}
$configured = TRUE;
$template =& CRM_Core_Smarty::singleton();
$extRoot = dirname(__FILE__) . DIRECTORY_SEPARATOR;
$extDir = $extRoot . 'templates';
if (is_array($template->template_dir)) {
array_unshift($template->template_dir, $extDir);
}
else {
$template->template_dir = array($extDir, $template->template_dir);
}
$include_path = $extRoot . PATH_SEPARATOR . get_include_path();
set_include_path($include_path);
}
/**
* (Delegated) Implements hook_civicrm_xmlMenu().
*
* @param $files array(string)
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_xmlMenu
*/
function _flexmailer_civix_civicrm_xmlMenu(&$files) {
foreach (_flexmailer_civix_glob(__DIR__ . '/xml/Menu/*.xml') as $file) {
$files[] = $file;
}
}
/**
* Implements hook_civicrm_install().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_install
*/
function _flexmailer_civix_civicrm_install() {
_flexmailer_civix_civicrm_config();
if ($upgrader = _flexmailer_civix_upgrader()) {
$upgrader->onInstall();
}
}
/**
* Implements hook_civicrm_postInstall().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_postInstall
*/
function _flexmailer_civix_civicrm_postInstall() {
_flexmailer_civix_civicrm_config();
if ($upgrader = _flexmailer_civix_upgrader()) {
if (is_callable(array($upgrader, 'onPostInstall'))) {
$upgrader->onPostInstall();
}
}
}
/**
* Implements hook_civicrm_uninstall().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_uninstall
*/
function _flexmailer_civix_civicrm_uninstall() {
_flexmailer_civix_civicrm_config();
if ($upgrader = _flexmailer_civix_upgrader()) {
$upgrader->onUninstall();
}
}
/**
* (Delegated) Implements hook_civicrm_enable().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_enable
*/
function _flexmailer_civix_civicrm_enable() {
_flexmailer_civix_civicrm_config();
if ($upgrader = _flexmailer_civix_upgrader()) {
if (is_callable(array($upgrader, 'onEnable'))) {
$upgrader->onEnable();
}
}
}
/**
* (Delegated) Implements hook_civicrm_disable().
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_disable
* @return mixed
*/
function _flexmailer_civix_civicrm_disable() {
_flexmailer_civix_civicrm_config();
if ($upgrader = _flexmailer_civix_upgrader()) {
if (is_callable(array($upgrader, 'onDisable'))) {
$upgrader->onDisable();
}
}
}
/**
* (Delegated) Implements hook_civicrm_upgrade().
*
* @param $op string, the type of operation being performed; 'check' or 'enqueue'
* @param $queue CRM_Queue_Queue, (for 'enqueue') the modifiable list of pending up upgrade tasks
*
* @return mixed based on op. for 'check', returns array(boolean) (TRUE if upgrades are pending)
* for 'enqueue', returns void
*
* @link http://wiki.civicrm.org/confluence/display/CRMDOC/hook_civicrm_upgrade
*/
function _flexmailer_civix_civicrm_upgrade($op, CRM_Queue_Queue $queue = NULL) {
if ($upgrader = _flexmailer_civix_upgrader()) {
return $upgrader->onUpgrade($op, $queue);
}
}
/**
* @return CRM_Flexmailer_Upgrader
*/
function _flexmailer_civix_upgrader() {
if (!file_exists(__DIR__ . '/CRM/Flexmailer/Upgrader.php')) {
return NULL;
}
else {
return CRM_Flexmailer_Upgrader_Base::instance();
}
}
/**
* Search directory tree for files which match a glob pattern
*
* Note: Dot-directories (like "..", ".git", or ".svn") will be ignored.
* Note: In Civi 4.3+, delegate to CRM_Utils_File::findFiles()
*
* @param $dir string, base dir
* @param $pattern string, glob pattern, eg "*.txt"
* @return array(string)
*/
function _flexmailer_civix_find_files($dir, $pattern) {
if (is_callable(array('CRM_Utils_File', 'findFiles'))) {
return CRM_Utils_File::findFiles($dir, $pattern);
}
$todos = array($dir);
$result = array();
while (!empty($todos)) {
$subdir = array_shift($todos);
foreach (_flexmailer_civix_glob("$subdir/$pattern") as $match) {
if (!is_dir($match)) {
$result[] = $match;
}
}
if ($dh = opendir($subdir)) {
while (FALSE !== ($entry = readdir($dh))) {
$path = $subdir . DIRECTORY_SEPARATOR . $entry;
if ($entry{0} == '.') {
}
elseif (is_dir($path)) {
$todos[] = $path;
}
}
closedir($dh);
}