Scheduled reminder emails sent to contacts with do_not_email set or with email on hold

I have no idea how new this is, but contacts with "Do not email" checked receive emailed scheduled reminders. If "Do not email" is checked, a contact should never be sent email under any circumstances.

Pull request for tests only at https://github.com/civicrm/civicrm-core/pull/15778

Okay @tommybobo pointed out that the documentation says that people with "Do not email" are expected to receive scheduled reminders: https://docs.civicrm.org/user/en/latest/email/scheduled-reminders/#scheduled-reminders-and-privacy-options

This is ridiculous: the whole point of "Do not email" is that some people should absolutely, positively never get an email, period. The whole reason there's a "no bulk emails" field is that someone might want to receive receipts and reminders but not CiviMail messages.

The stakes for this sort of thing are high--donors get mad when you email them when they've asked you not to. It's also a matter of email reputation: they'll hit Spam in their email client, and there's nothing you can do to prevent them from being emailed in the future.

It gets worse: I just thought that while I was in there writing test cases, I should check that they're not sent if emails are on hold. Welp, it appears that we aren't suppressing those emails, either.

I've updated the PR, so the tests now fail doubly.

changed title from Scheduled reminder emails sent to contacts with do_not_email set to Scheduled reminder emails sent to contacts with do_not_email set or with email on hold

I think the work flow should default to obeying preferences, but allowing an option for a scheduled reminder to override preferences.

Specifically I think a donor with an ongoing recurring gift who has opted out of emails would like to get a notice that their donation has failed. Or if participant has a location change. These are an extension of the receipt process which doesn't obey the preference.

These are an extension of the receipt process which doesn't obey the preference.

I think that's the key thing: the times when you would want the scheduled reminder to override someone's preferences are special circumstances, usually stemming from something that just happened. It could be argued that those aren't really cases for scheduled reminders.

In fact, if you were to have a last-minute location change for an event, you'd probably use the "send an email" feature to send out a notice to all of the participants, and that would exclude the "do not email" people.

I figured a relatively quick fix--see the PR for details--so the PR is now to actually suppress scheduled reminder emails in these cases. Already, SMS reminders are suppressed if you have "Do not SMS", so this isn't just making "Do not email" work as those words mean; it also makes "Do not email" work the same as "Do not SMS".

We are of the same thinking @andrewhunt that 'no bulk email' only restricts mail from the CiviMail bulk system. Whereas 'do not email' should restrict all emails from Civi except transaction (e.g. contribution pages, even reg) receipts. I can't think of a valid use case otherwise.

added triaged label

added sig:bug sig:usability labels

(Note: In the PR, there's some additional discussion along these lines. @agh1 also proposed using a table to try to compare the ways one interprets "Do not email", and I copied it to gdoc along with some additional examples of contextual reasoning.)

@tommybobo wrote:

I think the work flow should default to obeying preferences, but allowing an option for a scheduled reminder to override preferences.

I agree with this and would (on principle) go a step further.

• One's respect for communication preferences depends on the subject-matter and the data:
  • The subject matter -- a confirmation or receipt; an end-of-year tax report; a notice about changes in an event; an inquiry about event attendee's arrival/accommodation/diet; an inquiry/support-thread initiated by the constituent; an appeal to renew an existing membership; an appeal to fill out a general survey; an appeal to fill out a survey about something which you just did; notifications about progress in case-work; education about the org's work; ad nauseum.
  • The data -- a database which is only used for donations; a database which is used for a mix of donations, events, case-work, support; etc. (The meaning of your data depends on where it comes from, what forms you presented to people, the context of those forms, etc.)
• The same subject-matter and data can be processed through different tools -- one-on-one email; search+checkbox+email; CiviMail blast to a smart-group; CiviRules trigger; scheduled reminders; sui-generis application logic; etal.

For example... depending on how the specific membership program works, it's entirely reasonable to send content about renewals through a scheduled-reminder... or a CiviMail blast... or a one-on-one message. For a last-minute notification about changes in an event, I can totally see people using the individual mailer, the adhoc mail-blast, or a normal mail-blast.

So I think a general way to reconcile these is to say:

• Any built-in communication tool should have built-in support for comm-prefs.
• Built-in support for comm-prefs means:
  • It defaults to behavior which respects the comm-prefs.
  • It warns the backend-user/admin if they're requesting an action that would break a comm-pref.
  • It warns the backend-user/admin if some action is being skipped due to a comm-pref.
  • It allows the backend-user/admin to decide how/whether/which comm-pref is appropriate to the task at hand.

I want to make sure this doesn't wither on the vine. As best as I can see, it looks like:

1. Sometimes people want to override communication preferences.
2. Sometimes people have poorly-implemented communication preferences (people saying Do Not Email when they just want opt-out, or vice-versa).
3. Nobody has suggested that communication preferences should be ignored by default.
4. Nobody has given any reason why you'd want to send scheduled reminder emails to an address that's on hold.

Based upon (3), it seems that the concerns about PR #15778 really just boil down to wishes for a feature that allows overrides for certain purposes ((1) above).

I don't think anyone intends for it to be impossible to suppress scheduled reminder emails whether through do_not_email or on_hold. That's what PR #15778 resolves. I'm all for options for per-reminder overrides, but that's a new feature outside of the scope of that PR.

I do think (2) is an issue, but that can be resolved through help text and documentation, not making the Do Not Email non-functional.

@andrewhunt agreed

I approve this PR and feature being implemented. Let's get it done already.

mentioned in commit 157e2097

mentioned in commit 83aa0e63

mentioned in commit 206a21e8

mentioned in commit f0be7552

mentioned in commit deecfdef

mentioned in issue #1513 (closed)

Closing because PR 15578 is merged, and that addresses the scope of the original issue.

To ensure that we don't lose the discussion about related usability issues, I forked off #1513 (closed).

closed