Community issueshttps://lab.civicrm.org/groups/community/-/issues2022-10-29T21:00:20Zhttps://lab.civicrm.org/community/community-engagement/-/issues/24Meta - recognizing community engagement2022-10-29T21:00:20ZbgmMeta - recognizing community engagementVery random notes from discussions at the Manchester 2022 sprint, in no particular order. Feel free to add more.
- [ ] infra/gitlab#43 When someone opens an issue on Gitlab for the first time, add a "new-contributor" label (apparently D...Very random notes from discussions at the Manchester 2022 sprint, in no particular order. Feel free to add more.
- [ ] infra/gitlab#43 When someone opens an issue on Gitlab for the first time, add a "new-contributor" label (apparently Drupal are working with Gitlab about this? but it will take a while, and adding a label is simple)
- [ ] infra/gitlab#43 Post "new-contributor" alerts in a mattermost channel, where people interested could follow and help engaging with the request
- [ ] Contributor listing on the website (there is already an issue for this somewhere, we were waiting after SearchKit, but that's now usable for our requirements)
- [ ] (very old problem) people willing to be pinged on certain topics
- Joe uses email filters for subscribing to specific labels?
- Justin mentioned it would be nice to have a ping in new issues, after 2 weeks if no engagement, to try to keep it alive and moving forward (and that ping could ping more people based on tags or keywords)
What we already do (please complete this list):
- Jaap does Gitlab issue triage, and tries to be more attentive to first-time contributors
- Eileen, Seamus and DaveD keep a very close eye on Github pull-requests, and often spend a lot of time helping with the "other 50%" required for a PR to be merged.
- Developer trainings at in-person events
- Monthly release blog posts mention who contributed code and commented on PRs.
- Monthly release blog posts mention new extensions (still a form of recognition)
- Dev-digest will try to bring some attention to specific issues that need wider feedbackhttps://lab.civicrm.org/community/community-engagement/-/issues/232022 Manchester developer training2022-09-30T18:15:42Zeileen2022 Manchester developer trainingManchester training runs from 6-7 Oct & this gitlab is for planning and preparation around it. [The signup form is here](https://civicrm.org/civicrm/event/info?reset=1&id=1674)
# Capacity / signups
As of writing we have
Capacity ?
Cap...Manchester training runs from 6-7 Oct & this gitlab is for planning and preparation around it. [The signup form is here](https://civicrm.org/civicrm/event/info?reset=1&id=1674)
# Capacity / signups
As of writing we have
Capacity ?
Capacity on the event page : 12 (I just increased it to 12 but maybe 14)
Catering ordered for : 10
Trainers: 4 (Matt, Aiden, Tim, Eileen)
Signups: 10
Paid 6
Not yet paid 4
Per the image we can probably increase the number of spaces to 14 fairly comfortably & I propose we do that. We should also determine if all the not-yet-paid are serious (two have been in touch and are known to us - the last two are new to our database and may not be genuine sign-ups). (I just enabled waitlist functionality on it in the course of writing this)
![image](/uploads/1a5ace18cee35aa4947032c965bd4e0e/image.png)
# Preparation
Generally when we run sprints getting a functional development environment running is challenging for some people, although we know that at least three people will already have them. Our goal is that people are prepared for the spring by having either
- a functional local developer environment with CiviCRM & preferably civix and buildkit already installed or
- virtual box installed and we will provide a USB stick with an image on it
Our preference is for the former but in the case of the latter it seems the image/s we bring will depend somewhat on the devices people bring - so we need to gather that information. In order to do that I (Eileen) will send a pre-spring email to attendees.
# Agenda (rough)
### Day one -
**Morning**
Introductions
Key concepts: APIs, Hooks, Extensions, Entities, Pages, Forms [xml, DAO, BAO, Managed Entities, Settings ?], CMS/UF (**Matt**)
Setting up dev-env / Civi / buildkit
**Afternoon**
Admin UI - show how it works with the packaged search, exporting, searches show debug (**Aidan**)
Rest of the day structured around creating `CiviGoat` (or another extension) with basic features - with an entity, Page, settings, search kit,
Civix - create first extension
API - explorer etc
Unit tests
Creating entities / accessing them via search kit/ form builder
Creating settings
Big E / ts()
### Day two
**Morning**
Debugging live sites. ConfigAndLog, Query logging
PR process & giving code back
**Afternoon**
Flexible to adjust to what we did and didn't cover
# Pre-sprint survey form:
https://docs.google.com/forms/d/e/1FAIpQLSf8xoYweZfXW7wM4jSmI9ezbWdKv4_1fr-QfhBKaJwf9IFnqQ/viewform?usp=sf_link
# Pre-sprit email text
Hello,
Thank you for signing up to the [CiviCRM developer training](https://civicrm.org/civicrm/event/info?id=1674&reset=1) - we look forward to seeing you.
The training will start at 9am on October 6th, 2022 and you will need to bring a laptop to work on while there. You will need to have a working development environment, including Civix and preferably Buildkit. If you do not already have one, then we need you to either connect with us on [chat](https://chat.civicrm.org/civicrm/channels/sprint) to get one set up, or install [VirtualBox](https://www.virtualbox.org/) (which will make it easy for us to set up your development environment for you after you arrive).
To help us prepare please fill in [this google form](https://docs.google.com/forms/d/1uNFEJaN5kS6vHd53flvl_Mfz-7JNE4Z_wdgMi4v8tdc/edit) with details about your experience and your development environment.
If you have not paid please do so to secure your place as places are limited.
In open source manner we are tracking our preparations for the training in gitlab - see https://lab.civicrm.org/community/community-engagement/-/issues/23
**Assignments / Action Items:**
Tim: Update VirtualBox. Get USB keys.
Matt: Key concepts plan
Aidan: Admin UI plan
Eileen: Send surveyhttps://lab.civicrm.org/community/feature-request/-/issues/30Cron issue with Joomla 42022-06-22T18:40:31ZmcherkesCron issue with Joomla 4After the Joomla update, from v3 to v4, the automated launch of cron failed.
![Screenshot_from_2022-06-22_17-36-37](/uploads/8d4f2bdbfdcba759eccfc39ef932f071/Screenshot_from_2022-06-22_17-36-37.png)
Have tried to launch it manually with...After the Joomla update, from v3 to v4, the automated launch of cron failed.
![Screenshot_from_2022-06-22_17-36-37](/uploads/8d4f2bdbfdcba759eccfc39ef932f071/Screenshot_from_2022-06-22_17-36-37.png)
Have tried to launch it manually with wget , but received errors on the screen bellow
![Screenshot_from_2022-06-22_17-30-55](/uploads/059bacf38dc1d4e82cbcd6a4ce460185/Screenshot_from_2022-06-22_17-30-55.png)
Our assumption is the folders and files in Joomla 4 differ from Joomla 3.https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/31CiviCrm Upgrade error2022-11-18T10:21:01ZmarkdevarajCiviCrm Upgrade errorhi,
I was looking to upgrade my current version of CiviCrm 5.28.3 to 5.36.0 and was getting the following error - see attached files too which has a screen shot and a copy of the log file.
The upgrade fails at upgrade to 5.31.alpha1
|...hi,
I was looking to upgrade my current version of CiviCrm 5.28.3 to 5.36.0 and was getting the following error - see attached files too which has a screen shot and a copy of the log file.
The upgrade fails at upgrade to 5.31.alpha1
| Error Field | Error Value |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Type | DB_Error |
| Code | -1 |
| Message | DB Error: unknown error |
| Mode | 16 |
| UserInfo | ALTER TABLE civicrm_group CHANGE `title` `title` varchar(255) DEFAULT NULL COMMENT 'Name of Group.' [nativecode=1071 ** Specified key was too long; max key length is 767 bytes] |
| DebugInfo | ALTER TABLE civicrm_group CHANGE `title` `title` varchar(255) DEFAULT NULL COMMENT 'Name of Group.' [nativecode=1071 ** Specified key was too long; max key length is 767 bytes] |
Does anyone know how to fix this?
cheers
mark
![Screen_Shot_2021-06-04_at_2.10.42_pm](/uploads/c1dea707294f13740cdc40afdb1fc3be/Screen_Shot_2021-06-04_at_2.10.42_pm.png)
![Screen_Shot_2021-06-04_at_2.14.21_pm](/uploads/735e517da4871c7028ac386ae514f7a5/Screen_Shot_2021-06-04_at_2.14.21_pm.png)https://lab.civicrm.org/community/gsoc/-/issues/11Machine Learning Tool for Email Spam Score Calculator2021-04-11T20:50:28ZPratik10100Machine Learning Tool for Email Spam Score CalculatorHello @JoeMcLaughlin and potential mentors,
This project aims to revolutionize the way we engage and interact with email. Email marketing sending multiple emails as part of a campaign is already enough of a challenge. According to some...Hello @JoeMcLaughlin and potential mentors,
This project aims to revolutionize the way we engage and interact with email. Email marketing sending multiple emails as part of a campaign is already enough of a challenge. According to some estimation, about one-fifth of permission-based emails sent by legitimate marketers land in recipients' spam folders. Without using some spam tester tool, emails stand an even greater chance of being marked as spam.
So I'm trying to develop a tool that produces a score of text provided to the model. If the score goes above five on the scale of ten, it will be marked as spam; knowing this beforehand helps a lot to the mass mailer, and then needy textual changes can be made.
I have gone through CiviCRM stack exchange there; I have seen people struggling with low opening rates in the final part of A/B testing and thinking spamminess is one factor. So by having a tool that deals with spam classification would be helpful and handy to the mass mailer.
I believe CRM is known to have a big data hub, so it makes sense to try and utilize those insights for machine learning in creating a tool that helps the user in the domain of email marketing.
Two significant steps involved in building a new open-source email spam score calculator are.
- Experimenting with various spam classification techniques to figure out which one provides a required balance of precision (the fraction of results classified as positive, which are indeed positive) and recall(the fraction of all positive results which were detected).
- Providing an independent web service (like ORES) that can entertain the request to calculate the spamminess(score) of the email.
I am looking forward to know your opinion about this project, and I will soon come up with a detailed proposal that will cover all the algorithmic and implementation parts.
Thanks 😊https://lab.civicrm.org/community/gsoc/-/issues/10Integrate Spoke Peer-to-Peer Texting2021-04-11T02:23:07ZashaIntegrate Spoke Peer-to-Peer TextingI would like to contribute for 'Integrate Spoke Peer-to-Peer Texting' project for this summer. In my understanding, we have to create an extension to integrate CiviCRM with Spoke. The org's contacts and their tags( eg: Volunteer) shoul...I would like to contribute for 'Integrate Spoke Peer-to-Peer Texting' project for this summer. In my understanding, we have to create an extension to integrate CiviCRM with Spoke. The org's contacts and their tags( eg: Volunteer) should be synchronize with Spoke. Can i know what does it mean by question and answer? does that mean the peer-to-peer texting?https://lab.civicrm.org/community/gsoc/-/issues/9Contribution to the project: "Mailing list functionality"2021-03-30T18:19:55ZMAVIN-07Contribution to the project: "Mailing list functionality"Hi,
I am an undergraduate and wish to start contributing to CiviCRM. I found one of the listed projects very appealing i.e. **"Mailing list functionality"**. Can anyone please guide me, how to proceed with it?Hi,
I am an undergraduate and wish to start contributing to CiviCRM. I found one of the listed projects very appealing i.e. **"Mailing list functionality"**. Can anyone please guide me, how to proceed with it?https://lab.civicrm.org/community/sustainability/-/issues/21access to old survey results on d7.c.o?2022-12-03T20:18:34ZJoeMurrayaccess to old survey results on d7.c.o?@josh could you approve @bgm providing access to or export of Activities of type Community Survey on d7.c.o as part of our efforts on #17 ? I think it is from 2 or 3 surveys ago.@josh could you approve @bgm providing access to or export of Activities of type Community Survey on d7.c.o as part of our efforts on #17 ? I think it is from 2 or 3 surveys ago.joshjosh@civicrm.orgjoshjosh@civicrm.orghttps://lab.civicrm.org/community/feature-request/-/issues/29Add "About" page to simplify AGPL Section 13 compliance2021-02-18T21:01:17ZtottenAdd "About" page to simplify AGPL Section 13 compliance# Background
1. CiviCRM is published under AGPLv3.
2. Most CiviCRM extensions are also published AGPLv3.
3. It is common in our community for people to deploy, fork, patch, etc. code from others. (Witness: open-source in action!)
4. ...# Background
1. CiviCRM is published under AGPLv3.
2. Most CiviCRM extensions are also published AGPLv3.
3. It is common in our community for people to deploy, fork, patch, etc. code from others. (Witness: open-source in action!)
4. Under AGPLv3 Section 13, if you make a modified/derivative work, then there is an obligation to provide download links to your users.
5. CiviCRM Core has a standard download link in the footer -- but you can't do that for everything, and I've never noticed an extension providing download links to non-admins, and most people who use patches aren't going to edit code to tweak that notification.
6. There is no general/standard practice in how to address Section 13.
# Proposal
The footer should include a Legal/About. Add a built-in core page `civicrm/about` which:
* (a) summarizes CiviCRM's mission+model
* (b) summarizes the works being used for this deployment - authorship/copyright/publication URLs
* (c) optionally links to any site(s) where modifications are published (e.g. "All modified works are published at https://github.com/myorg")
Parts (a) and (b) can be standard/no-effort/out-of-the-box.
Part (c) would be unnecessary and blank for stock deployments. If you are an active implementer/developer in the habit of patching things, it can be a bit onerous to identify every patch in-app. But it should be easy to add your Github/Gitlab org-link. If someone suggests that you are out-of-compliance on Section 13, then this becomes an easy fix.
# Comments
(1) When I was an implementer, we used a patch similar to this on our Civi deployments. It actually went a bit further and allowed you download a live zip file of the civicrm tree (to ensure that we were meeting the obligation while keeping our project-management stuff private). However, I suspect many folks would have technical questions (e.g. "persuade me about performance/security/usability of a live zip on every Civi deployment"). Many folks already publish their forks on Github/Gitlab - so that feels like the path of least resistance.
(2) Item (b) would actually go beyond what's required. However, I think it's nice for our community to over-communicate credits.https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/30URLs on Mosaico emails and on Wordpress posts2021-02-03T20:06:33ZmarkdevarajURLs on Mosaico emails and on Wordpress postsWe use Mosaico to send out a weekly newsletter.
In the Newsletter email we have URL links to more information on our Wordpress posts and calendar events.
The URL link in the email is
https://example.org/civicrm?civiwp=CiviCRM&q=civic...We use Mosaico to send out a weekly newsletter.
In the Newsletter email we have URL links to more information on our Wordpress posts and calendar events.
The URL link in the email is
https://example.org/civicrm?civiwp=CiviCRM&q=civicrm/mailing/url&u=11088&qid=223581
When we clink on this in the email it takes to a the particular event on our website calendar
https://example.org/event/group-2021/2021-02-01/
Which is what we want it to do. So all good so far...
However,
If I was to create a post on our website with the URL link (same as above)
https://example.org/civicrm?civiwp=CiviCRM&q=civicrm/mailing/url&u=11088&qid=223581
and click on this link, I just get to our main front webpage
https://example.org/
Is this a Mosaico configuration issue or a CiviCrm config issue or a Wordpress issue?
Any suggestion on how to resolve the URL on the post page taking the reader to the event ??
Many thankshttps://lab.civicrm.org/community/region/melbourne-meetups/-/issues/29CiviBooking on Wordpress2021-02-03T12:30:53ZmarkdevarajCiviBooking on WordpressThis is not an issue but more of a question.
We are a non profit with Rooms that we would like our members to hire and pay booking fee for use.
Has anyone installed CiviBooking on Wordpress? Is this a Drupal centric extension only? Wil...This is not an issue but more of a question.
We are a non profit with Rooms that we would like our members to hire and pay booking fee for use.
Has anyone installed CiviBooking on Wordpress? Is this a Drupal centric extension only? Will this work on Wordpress?https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/28Errors in ConfigAndLog 21 Nov 20202020-11-26T22:21:32ZmarkdevarajErrors in ConfigAndLog 21 Nov 2020Hello all,
I'm wondering if anyone has seen these errors and how to resolve them.
[Errors_ConfigAndLog_Nov_21_2020.docx](/uploads/a3c3853539544dad2d8c4b9ea3297ce9/Errors_ConfigAndLog_Nov_21_2020.docx)
My environment
Wordpress 5.5.3...Hello all,
I'm wondering if anyone has seen these errors and how to resolve them.
[Errors_ConfigAndLog_Nov_21_2020.docx](/uploads/a3c3853539544dad2d8c4b9ea3297ce9/Errors_ConfigAndLog_Nov_21_2020.docx)
My environment
Wordpress 5.5.3
Civicrm 5.28.3
Mosaico 2.5.1597918155
We do have intermittent problems with saving working mosaico newsletters/templates
Thanks in Advancehttps://lab.civicrm.org/community/feature-request/-/issues/27OAuth2 administration (email focus)2020-10-22T21:43:12ZtottenOAuth2 administration (email focus)# Background
CiviCRM integrates with various third-party services. When submitting read/write requests to contemporary web APIs, it is quite common to authenticate via OAuth - esp OAuth2 (https://tools.ietf.org/html/rfc6749). For this u...# Background
CiviCRM integrates with various third-party services. When submitting read/write requests to contemporary web APIs, it is quite common to authenticate via OAuth - esp OAuth2 (https://tools.ietf.org/html/rfc6749). For this use-case, we consider CiviCRM (the web-app) acting a client to some service-provider. Two important workflows are:
* _Client registration_: This is a manual, non-standardized process. The site admin logs into the Google/Microsoft/Facebook/Twitter/etc administrative system and generates a `client_id` and `client_secret`. These values must be copied into the web-app (CiviCRM) configuration.
* _Authorization Code Grant_: This process obtains permissions for a specific user/resource/service. It involves (a) performing a semi-manual workflow to confirm permissions and (b) periodically requesting access tokens.
For CiviCRM integrations (core or contrib) that wish to speak to other systems, this creates a recurring administrative problem:
* (Developer) You need to define several artifacts (docs, settings, routes) for managing `client_id`, `client_secret`, and each of the authorizations/grants.
* (Administrator) You need to examine the separate artifacts (docs, settings, etc) required by each integration, and there will likely be arbitrary differences.
This proposal arises from https://lab.civicrm.org/dev/mail/-/issues/59 and parallel issues for Microsoft's email service. When we need examples to make this more concrete, we'll refer to IMAP and cloud email.
# Goal
Define a general OAuth2 administration mechanism in CiviCRM.
To keep the scope limited, the initial concern is managing credentials for *general system-level integrations* (e.g. `Administer => System Settings => Credentials`). For example, one might use this to:
* Connect CiviMail with a cloud mail service
* Connect CiviContribute with a cloud accounting system
This does not address the per-user authorizations. For example, if you wished to allow each staff member or peer-to-peer fundraiser to link their own email account (for use with sending one-on-one emails), then that would be a different endeavor. (I believe it is a *superset* of the endeavor described here, and the "Add-ons/Strech-goals/Future-directions" touches on this.)
# Model
We track information in three layers:
1. _Provider (Client-Type)_: Suppose, for example, that many CiviCRM deployments connect to services provided by Google and Microsoft. Anytime you connect to Google, it should go to a particular URL; similarly, anytime you connect to Microsoft, it goes a particular URL. You need store a list of these common URLs. For each provider, you'd expect a few key properties like:
* `authorize_url`
* `token_url`
* `default_scopes`
* `provider_class` (string; ex: in `theleague/oauth2-client`, it has `\League\OAuth2\Client\Provider\GenericProvider` for generic use; but [alternate classes](https://oauth2-client.thephpleague.com/providers/thirdparty/) can provide extra options+data)
2. _Client_: When a CiviCRM site (`example.com`) connects to a provider, it must identify itself. Key properties:
* `client_id` (unique id; functions a username when contacting the OAuth2 service)
* `client_secret` (secret credential; functions a password when contacting the OAuth2 service)
3. _Account Authorization_: When a CiviCRM site is configured as a client, it may not have authorization to any particular resources/accounts/services. To acquire access, you need the user ("resource owner") to open the web-app and go through some redirects/authorization screens.
# Ex: IMAP: UI Mockup
The current IMAP configuration screen accepts a username+password. Instead, one needs to use credentials that are stored in an OAuth2 subsystem. Compare:
* With username+password: [New_Mail_Account_Orig](/uploads/d05f8e3429ae75ded610dacf2c52c905/New_Mail_Account_Orig.png)
* With choice of username+password or OAuth credentials: [New_Mail_Account_UA](/uploads/0b6338b92a3c89087f3566f4d26ccdc2/New_Mail_Account_UA.png) ~~[New_Mail_Account_XOAuth2](/uploads/71a3587f77872b7e230b86cfc7bcc1df/New_Mail_Account_XOAuth2.png)~~
The "Manage Credentials" screen gives the Civi administrator the ability to configure the client identity and the account authorizations. The layout might be:
* Tabular layout, akin to many of Civi's other administrative screens -- e.g. [Manage_Credentials_Table](/uploads/15d05b998375bdd9f73c3fa53adba3da/Manage_Credentials_Table.png)
* Bespoke layout, with more key details on the admin screen -- e.g. [Manage_Credentials_Sections](/uploads/eb95b49205be2b9b04a9db57be4706a0/Manage_Credentials_Sections.png)
# Ex: IMAP: General tasks
A loose/general list of tasks:
* _Add OAuth data structures_:
* _Client types_: Define a data-structure (e.g. JSON file, hooks) for programmatically defining client types.
* (Should this have a SQL storage option? IMHO, no, that's an extremely niche proposition. To require that the administrator to first create the "Client Type" in the web UI seems like an exercise in software brutalism.)
* _Clients_: Define a data-structure for client_ids, client_secrets, etc. These details are necessarily different for each deployment, so it's necessary to store them somewhere in the admin's purview -- ie SQL and/or `civicrm.settings.php`.
* _Account Authorizations and Tokens_: Define a data-structure for account authorizations and their tokens. These must be stored in SQL.
* _Update IMAP/POP data-structure_:
* Store a reference to OAuth
* _Implement UI_: Like in the mockups above:
* Implement the "Manage credentials" screen and "Add account" flow (incl [thephpleague/oauth2-client](https://github.com/thephpleague/oauth2-client))
* Update the "New Mail Account" screen
* _Update IMAP/POP driver_: The current IMAP/pop driver uses username/password. If a specific mailbox uses IMAP
* _Documentation_:
* Developer: How to define or edit a client-type
* Developer: How to send a web-service request with stored credentials and Guzzle
* Sysadmin: How to manage clients/connections
# Add-ons / Stretch goals / Future directions
Here are few topics for potential exploration:
* _E2E Testing_: Setup a test-case which makes real IMAP+OAuth2 connections
* _Reuse client-type/provider metadata_: The list of client-types is, essentially, metadata. The `oauthd` project has a ton of metadata stored as JSON files. Consider using their metadata. https://github.com/oauth-io/oauthd/tree/master/providers
* _Aggregator support_: This issue is oriented toward singular/dedicated deployments. However, if you have multi-tenant/multi-site deployments, then deploying OAuth2 is more onerous. (*To wit: do you register every site with its own client_ids -- which is a lot of manual setup -- or do you copy/share the IDs -- which is less secure and may not be supported by all providers*). An OAuth aggregator allows your various tenants/subsites to use the same client ids more securely. Generally, the goal would be to integrate the list of "Clients" in Civi with the aggregator.
* (The oauthd project linked above is an aggregator. It seems it isn't being updated, but it does have a lot of forks. Curious.)
* _In-situ authorization_ or _Wizards_: The "Manage Credentials" screen is disjoint from the "New Mail Account" screen. With in-situ authorization or wizards, you could handle the authorization in-siteu (e.g. in "New Mail Account" or in CiviMail).
* _Authorization (Meta)Access Controls_: This issue is oriented toward a global/shared service (like the CiviCase IMAP checker), but OAuth can also be used for per-user services. Example: Suppose staffer Alice wants to send a mail-merge document to Google Cloud Print. This probably requires OAuth authorization. Civi can store that credential and re-use it whenever Alice wants to print. But if Bob wants to print, then he needs to go through a different authorization (because he uses different printers).https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/27Unable to deliver mail to bigpond email addresses - Mosaico Flexmailer CiviMa...2020-11-01T12:32:32ZmarkdevarajUnable to deliver mail to bigpond email addresses - Mosaico Flexmailer CiviMail - suspected spam IB 703 being reported by TelstraYesterday we used Flexmailer and Mosaico for the first time to send a bulk email newsletter (450 emails) through CiviCrm. The email had links, images and text.
We have been using Flexmailer previously with traditional email template la...Yesterday we used Flexmailer and Mosaico for the first time to send a bulk email newsletter (450 emails) through CiviCrm. The email had links, images and text.
We have been using Flexmailer previously with traditional email template last week and did not have a problem. I am suspecting the combination of Mosaico, Flexmailer and Civi.
We are on Wordpress v5.5 and CiviCRM v5.28.3
This (n-1) version of CiviCrm has been running for a couple of weeks without any issues.
We received bounced messages from all bigpond email addresses only.
The message is as follows:-
The response from the remote server was:
558 5.7.1 5ED4071B1EC49D20 Message content rejected due to suspected spam. IB703
The sending server was 134.209.3.79
The sending server is not blacklisted (as per Spamhaus) and there are no shortened URL links in the email.
Our server sends these through a Google SMTP gateway.
We have SPF, DKIM and Dmarc records all set in our DNS. Inspection of the headers shows a pass for SPF and DKIM
Verp enabled or disabled has been tried but no change as we still get a bounce from Telstra.
We do not have an issue with any other receiving email servers except Telstra mail servers to email addresses that are bigpond.com, bigpond.net.au
Is there a setting that has to set for bigpond email address so that Telstra mail servers let these pass thru to the recipient?
Could anyone pls let me know if you have seen this problem with bigpond email addresses and how did you resolve the problem.
thanks
markhttps://lab.civicrm.org/community/region/melbourne-meetups/-/issues/26Server migration2020-09-21T08:17:35Zdavid.hohmServer migrationHello,
sorry for what is probably a ridiculous question.
Our wordpress site was migrated to a new server by our host, the domain is unchanged. Now several of the CIVICRM features are not working anymore. Such as payment processing for e...Hello,
sorry for what is probably a ridiculous question.
Our wordpress site was migrated to a new server by our host, the domain is unchanged. Now several of the CIVICRM features are not working anymore. Such as payment processing for example or signing up to our newsletter. One of the issues for example is our stripe webhook failing as it now can't reach https://example.org.au/?page=CiviCRM&q=civicrm/payment/ipn/3. When I try to go to the live links for contribution pages, it just shows: "Do not delete this page. Page content is generated by CiviCRM."
I've cleared the cache and reset the paths and looked at the civicrm.settings.php files but couldn't find a place where I had to link to a new IP address. I've also looked for a guide on the CiviCRM page but was unable to find clear instructions on what to do.
If someone can point me in the right direction it would be super appreciated, thank you for the help
cheers, Davidhttps://lab.civicrm.org/community/region/melbourne-meetups/-/issues/25Installation of Mosaico and Flexmailer extension thru GUI2020-08-18T02:04:30ZmarkdevarajInstallation of Mosaico and Flexmailer extension thru GUIAfter the setup of my Wordpress staging server as a subdomain I was looking to test out the Mosaico extension.
I installed Flexmailer (v 1.1.1) and Mosaico (v 2.4.1592820024) thru the CiviCrm GUI.
I am unable to view the standard defau...After the setup of my Wordpress staging server as a subdomain I was looking to test out the Mosaico extension.
I installed Flexmailer (v 1.1.1) and Mosaico (v 2.4.1592820024) thru the CiviCrm GUI.
I am unable to view the standard default Mosaico templates! when I go into Mailings>>Mosaico Templates
I get the attached screen shot
[Screen_Shot_2020-08-03_at_10.43.08_am](/uploads/cd8c5082cf12067e49d87ef68a09cf32/Screen_Shot_2020-08-03_at_10.43.08_am.png)
A couple of questions...
1. what should be the settings in Administer>>Civimail>>Mosaico Settings be?
2. what should be the settings in Administer>>Civimail>>Flexmailer settings be?
Any help from someone who has installed Mosaico would be appreciated.
Many thankshttps://lab.civicrm.org/community/region/melbourne-meetups/-/issues/24Running CiviCrm on a separate domain on the same server2020-08-03T02:44:24ZmarkdevarajRunning CiviCrm on a separate domain on the same serverOn my production server (Ubuntu) I have a Wordpress and also a CiviCrm database.
My aim is to have a staging server on the same server but as a separate domain. I have been able to successfully copy my production Wordpress files to the...On my production server (Ubuntu) I have a Wordpress and also a CiviCrm database.
My aim is to have a staging server on the same server but as a separate domain. I have been able to successfully copy my production Wordpress files to the staging server and get to it from the internet. That is I can get to the entire staging website from the internet.
The problem that I am having is setting up CiviCrm in its separate db on the staging server.
I use Plesk on this server and used Plesk to setup a staging server/subdomain.
Plesk copied all the files from Prod to the staging server in sub-directory structure - /var/www/vhosts/example.org/subdomain.example.org.
My prod files are in /var/www/vhosts/example.org
I have copied the civicrm db from prod to staging and given it a different name, user and password. (civicrmstaging)
My questions are
1. What CiviCrm files do I need to change in the staging environment? Is it only civicrm.setting.php?
2. what statements in civicrm.setting.php do I need to change? I have changed the user and sql database password for civicrmstaging db in civicrm.setting.php
3. Do I need to change something within the CiviCrm sql tables?
4. are there any other files that needs to be changed in the staging environment?
I cant seem to find the appropriate docs for my setup scenario.
Any help would grateful.https://lab.civicrm.org/community/feature-request/-/issues/26Multi-lingual for multinational/global-scale2022-06-11T16:02:17ZtottenMulti-lingual for multinational/global-scale(This issue is not a singular problem with a singular fix. Think of it more as a long-term epic or general problem statement - it may entail several different approahces, issues, extensions, patches, phases, or experiments. My hope is to...(This issue is not a singular problem with a singular fix. Think of it more as a long-term epic or general problem statement - it may entail several different approahces, issues, extensions, patches, phases, or experiments. My hope is to capture some discussion from sprints+calls over the years, esp points from @ayduns @BjoernE @bgm @eileen @ejegg etal)
# Background
Internationalization (i18n) is the process of adapting a software system to support different languages and locales. For some organizations, their i18n needs are met by flagging one language for the entire system (e.g. an English organization uses the English language; a French organization uses the French language).
Other organizations require a more advanced form of i18n called *multilingual* -- this means that one organization offers its business-programs (eg events, conferences, newsletters, donation pages) in multiple languages *concurrently*. There are a few contexts in which multilingual makes sense:
* The organization targets a single region with multiple endemic languages. For example, in Canada, you may have an organization serving significant populations of French and English speakers. Similarly in Belgium (Flemish/French), southwestern US (English/Spanish), Switzerland (Swiss German/Swiss French/Swiss Italian), and so on. As a rule-of-thumb, few locales would have more than 3 endemic languages.
* The organization targets a large number of countries with diverse languages. For example, a pan-European organization might have 8 languages, and a global organization might have 30 languages.
When adapting business software to a multilingual organization, one might take an informal approach or a formal approach, by which I mean:
* __Informal translation (duplicates)__: The software remains effectively unchanged. If the user needs more languages, then he simply adds more records. Thus, if the user wants to make a bilingual newsletter, then he actually makes two distinct `Mailing`s (one per language). For a bilingual fundraising campaign, he makes two `ContributionPage`s. (*This requires less upfront technical work, but it also requires on-going training. Fine-tuning workflows+reports may be tricky.*)
* __Informal translation (in-line)__: As above, the software remains effectively unchanged. If the user needs more languages, then he just does everything in-situ. Thus, a bilingual newsletter has one "Subject" with two languages (`Subject: Hello world / Bonjour tout le monde`). (*This again requires less upfront technical work, but it's progressively uglier as you add each new language. Fine-tuning workflows+reports may still be tricky.*)
* __Formal translation__: The software is updated to allow translation of different records. Thus, a bilingual newsletter is one `Mailing` with two different `Subject`s. A bilingual fundraising campaign is one `ContributionPage` with two descriptions. (*This requires more upfront technical work, but it allows more tuning of the workflows and reports.*)
There is some documentation about CiviCRM and i18n, eg
* https://docs.civicrm.org/user/en/latest/the-civicrm-community/localising-civicrm/
* https://docs.civicrm.org/dev/en/latest/translation/
# Problem/Goal (General)
In its default/basic mode, CiviCRM supports *single language* (which can also be used for *informal translation*). Additionally, the Civi administrator can enable *formal* multilingual. The formal mode is well suited to some organizations and challenging for others.
The next section will summarize some specific technical issues, and the needs obviously vary case-by-case, but (broadly speaking) I found it helpful to consider two types of multilingual organizations:
* __(Loosely) Bilingual orgs__: For regional organizations supporting 2-3 endemic languages (loosely - "bilingual" orgs), Civi's formal multilingual is "pretty good"; it's often "optimal"; and (even at its worst) it is at least "acceptable".
* __(Loosely) Multinational orgs__: For far-flung organizations supporting 8+ languages (loosely - "multinational" or "global" or "pan-continental"), the design is... less optimal.
# Problems (More specific)
* __MySQL Columns__: For every language, Civi's MySQL schema replicates ~100 DB columns in ~25 tables. (For 8 languages, that would be a total ~800 columns.) This can max-out some hard limits in MySQL.
* __Roles__: For an organization supporting 1-3 endemic languages, there are many staff who speak each of the languages, and (e.g.) it's realistic to say that the event-manager role is *responsible* for translating the event-description. However, with 10+ languages, it is not realistic -- so you may create more specialized roles that don't exist in a unilingual/bilingual organization. For example, one person may set the initial terms of the `ContributionPage`, and 9 other people translate the description. More nuanced roles mean you also need to think about:
* __Workflows__: If your role is "Italian translator", then you don't want to manually fish through all the `Event`s and `ContributionPage`s to see which ones need translation. The workflow should draw attention to the things that need you.
* __Permissions__: If your role is "Italian translator", then you certainly need permission to edit the Italian "Description" of an `Event`... but maybe you shouldn't have permission to change the registration-deadlines, the price-structure, or the Spanish translation.
* __Negotiation/Fallback/Sparsity__: With a larger number of languages, it takes a lot of labor to fill every translatable field on every record in every locale in a timely way. There will be gaps in the translation matrix -- whether purposeful ("it's not worthwhile to translate this one between en_US and en_CA") or incidental ("the translator hasn't gotten to this yet"). With sparser translations, the fallback/language-negotiation becomes more important. (Ex: If `en_US` is missing, then fallback to `en_CA` or or `en_GB`. If `fr_CA` is missing, then fallback to `fr_FR`.)
* __Entities/Fields__: The multilingual translation support is targeted at specific entities+fields. However, the value of formally translating any specific entity or field may be assessed differently. Consider two opposing examples:
* __ContributionPage__: For a regional organization with bilingual constituents, you might commit to formally translating every `ContributionPage`. In a global organization with different countries, you might find that each country has so many differences (language, pricing, taxes, etc) that you prefer to create new `ContributionPage` for each.... so you never use *formal* translation of `ContributionPage`s. (*Or, if we're really exacting, you might have one `ContributionPage` for each country... but within bilingual countries, you'd want different translations of the page!*)
* __MessageTemplate__: For a regional organization with bilingual constituents, you may decide it's preferable to send bilingual notifications (ie send a receipt with both English+French text), so you don't need *formal* translation of `MessageTemplate`s. However, in a global organization, it's crazy to put 10 languages into 1 receipt -- instead, you should formally distinguish the translations for each language.
Considerations
----------------------
For purposes of this issue, I would say that support is "complete" when it is possible (by a mix of extensions/patches) to configure the MySQL schema/roles/workflows/permissions/fallbacks/entities/fields in a way that satisfies (a) a regional/bilingual organization and (b) an multinational/global organization.
However, this still leaves considerable room for interpretation/variation. Ask yourself: is it true that multilingual users fall in two distinct buckets (bilingual vs multinational)? Or are those idealized extremes, with most organizations taking some place in between?
* If it is truly two categories, then you might address this by implementing two separate subsystems:
* The existing "bilingual" subsystem provides permissions/workflows/entities that are suitable for bilingual orgs.
* A new "multinational/global" subsystem provides permissions/workflows/entities for multinational orgs.
* If there are many shades of grey, then you might address this by giving more subtlty to each aspect, eg
* The data-storage layer might use localized columns, or a dedicated string table, or something else. (Perhaps you can swap the data-storage while keeping the rest.)
* Different modules might provide different workflows/UIs - so one module allows editing translations in-situ (per-entity), while another another module provides more centralized translations (per-language). (Perhaps these different workflows/UIs can used separately - or perhaps they can coexist giving alternate access to the same data.)
* The list of localized entities/fields might be configurable.
* The process of configuring l10n/i18n might be softened - instead of a system-level flag (configuring translation for everything), it could be more case-by-case/opportunistic -- where you can optionally "attach" n-ary translations to any given field.https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/22CiviMail setting to use Google's smtp servers - this is not an issue but info...2020-03-30T02:42:20ZmarkdevarajCiviMail setting to use Google's smtp servers - this is not an issue but info if you are having issues with google smtpHello All
This is not an issue but info for others if you are having problems setting up CiviMail to send mail thru Google SMTP service.
We use a Gsuite account for our mail. We setup 3x email accounts on Gsuite to catch the bounce, a...Hello All
This is not an issue but info for others if you are having problems setting up CiviMail to send mail thru Google SMTP service.
We use a Gsuite account for our mail. We setup 3x email accounts on Gsuite to catch the bounce, an email account to send from and email to activity account.
After much trial and error and reading all the posts on CivMail to Google issues, the setting that worked for us is as follows:-
SMTP Server: ssl://smtp.gmail.com
SMTP Port: 465
Authentication : Yes
SMTP Username : xxxxx@melbournecameraclub.org.au
SMTP Password :xxxxxxx
we send under 500 emails weekly from a Civi mailing list.
Settings - for CiviMail
Mailer Batch limit = 100
Mailer throttle Time =1000000 (micro secs)
Mailer Job size =0
Mailer Cron Job limit =0
very separator = .
Enable custom reply to =no
Cron was setup to execute job.execute every 5 mins (*/5 * * * * etc)
Cron was setup to execute fetch bounces on the hour (0 * * * * etc)
We sent 460 emails in just under 25mins as mail was sent in 100 email batches every 5 mins. Speed of email delivery is not important for us as it is just a weekly newsletter.
Bounces was recorded in Civi reports correctly.
I hope that this helps someone.https://lab.civicrm.org/community/region/melbourne-meetups/-/issues/21Nextgen Gallery plugin on WP stops Civicrm Scheduled jobs from running2020-03-27T03:46:38ZmarkdevarajNextgen Gallery plugin on WP stops Civicrm Scheduled jobs from runningWe are running Civicrm 5.14.1 on Wordpress(WP) and have encountered that the latest Update of NextGen Gallery (v3.3.0) stops Civicrm Scheduled jobs from running. We were running NextGen Gallery without any issues until the latest upgrade...We are running Civicrm 5.14.1 on Wordpress(WP) and have encountered that the latest Update of NextGen Gallery (v3.3.0) stops Civicrm Scheduled jobs from running. We were running NextGen Gallery without any issues until the latest upgrade to v3.3.0.
Once we deactivated this plugin Civicrm Scheduled jobs started working again.
Has anyone else encountered this issue? Any suggested solutions?