Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • documentation/docs/dev
  • totten/dev
  • bgm/dev
  • ivan_compucorp/dev
  • seamuslee/dev
  • artfulrobot/dev
  • ufundo/dev
  • wmortada/dev
  • lucky091588/dev
  • DaveD/dev
  • jtwyman/dev
  • rukkykofi/dev
  • JonGold/dev
  • jaapjansma/developer-docs
  • alainb/dev
  • noah/dev
  • justinfreeman/dev
  • pradeep/dev
  • larssg/dev
  • eileen/dev
  • darrick/dev
  • mattwire/dev
  • colemanw/dev
  • homotechsual/dev
  • JoeMurray/dev
  • maynardsmith/dev
  • kurund/dev
  • rocxa/dev
  • AllenShaw/dev
  • bradleyt/dev
  • chrisgaraffa/dev
  • martin.w/dev
  • herbdool/dev
  • MattTrim1/dev
  • Detlev/dev
  • ErikHommel/dev
  • brienne/devdocs
  • pminf/dev
  • SarahFG/dev
  • ayduns/dev
  • JKingsnorth/dev
  • ginkgomzd/dev
  • nicol/dev
  • almeidam/dev
  • arthurm/dev
  • damilare/dev
  • semseysandor/dev
  • major/devdocs
  • usha.makoa/dev
  • yurg/dev
  • shaneonabike/dev
  • andie/dev
  • mmyriam/dev
  • gngn/dev
  • florian-dieckmann/dev
  • jade/dev
  • luke.stewart/dev
  • vinaygawade/dev
58 results
Show changes
Showing
with 1663 additions and 1008 deletions
# APIv4 Pseudoconstants (Option Lists)
Some fields in CiviCRM are linked to an option list (aka *Pseudoconstant*). These fields are typically shown to the user
as checkboxes, radios, or a dropdown-select of options from the pseudoconstant list.
The API provides tools for working with such fields; so that downstream developers do not need to translate between database values and option properties.
## Option list properties
Each option in the list consists of attributes like `id`, `name`, `label`, `description`, `color` and `icon`.
For a complete list and description of each attribute see the framework chapter: [Pseudoconstant (Option List) Reference](../../framework/pseudoconstant.md#attributes).
## Which fields have option lists?
To find out which fields have option lists for a particular entity, use the `getFields` action from the API or API Explorer, e.g.
```php
civicrm_api4('Contact', 'getFields', [
'where' => [['options', '!=', FALSE]],
])
```
This will return a list of all fields for a given entity (in this example, "Contact") which have an option list.
## Retrieving an option list {:#retrieve}
If you need to retrieve the full option list for a field, use the `getFields` action and set the `loadOptions` parameter
to an array of the above properties you would like to retrieve. Try it in the API explorer, e.g.
```php
civicrm_api4('Contact', 'getFields', [
'where' => [['name', '=', 'contact_type']],
'loadOptions' => ['name', 'label', 'description'],
])->first();
```
This will load information about the `contact_type` field including the list of available options (names will be ['Individual', 'Household', 'Organization'], labels will be localized translations if the site is not in English).
!!! tip
Unless you are building a form and need to render an option list for the user to select, in most cases
you will not need to query `getFields` to e.g. match option ids to labels. The syntax below can automatically
perform such transformations.
## Option transformations
When reading or writing data from the API, a special suffix tells the API to transform between option ids and other properties. For example:
### Writing
```php
Activity::create()
->addValue('activity_type_id:name', 'Meeting')
...
```
Using this syntax saves you from needing to look up the id of an activity type.
It also makes your code more portable, as option ids can change from one database to another, but names are consistent.
### Reading
Get operations use the same syntax to reverse the lookup and transform an option id to another property.
```php
Activity::get()
->addSelect('*', 'activity_type_id:label', 'activity_type_id:icon')
...
```
This tells the API to return all fields (`*`) plus the label and icon of the activity type, giving the data needed for a user-facing display.
!!! note "Option Lists or Joins"
The syntax for option transformations looks very similar to [implicit joins](implicit-joins.md) (the difference being a colon instead of a dot). Your choice of which to use depends on the field.
For example, the Contact `gender_id` field has an option list but not a foreign key, so you must use the option syntax (`gender_id:label`).
But the Event `campaign_id` field has a foreign key instead of an option list, so you would use a join (`campaign_id.title`).
\ No newline at end of file
# APIv4 REST
## Overview
<!--
FIXME: When the explorer is updated, enable the comments about it...
!!! tip "Get started with APIv4 Explorer"
CiviCRM has a built-in tool for working with APIv4 (`Support => Developer => APIv4 Explorer`). You may use
this to generate complete HTTP/REST examples that are tuned to your instance of CiviCRM.
This documentation provides a deeper explanation of the HTTP/REST interface.
-->
CiviCRM APIv4 provides HTTP/REST bindings, enabling remote applications to read and write data.
As you may recall, the APIv4 architecture uses the triplet [(`$entity`, `$action`, `$params`)](usage.md), e.g.
```php
$result = civicrm_api4($entity, $action, $params);
$result = civicrm_api4('Contact', 'get', ['limit' => 10]);
```
A typical APIv4 REST request includes these three elements and an authentication credential. For example:
```
https://{$website}/civicrm/ajax/api4/{$entity}/{$action}
?params={$params}&_authx={$credential}
```
In the remaining sections, we will examine this formula more closesly - considering more concrete examples
and more precise descriptions of each part.
??? question "Is APIv4 a REST-ful interface?"
HTTP-based APIs are sometimes described as "REST-ful" if they closely follow certain HTTP conventions. CiviCRM APIv4 is more REST-ful than APIv3, but it is not strictly REST-ful.
Never-the-less, it is generally used for reading and writing resource records, and (by tradition) the subsystem is referred to as "REST".
## Examples
As we review the structure of an APIv4 REST call, it may help to have a few examples. Here is a representative example:
```
1: GET https://www.example.org/civicrm/ajax/api4/Contact/get
2: ?params=%7B%22where%22%3A%5B%5B%22id%22%2C%22%3D%22%2C22%5D%5D%7D
3: &_authx=ABCD1234
```
In line 1, we see an *end-point (base) URL* (`https://www.example.org/civicrm/ajax/api4`) along with entity and action (`Contact/get`). Line 2 conveys the specific API data-parameters (encoded as JSON form-data). Line 3 provides the *authentication* parameters.
Similar elements should appear in any API call, but the content of each can differ. Here is another example:
```
1: POST https://crm.example.org/civicrm/ajax/api4/OptionGroup/get
2: Authorization: Basic dXNlcjpwYXNz
3: X-Requested-With: XMLHttpRequest
4:
5: %7B%22where%22%3A%5B%5B%22name%22%2C%22%3D%22%2C%22activity_type%22%5D%5D%2C%22limit%22%3A25%2C%22debug%22%3Atrue%7D
```
Here, we have a different *end-point (base) URL* (line 1; `https://crm.example.org/civicrm/ajax/api4`) with a different entity and action (`OptionGroup/get`). Line 2 shows a different *authentication* style. The specific API data also differs (line 5).
## End-Point URL
All requests should be submitted to the CiviCRM route `civicrm/ajax/api4/{ENTITY}/{ACTION}`.
Requests are typically submitted with HTTP POST, but read-only operations may use HTTP GET.
??? info "Comparison: URLs on Backdrop, Drupal, Joomla, WordPress"
On Drupal, Backdrop, and some WordPress deployments, you may access it with a clean URL.
```
https://example.org/civicrm/ajax/rest
```
In other deployments, you may need a verbose URL - such as these WordPress and Joomla URLs:
```
https://wordpress.example.org/wp-admin/admin.php?page=CiviCRM&q=civicrm/ajax/rest
https://joomla.example.org/administrator/index.php?option=com_civicrm&task=civicrm/ajax/rest
```
The `civicrm/ajax/rest` end-point is frequently used for browser-based API calls ("AJAX"). In newer versions (CiviCRM v5.36+), it can also be used for remote applications.
Stylisitcally, this is similar to [APIv4 REST](../v4/rest.md) end-point.
## Authentication
Every request for APIv4 should include authentication details. These may be submitted in a few forms, such as:
```
/* HTTP parameter */
?_authx=Bearer+MY_API_KEY
?_authx=Bearer+MY_JSON_WEB_TOKEN
?_authx=Basic+B64(USER:PASS)
/* Civi HTTP header */
X-Civi-Auth: Bearer MY_API_KEY
X-Civi-Auth: Bearer MY_JSON_WEB_TOKEN
X-Civi-Auth: Basic B64(USER:PASS)
/* Conventional HTTP header */
Authorization: Bearer MY_API_KEY
Authorization: Bearer MY_JSON_WEB_TOKEN
Authorization: Basic B64(USER:PASS)
```
!!! tip "Which authentication forms can I use?"
By default, new AuthX deployments are configured to accept:
* __[Credentials](../../framework/authx.md#credentials)__: API keys or JSON Web Tokens (*Passwords are not accepted by default.*)
* __[Flows](../../framework/authx.md#flows)__: `?_authx=`, `X-Civi-Auth:`, or `Authorization:`
The system administrator may enable or disable support for each credential and flow.
Some hosting environments may have compatibility issues with `Authorization:`. If you need to communicate with
several different CiviCRM deployments, then `?_authx=` and `X-Civi-Auth:` may be more reliable.
For API keys and passwords, the user must have the matching permission (`authenticate with api key` or
`authenticate with password`).
## X-Requested-With
To ensure broad compatibility, APIv4 REST clients should set this HTTP header:
```http
X-Requested-With: XMLHttpRequest
```
The header is not required for all requests, but sometimes it is required, and there is generally no harm to
setting it.
For more complete details, see [Cross-Site Request Forgery (CSRF)](../../security/csrf.md) and specifically
[CSRF: APIv4/APIv4 REST](../../security/csrf.md#rest).
## Parameters
APIv4 `$params` are transmitted with JSON and URL encoding. Thus, if the PHP `$params` are:
```php
['limit' => 10]
```
Then the equivalent JSON will be:
```javascript
{"limit":10}
```
and the URL-encoded JSON will be:
```
%7B%22limit%22%3A10%7D
```
## Response
The response will be a JSON document. A well-formed response provides a list of `values`:
```javascript
{
"version": 4,
"count": 10,
"values": [
{
"contact_id": "1",
"contact_type": "Individual",
"contact_sub_type": "",
"display_name": "Example Person",
...
```
Alternatively, if there is an error, then it will be reported with `error_message` and `error_code`:
```javascript
{
"error_code": 0,
"error_message": "Api FakeEntity fakeAction version 4 does not exist.",
}
```
## History
* __CiviCRM v5.19__: Added HTTP bindings for APIv4 to `civicrm-core`. However, these were only available for
browser-based AJAX applications -- not for remote REST applications.
* __CiviCRM v5.36__: Added the [AuthX extension](../../framework/authx.md) to `civicrm-core`. This enabled REST-style
authentication for APIv4. However, configuring AuthX may require CLI interaction to update hidden settings.
* __CiviCRM v5.47__: Added the AuthX configuration screen, making it easier for more sysadmins to setup.
# APIv4 SET Operations
## Background
SQL **Set Operations** are ways of combining the results of multiple queries.
- **UNION ALL**: Combines the results of two or more SELECT statements into a single result set, eliminating duplicate rows.
- **UNION DISTINCT**: A UNION which eliminates duplicate rows.
- **INTERSECT**: Returns the common elements between two or more SELECT statements, resulting in a result set that contains only the shared rows.
- **EXCEPT**: Retrieves the rows from the first SELECT statement that are not present in the result set of the second SELECT statement, effectively finding the difference between the two result sets.
As of version 5.64, APIv4 adds support for UNION ALL and UNION DISTINCT. It does not yet support INTERSECT or EXCEPT because
the minimum version of MySql would need to be bumped to 8.0. Once that's done in a future version, those operations can be
supported as well.
## The `EntitySet` API
By calling the `EntitySet` API and adding 2 or more sets to combine, you effectively create a new entity/table which
can be operated on as a whole (SELECT, WHERE, HAVING, ORDER BY, LIMIT).
For example, you could UNION together a set of Contributions and Contribution Soft Credits,
then apply a GROUP BY clause to the combined results.
!!! warning Columns Must Match
Per the rules of SQL Set Operations, each set must have the same number of fields selected,
with matching data types.
They will be combined in that order (not by name) and the field names of the first set will be used for the entire EntitySet.
Try it yourself in the API Explorer:
![APIv4 Explorer EntitySet](../../img/api4/entityset-unions.png)
\ No newline at end of file
# APIv4 Usage
## API Input
Every API call consists of three elements: the *entity*, *action*, and *parameters*:
**Entity:** The "name" of the API.
CiviCRM entities include Contacts, Activities, Contributions, Events, etc.
Each API entity usually (but not always) corresponds to a table in the database (e.g. the Contact entity is the `civicrm_contact` table).
!!! info "Supported Entities"
The ongoing effort to port APIv3 entities to v4 is nearing completion.
For updates or to help port a new entity, see this [tracking issue](https://lab.civicrm.org/dev/core/-/issues/2486).
**Action:** The "verb" of the API. The [list of available actions](actions.md) varies for each entity, but most support
a set of CRUD actions to `create`, `get`, `update` & `delete`, etc.
**Parameters:** Settings or data to pass to the api function. Each action accepts a different set of parameters.
Consider these samples with commonly-used actions and parameters for the `Contact` entity:
| Action | Parameters | Description |
|----------|--------------------------------------|---------------------------------|
| `create` | `#!json {"values":{"contact_type":"Individual", "first_name":"Jane", "last_name":"Doe"}}` | Create a new contact of type "Individual" with first name "Jane" and last name "Doe" |
| `get` | `#!json {"where":[["last_name", "=", "Doe"]], "limit":25}` | Fetch the first 25 contacts with the last name "Doe" |
| `delete` | `#!json {"where":[["id", "=", 42]]}` | Delete the contact with id "42" |
!!! tip "API Explorer"
For full, up-to-date details about specific entities, actions and parameters, use the [API Explorer](../index.md#api-explorer).
## API Output
The API returns an array, with each item representing one record (also an array).
In PHP, this output is contained in an ArrayObject, which can be accessed like an array using e.g. `$result[0]` or `foreach ($result as ...)`.
See the [php docs](https://www.php.net/manual/en/class.arrayobject.php) for a full list of methods ArrayObjects support
It also has the following methods in addition to or overriding the base ArrayObject:
- `$result->first()`: returns the first item, or NULL if not found.
- `$result->single()`: returns a single item, or throws an exception if not exactly one item was returned.
- `$result->last()`: returns the last item, or NULL if not found.
- `$result->itemAt($index)`: returns the item at a given index, or NULL if not found.
- `$result->indexBy($field)`: reindexes the array by the value of a field.
- `$result->column($field)`: flattens the array, discarding all but a single field (call this second if using in conjunction with `indexBy` to create a key=>value array).
- `$result->count()`: get the number of fetched (if LIMIT used) or matching results (if `row_count` included in select). See below.
- From CiviCRM 5.50+ `$result->countFetched()`: get the number of *fetched* results. If you used LIMIT this won't exceed that number (but could be less).
- From CiviCRM 5.50+ `$result->countMatched()`: get the total number of *matching* results. If you only fetched a subset (with LIMIT) you are required to add `row_count` to the select clause.
!!! Converting ArrayObject to a Real Array
If you need to use functions such as `is_array` on the result you can cast it you cast it to an array, `(array) $result`, or use the [`getArrayCopy()`](https://www.php.net/manual/en/arrayobject.getarraycopy.php) method.
### Counting results
Prior to 5.50, there was just one count available via `count()`. From 5.50+ it is clearer to use one of the new methods instead: `countFetched()` and `countMatched()` because `count()` can return different counts based on a number of factors (see below). Note that the `rowCount` *property* should not be considered a public interface and any code using it directly should be migrated to use a method instead.
**TL;DR**: If you need to know the number of *matched* rows regardless of any fetch limit, you must add `row_count` to the select. You can do this by calling `selectRowCount()` (note that this will replace any prior selects).
e.g. consider the database contains 100 contacts and you're using the `Contact.get` API.
`$result = $api->addSelect('display_name')->execute()` will fetch the names of all contacts.
- `$result->countFetched()` will return 100.
- `$result->countMatched()` will return 100 too, since this is known as there was no limit.
- `$result->count()` will return 100.
`$result = $api->addSelect('display_name')->setLimit(10)->execute()` will fetch the names of 10 contacts.
- `$result->countFetched()` will return 10.
- `$result->countMatched()` will return throw an exception because it cannot be calculated from the result.
- `$result->count()` will return 10.
`$result = $api->selectRowCount()->addSelect('display_name')->setLimit(10)->execute()` will fetch the names of 10 contacts and also calculate the total matched.
- `$result->countFetched()` will return 10.
- `$result->countMatched()` will return 100.
- `$result->count()` will return 100.
`$result = $api->selectRowCount()->execute()` will not fetch any records, but will make the count available:
- `$result->countFetched()` will return 0 (since we did not fetch any data rows).
- `$result->countMatched()` will return 100.
- `$result->count()` will return 100.
`$result = $api->selectRowCount()->setLimit(10)->execute()` is a funny thing to do (set a limit for records without asking for records anyway) but it will behave as if you had not included the `setLimit()` clause, i.e. exactly as the last example.
`$result = $api->selectRowCount()->addSelect('display_name')->execute()` is strictly unnecessary since it fetches all records anyway, so the `selectRowCount()` is superfluous.
- `$result->countFetched()` will return 10.
- `$result->countMatched()` will return 100.
- `$result->count()` will return 100.
## Example
<iframe src="https://learn.civi.be/wp-admin/admin-ajax.php?action=h5p_embed&id=6" width="680" height="480" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
## API Security
By default, APIv4 validates every action according to the permission level of the logged-in user.
This can be disabled only when calling via [PHP code](usage.md#php). API Permissions are also able to be altered via hook.
More information on API Security can be found in the [Security Documentation](../../security/permissions.md).
## API Interfaces
The API is available in many environments (such as PHP, CLI, and JavaScript), and the notation differs slightly in each language.
However, if you understand the canonical notation, then others will appear as small adaptations.
!!! note "Use Outside of CiviCRM"
If you're writing a Drupal module, a Joomla extension, a WordPress plugin, or a standalone script, then you may need to [bootstrap](../../framework/bootstrap.md) CiviCRM before using the API.
### PHP
This is the canonical API; all other environments are essentially wrappers around the PHP API.
There are two ways to call the api from PHP - which one you choose is a matter of convenience and personal preference.
For example, you may prefer OOP syntax because IDE code editors provide autocompletion.
Or if you need to work with the parameters as an array, traditional syntax will be more convenient.
![APIv4 PHP Examples](../../img/api4/Api4-PHP-Styles.svg)
#### Traditional (Procedural)
*The function `civicrm_api4($entity, $action, [$params], [$index])` accepts an array of parameters and returns the Result.*
```php
$result = civicrm_api4('Contact', 'get', [
'where' => [
['last_name', '=', 'Adams'],
// Option transformation.
['gender_id:name', '=', 'Male'],
// Implicit join.
['employer_id.is_opt_out', '=', FALSE],
],
'limit' => 25,
]);
```
`$index` provides a convenient shorthand for reformatting the Result array. It has different modes depending on the variable type passed:
* **Integer:** return a single result array; e.g. `$index = 0` will return the first result, 1 will return the second, and -1 will return the last.
* **String:** index the results by a field value; e.g. `$index = "name"` will return an associative array with the field 'name' as keys.
* **Non-associative array:** return a single value from each result; e.g. `$index = ['title']` will return a non-associative array of strings - the 'title' field from each result.
* **Associative array:** a combination of the previous two modes; e.g. `$index = ['name' => 'title']` will return an array of strings - the 'title' field keyed by the 'name' field.
#### Object-Oriented (OOP)
*An `Action` class provides setter methods for each parameter. The `execute()` method returns the Result.*
```php
$result = \Civi\Api4\Contact::get()
->addWhere('last_name', '=', 'Adams')
->addWhere('gender_id:name', '=', 'Male')
->setLimit(25)
->execute();
```
!!! warning "Warning: Class names don't always match entity names!"
For example, the entity "Case" is a reserved word in PHP and therefore could not be the name of a class, so it had to be named something else ("CiviCase").
Internally, calling `civicrm_api4('Case', 'getFields')` will translate to `Civi\Api4\CiviCase::getFields`.
[Custom data entities](custom-data.md) are another example.
When writing generic functions that work with multiple entities, we strongly recommend against clever code like this: `call_user_func(['\Civi\Api4\' . $entity, 'get'], ...)`,
and suggest using this more reliable alternative: `civicrm_api4($entity, 'get', ...)`.
### AJAX
The AJAX interface is automatically available for web-pages generated through CiviCRM (such as standard CiviCRM web-pages, CiviCRM extensions and custom CiviCRM templates).
Inputs are identical to the traditional PHP syntax:
```javascript
CRM.api4('entity', 'action', [params], [index])
```
From an Angular app, use the service `crmApi4()` which has an identical signature but works within the `$scope.digest` lifecycle.
Both functions return a Promise, which resolves to a Result array.
```javascript
CRM.api4('Contact', 'get', {
where: [
['last_name', '=', 'Adams'],
],
limit: 25
}).then(function(results) {
// do something with results array
}, function(failure) {
// handle failure
});
```
!!! note "Use on Non-CiviCRM Pages"
The AJAX interface could be made available to other parts of the same website (e.g. a Drupal module or WordPress widget) by calling `#!php Civi::resources()->addCoreResources()`
from PHP. Please note that the AJAX interface is subject to [API Security](../../security/permissions.md#api-permissions)
and [Same Origin Policy](http://en.wikipedia.org/wiki/Same_origin_policy).
### Command line
#### cv
`cv` supports multiple input formats for APIv4. The API Explorer uses the JSON format in generated code:
```bash
cv api4 Contact.get '{"where":[["first_name", "=", "Alice"], ["last_name", "=", "Roberts"]]}'
```
This format may be cumbersome to enter manually, so the same API request could also be written like this:
```bash
cv api4 Contact.get +w 'first_name = "Alice"' +w 'last_name = "Roberts"'
```
For more examples, refer to the output of `cv --help api4`.
#### drush
Adding `version=4` to a drush command will cause v4 api to be called rather than v3.
#### wp-cli
APIv4 is not yet available as a wp-cli command.
### Smarty
Because Smarty is an older technology, there are no plans to port the APIv3 smarty function to APIv4.
### Scheduled jobs
APIv4 is not yet available for scheduled jobs.
# CiviCRM Developer Community
Developers from around the globe use the following systems to communicate and
collaborate:
## Collaboration tools
Developers from around the globe use the following systems to communicate and collaborate:
- [CiviCRM.org](https://civicrm.org)
- [Extensions](https://civicrm.org/extensions) publishing
- [Blog posts](https://civicrm.org/blog/) are written by both the CiviCRM core team and other community members, and cover a wide range of topics. Subscribe to the [RSS feed](https://civicrm.org/blog/feed) to make sure you stay up to date.
- [Events](https://civicrm.org/events) -
meetups, conferences, camps, sprints, webinars
- [Events](https://civicrm.org/events) - meetups, conferences, camps, sprints, webinars
- [Job Board](https://civicrm.org/jobs) - Post and find paid work
- [Documentation](https://docs.civicrm.org)
- [Mattermost](https://chat.civicrm.org) offers live discussion in chat rooms and direct messages. It's a great place to go for any technical questions. Here, with a bit of luck, you'll often find an instant answer to your question or a pointer in the right direction. Definitely go here if you feel like you are banging your head against a wall.
- [Mattermost](https://chat.civicrm.org) offers live discussion in chat rooms and direct messages. It's a great place to go for any technical questions. Here, with a bit of luck, you'll often find an instant answer to your question or a pointer in the right direction. Definitely go here if you feel like you are banging your head against a wall. New developers can check out the [dev-newcomers](https://chat.civicrm.org/civicrm/channels/dev-newcomers) channel for discussion on getting started with developing for CiviCRM.
- [Stack Exchange](http://civicrm.stackexchange.com/) - Question & answer site covering a range of topics relevant to developers and users. It is a great place to find answers to the millions of questions that have already been asked about CiviCRM, and to ask ones that haven't been asked already.
- [Jira](https://issues.civicrm.org/jira) - Issue tracking
- [GitLab](https://lab.civicrm.org) - CiviCRM's tool for issue tracking community-based project management.
- [GitHub](https://github.com/civicrm) - Hosted git repositories for the entire CiviCRM codebase, as well as many smaller tools focused on developing CiviCRM.
- [Discussion Mailing Lists](https://lists.civicrm.org/lists/)
- [Newsletters](https://civicrm.org/civicrm/mailing/subscribe)
- Falling out of use
- [Wiki] - still used for specs & recipes, *but documentation
[is moving][migration] to the guides listed above*
- [IRC](http://irc.civicrm.org/) - #civicrm on irc.freenode.net
*but now mostly replaced by Mattermost*
- [Forum](https://forum.civicrm.org/) - *now mostly replaced by
StackExchange*
[wiki]: https://wiki.civicrm.org/confluence/display/CRMDOC/CiviCRM+Documentation
[migration]: https://wiki.civicrm.org/confluence/display/CRMDOC/Content+migration+from+wiki+to+Developer+Guide
The developer community is full of friendly people, eager to welcome newcomers.
Mattermost and in-person events are great starting points if you're looking to
get involved!
!!! tip "Before you begin"
Register for the accounts below. *All accounts require e-mail verification.*
1. [CiviCRM.org](https://civicrm.org/user/register) - Be prepared to give your name and organizational affiliation, and explain your role in the community.
*Account requests must be approved by an administrator before you will be granted access.* This process may take some time.
The following communities provide a single sign-on once your CiviCRM.org account is approved.
- GitLab
2. [Mattermost](https://chat.civicrm.org/signup_email) - Create an account and join a team on this open-source version of Slack.
3. [Stack Exchange](https://civicrm.stackexchange.com/users/signup?ssrc=head&returnurl=https%3a%2f%2fcivicrm.stackexchange.com%2f) - Read the user guide after joining. Write your own Q&A and vote on others'.
4. [GitHub](https://github.com/join?source=header) - Find repositories under CiviCRM's group after confirming your account.
!!! warning "Don't include a space in your username"
CiviCRM.org allows spaces in usernames but this won't work with GitLab. So it's best if you don't use any spaces in your CiviCRM.org username.
If you have already set up an account on CiviCRM.org you can easily log in and change your username.
## Discussion communities
Optional lists you may also want to register for:
- [Newsletters](https://civicrm.org/civicrm/mailing/subscribe) (after your CiviCRM account is active)
## Falling out of use
- [CiviCRM Groups](https://lists.civicrm.org/lists/firstpasswd/) - (listservs) *but now mostly replaced by Mattermost*
## Out of use
- [IRC](http://irc.civicrm.org/) - #civicrm on irc.freenode.net *No longer officially supported by the CiviCRM organization, and functionally replaced by Mattermost*
- [Wiki](https://wiki.civicrm.org/confluence/display/CRMDOC/CiviCRM+Documentation) - *Archived in November 2017, but still useful for some older documentation. Developer and admin docs have been moved to [MkDocs](https://docs.civicrm.org/). Other documentation should be moved to their respective [Gitlab wiki spaces](https://lab.civicrm.org/explore/groups).*
- [Forum](https://forum.civicrm.org/) - *Archived on November 25, 2017*
- [Jira](https://issues.civicrm.org) - *Archived on December 27, 2018. You can still access old (read-only) JIRA issues if you know their issue number and you can also browse the archived issues for a specific project.*
The developer community is full of friendly people, eager to welcome newcomers. Mattermost and in-person events are great starting points if you're looking to get involved!
......@@ -2,110 +2,59 @@
## Community input
Before you start on any code to extend CiviCRM, it is really important
to discuss your ideas with the community. Here are a few of the reasons
why this is a good idea:
- It may have been done already
- You'll get suggestions and advice on suitable ways to approach the
problem
- Other people will know what you are doing, and be able to contact
you if they want to collaborate
A typical pre-development workflow will start with a discussion on
[Mattermost](https://chat.civicrm.org/) (in the Developer channel) about what
you want to do. Here you'll receive feedback from other members of the
community and core team about your ideas. You might be lucky and find
out that there is a already a way to do what you want using the user
interface (and that no coding is necessary). Or it might be that someone
has done something similar already and all that is required is a few
tweaks to their code.
Before you start on any code to extend CiviCRM, it is really important to discuss your ideas with the community. Here are a few of the reasons why this is a good idea:
- It may have been done already.
- You'll get suggestions and advice on suitable ways to approach the problem.
- Other people will know what you are doing, and be able to contact you if they want to collaborate.
A typical pre-development workflow will start with a discussion on [Mattermost](https://chat.civicrm.org/) (in the Developer channel) about what you want to do. Here you'll receive feedback from other members of the community and core team about your ideas. You might be lucky and find out that there is a already a way to do what you want using the user interface (and that no coding is necessary). Or it might be that someone has done something similar already and all that is required is a few tweaks to their code.
## Requirements and specifications
If and when you have confirmed that some coding is required, it is good
practice, even for relatively small projects, to write
- a requirements document which describes in detail what you want the
finished code to do
- a specification that outlines how you are going to meet these
requirements with CiviCRM
The requirements are typically written to be understandable to end
users, and the specification can be thought of as a translation of those
requirements into the language of CiviCRM. Both requirements and
specification should go on the
[wiki](http://wiki.civicrm.org/confluence/display/CRM/CiviCRM+Wiki).
Once you've written the requirements and specification document, you
should go about soliciting feedback. Get feedback on the requirements
from your end users and feedback on the requirements and the
specification from anyone that is willing to comment. To encourage more
discussion, you can write a post on CiviCRM's
[blog](https://civicrm.org/blog/), tweet it out with
the [#civicrm](https://twitter.com/hashtag/civicrm) hashtag, tell similar
CiviCRM users and organisations and so on.
The more feedback you can get the better.
If and when you have confirmed that some coding is required, it is good practice, even for relatively small projects, to write:
- a requirements document which describes in detail what you want the finished code to do, and
- a specification that outlines how you are going to meet these requirements with CiviCRM.
The requirements are typically written to be understandable to end users, and the specification can be thought of as a translation of those requirements into the language of CiviCRM.
Store your requirements and specifications in a public location such as:
* a markdown file in your project's repository,
* a google doc, or
* the [CiviCRM wiki](http://wiki.civicrm.org/confluence/display/CRM/CiviCRM+Wiki).
Once you've written the requirements and specification document, you should go about soliciting feedback. Get feedback on the requirements from your end users and feedback on the requirements and the specification from anyone that is willing to comment. To encourage more discussion, you can write a post on CiviCRM's [blog](https://civicrm.org/blog/), tweet it out with the [#civicrm](https://twitter.com/hashtag/civicrm) hashtag, tell similar CiviCRM users and organisations and so on. The more feedback you can get the better.
!!!tip
If you get stuck writing your requirements and specification, or would
like to get more background, have a look at some
[existing requirements and specifications](https://wiki.civicrm.org/confluence/display/CRM/Requirements+and+specifications)
from CiviCRM developers.
If you get stuck writing your requirements and specification, or would like to get more background, have a look at some [existing requirements and specifications](https://wiki.civicrm.org/confluence/display/CRM/Requirements+and+specifications) from CiviCRM developers.
## Bugs
Before starting work on a bug, your first step should be to check the [issue tracking systems](../tools/issue-tracking.md) for any open issues before creating one yourself or working on a pull request.
In order to reproduce the bug you can reproduce the issue in the appropriate [CiviCRM Sandbox](https://civicrm.org/sandboxes). Enabling debugging can help to get more details.
Make sure to check [the contribution guidelines](../core/contributing.md) for more information on how to create a pull request.
## Recommendations
**Open a free [GitHub](https://github.com/) account** for revision control.
The official CiviCRM [repositories](https://github.com/civicrm)
are housed in `git` repositories on GitHub. If you use GitHub you will find
it easy to access the latest source-code, to submit pull requests
for any patches you create and to collaborate with many other
CiviCRM developers who also use GitHub.
**Install the [buildkit](https://github.com/civicrm/civicrm-buildkit)**,
ideally as a [vagrant virtual-machine](https://github.com/civicrm/civicrm-buildkit-vagrant)
or using one of available `docker` images ([1](https://github.com/progressivetech/docker-civicrm-buildkit),
[2](https://github.com/ErichBSchulz/dcbk)). The buildkit is not an absolute
requirement but it is definitely the fastest path to a good development
experience!
**From the outset, [automate testing](https://wiki.civicrm.org/confluence/display/CRMDOC/Testing)**.
In the current climate of rapid evolution of not just CiviCRM, but
also it's myriad of dependancies, automated testing of PHP code with `phpunit`
and javascript with tools like `karma` and `jasmine` is essential. Start all
your work by considering how you will provide automated testing for it.
Starting with the buildkit will make this much simpler for you to set up.
Getting started with unit-testing may seem daunting and onerous when you start,
but you will soon come to love the freedom it gives you. If you are
unsure how to proceed with testing ask the [community](/basics/community).
**Create a native [extension](/extensions/basics)**.
If you have new functionality to add to CiviCRM, it probably belongs in an
extension. "Native" extensions will install into all CiviCRM sites
regardless of the underlying CMS used (Drupal or Wordpress), making it easy to
share your extension with the CiviCRM community.
**Use the [API](/api/general) and [hooks](/hook)** to access and manage CiviCRM data in any patch,
native extension, CMS module, or external program that you
develop. The API will function as expected with every new release
and backwards compatibility of the API is maintained for several
versions of CiviCRM.
**Avoid [hacking the core](/core/hacking)** of CiviCRM unless you understand
the implications.
**Follow the
[Coding Standards](https://wiki.civicrm.org/confluence/display/CRMDOC/Coding+Standards)**
for uniform structure that will make everyone's development work easier.
**Open a free [GitHub](https://github.com/) account** for version control. The official CiviCRM [repositories](https://github.com/civicrm) are housed in `git` repositories on GitHub. If you use GitHub you will find it easy to access the latest source-code, to submit pull requests for any patches you create and to collaborate with many other CiviCRM developers who also use GitHub.
**Install the [buildkit](https://github.com/civicrm/civicrm-buildkit)**, ideally as a [vagrant virtual-machine](https://github.com/civicrm/civicrm-buildkit-vagrant) or using one of available `docker` images ([progressivetech](https://github.com/progressivetech/docker-civicrm-buildkit) or [EricBSchulz](https://github.com/ErichBSchulz/dcbk)). The buildkit is not an absolute requirement but it is definitely the fastest path to a good development experience!
**From the outset, [automate testing](../testing/index.md)**. In the current climate of rapid evolution of not just CiviCRM, but also it's myriad of dependencies, automated testing of PHP code with `phpunit` and javascript with tools like `karma` and `jasmine` is essential. Start all your work by considering how you will provide automated testing for it. Starting with the buildkit will make this much simpler for you to set up. Getting started with unit-testing may seem daunting and onerous when you start, but you will soon come to love the freedom it gives you. If you are unsure how to proceed with testing ask the [community](community.md).
**Create a native [extension](../extensions/index.md)**. If you have new functionality to add to CiviCRM, it probably belongs in an extension. "Native" extensions will install into all CiviCRM sites regardless of the underlying CMS used (Drupal, WordPress, Joomla or Backdrop), making it easy to share your extension with the CiviCRM community.
**Use the [API](../api/index.md) and [hooks](../hooks/index.md)** to access and manage CiviCRM data in any patch, native extension, CMS module, or external program that you develop. The API will function as expected with every new release and backwards compatibility of the API is maintained for several versions of CiviCRM.
**Avoid [hacking the core](../core/hacking.md)** of CiviCRM unless you understand the implications.
**Follow the [Coding Standards](../standards/index.md)** for uniform structure that will make everyone's development work easier.
## Make it happen
If you need or would like extra resources for your code improvement, you
might consider a [Make It Happen](https://civicrm.org/make-it-happen)
(aka MIH) campaign, which is a crowd-sourcing initiative for CiviCRM.
The MIH work is carried out by the
core team or trusted consultants, and has helped build many amazing new
features into CiviCRM in recent years.
If you need or would like extra resources for your code improvement, you might consider a [Make It Happen](https://civicrm.org/make-it-happen) (aka MIH) campaign, which is a crowd-sourcing initiative for CiviCRM. The MIH work is carried out by the core team or trusted consultants, and has helped build many amazing new features into CiviCRM in recent years.
# Development requirements
## Languages and Services
* Required
- Unix-like environment (Linux, OS X, or a virtual machine)
- [PHP v7.2+](http://php.net/) including the following extensions: `bcmath curl gd gettext imap intl imagick json mbstring openssl pdo_mysql phar posix soap zip`
- [MySQL v5.7.5+](http://mysql.com/) or [MariaDB 10.0.2+](https://mariadb.org/), including both client and server
- [NodeJS v8+](https://nodejs.org/)
- [Git](https://git-scm.com/)
* Recommended (for `civibuild` / `amp`):
- Apache HTTPD v2.2 or v2.4 including the `mod_rewrite` module and, on SUSE, possibly `mod_access_compat` (This list may not be exhaustive.)
Please also see [this Installation Guide page](https://docs.civicrm.org/installation/en/latest/general/requirements/) for more details on server configuration requirements to deploy CiviCRM installations using buildkit.
## Command Line
There are many ways to install MySQL, PHP, and other dependencies -- for example, `apt-get` and `yum` can download packages automatically; `php.net` and `mysql.com` provide standalone installers; and MAMP/XAMPP provide bundled installers.
Civi development should work with most packages -- with a priviso: ***the command-line must support standard command names*** (eg `git`, `php`, `node`, `mysql`, `mysqldump`, etc).
Some environments (e.g. most Linux distributions) are configured properly out-of-the-box. Other environments (e.g. MAMP and XAMPP) may require configuring the `PATH`.
<!-- FIXME: There should be a link about diagnosing/fixing paths for third-party binaries. TLDR: `find / -name php -executable` and then update `PATH` via bashrc/bash_profile/whatever -->
## Buildkit
The developer docs reference a large number of developer tools, such as `drush` (the Drupal command line), `civix` (the CiviCRM code-generator), and `karma` (the Javascript tester).
Many of these tools are commonly used by web developers, so you may have already installed a few. You could install all the tools individually -- but that takes a lot of work.
[civicrm-buildkit](../tools/buildkit.md) provides a script which downloads the full collection.
......@@ -2,88 +2,41 @@
## Learn how to *use* CiviCRM
Before diving into CiviCRM development, it's worth mentioning that a solid
understanding of CiviCRM from the *user's* perspective can really help. You
pick up the standard user interface patterns that you can re-use in your work,
and you might even realise that the UI already provides the functionality you
wish to develop. To that end, please see the
[User Guide](https://docs.civicrm.org/user/en/stable/).
Before diving into CiviCRM development, it's worth mentioning that a solid understanding of CiviCRM from the *user's* perspective can really help. You pick up the standard user interface patterns that you can re-use in your work, and you might even realise that the UI already provides the functionality you wish to develop. To that end, please see the [User Guide](https://docs.civicrm.org/user/en/stable/).
## Learn these developer skills
Below we've outlined the major technologies that CiviCRM is built with.
You don't need to be a pro at everything listed here to get started, but it's
useful to understand at least of couple of them well, and have a basic
understanding of how they fit together to create CiviCRM.
Below we've outlined the major technologies that CiviCRM is built with. You don't need to be a pro at everything listed here to get started, but it's useful to understand at least of couple of them well, and have a basic understanding of how they fit together to create CiviCRM.
Technologies which are contained within the CiviCRM project
(e.g. civix, buildkit) are covered in detail within this guide, but
other technologies (such as PHP and Git, which are not CiviCRM-specific) are
outside its scope. As such, this guide assumes that readers
arrive with a baseline understanding of these developer skills. This page lists
these prerequisite skills, along with pointers to usher the reader towards
appropriate resources for closing skills gaps, wherever necessary.
Items listed towards the top of this list are, generally-speaking, more
important skills for CiviCRM development, but the specific skills needed to
accomplishing a particular development goal, certainly vary.
Technologies which are contained within the CiviCRM project (e.g. civix, buildkit) are covered in detail within this guide, but
other technologies (such as PHP and Git, which are not CiviCRM-specific) are outside its scope. As such, this guide assumes that readers arrive with a baseline understanding of these developer skills. This page lists these prerequisite skills, along with pointers to usher the reader towards appropriate resources for closing skills gaps, wherever necessary. Items listed towards the top of this list are, generally-speaking, more important skills for CiviCRM development, but the specific skills needed to accomplishing a particular development goal, certainly vary.
- **PHP** - the main programming language in which CiviCRM is written
- **PHP** - the main programming language in which CiviCRM is written.
- [Language reference](http://php.net/manual/en/langref.php)
- **Git** - a version control system for tracking changes to source code
- [Official documentation](https://git-scm.com/documentation)
- [15 minute interactive tutorial](https://try.github.io/levels/1/challenges/1)
- [Another site with more interactive tutorials](http://learngitbranching.js.org/)
- **Command line / bash** - in general, "the command line" refers to using a
text-only interface to run programs such as `civix`, `git`, and many more.
Bash is the most common "shell" program used to execute these commands on
Unix computers (i.e. Linux and OS X).
- **Git, GitHub, and GitLab** - Git is a version control system for tracking changes to source code. GitHub and GitLab are web-based tools which host git repositories and issue-tracking systems.
- See [our recommended external resources](../tools/git.md#resources)
- **Command line / bash** - in general, "the command line" refers to using a text-only interface to run programs such as `civix`, `git`, and many more. Bash is the most common "shell" program used to execute these commands on Unix computers (i.e. Linux and OS X).
- [Unix command line tutorial](http://www.ee.surrey.ac.uk/Teaching/Unix/)
- *Microsoft Windows has a command line shell which functions very
differently from bash. While developing CiviCRM on Windows is possible,
it will be a slightly more uphill battle, for this reason and others.*
- **Javascript** - another programing language used in CiviCRM, especially
for any logic that happens within the web browser. *(Note that "javascript"
and "java" are entirely different technologies and should not be confused.)*
- *Microsoft Windows has a command line shell which functions very differently from bash. While developing CiviCRM on Windows is possible, it will be a slightly more uphill battle, for this reason and others.*
- **Javascript** - another programing language used in CiviCRM, especially for any logic that happens within the web browser. *(Note that "javascript" and "java" are entirely different technologies and should not be confused.)*
- [Javascript tutorial](http://www.w3schools.com/js/default.asp)
- **jQuery** - a javascript library that makes manipulating elements on a web
page easy
- **jQuery** - a javascript library that makes manipulating elements on a web page easy.
- [jQuery documentation](http://api.jquery.com/)
- **angularJS** - a front-end javascript/html library providing a client-side framework
for creating one-page MVC (model-view-controller) and MVVM (model-view-viewmodel) apps
- [angularJS documentation](https://angular.io/docs/ts/latest/)
- **AngularJS** - a front-end javascript/html library providing a client-side framework for creating one-page MVC (model-view-controller) and MVVM (model-view-viewmodel) apps.
- [angularJS documentation](https://docs.angularjs.org/)
- **HTML** - the markup used so transmit page content to a web browser.
- [HTML tutorial](http://www.w3schools.com/html/default.asp)
- **Smarty** - a "template engine" which allows developers to write an HTML
file for one web page, while easily including content dynamically generated
from PHP
- **Smarty** - a "template engine" which allows developers to write an HTML file for one web page, while easily including content dynamically generated from PHP.
- [Smarty documentation](http://www.smarty.net/docs/en/)
- **CSS** - a programming language used to specify consistent visual style to
be applied to many different elements within a web page. Different web
browsers interpret the CSS code in slightly different ways.
- **CSS** - a programming language used to specify consistent visual style to be applied to many different elements within a web page. Different web browsers interpret the CSS code in slightly different ways.
- [CSS tutorial](http://www.w3schools.com/css/default.asp)
- [Can I use](http://caniuse.com/) - good for seeing which web browsers
have implemented certain CSS features
- [Comparison of layout engines](https://en.wikipedia.org/wiki/Comparison_of_layout_engines_\(Cascading_Style_Sheets\))
another helpful comparison of the differences between web browsers
- **Drupal / Wordpress / Joomla!** - CiviCRM must be installed within one of
these content management systems, and learning more about the underlying
CMS will aid CiviCRM development. Drupal is favored by most CiviCRM
developers and CiviCRM actually borrows many development practices from
the project, so learning Drupal is a good place to start if you are unsure.
- [Can I use](http://caniuse.com/) - good for seeing which web browsers have implemented certain CSS features.
- [Comparison of layout engines](https://en.wikipedia.org/wiki/Comparison_of_layout_engines_\(Cascading_Style_Sheets\)) another helpful comparison of the differences between web browsers.
- **Drupal / WordPress / Joomla!** - CiviCRM must be installed within one of these content management systems, and learning more about the underlying CMS will aid CiviCRM development. Drupal is favored by most CiviCRM developers and CiviCRM actually borrows many development practices from the project, so learning Drupal is a good place to start if you are unsure. That said, WordPress is the most popular CMS and it has been so for several years now.
- [Drupal documentation](https://www.drupal.org/docs/)
- [Wordpress documentation]()
- [Joomla documentation]()
- **SQL / MySQL** - "SQL" is a standardized language used by many different
kinds of databases to manipulate data in the database. "MySQL" is one kind
of database which CiviCRM uses to store all its data. The query syntax
that MySQL uses conforms [almost](http://troels.arvin.dk/db/rdbms/)
entirely to the SQL standard, so learning SQL is basically synonymous to
learning MySQL.
- [WordPress documentation](https://codex.wordpress.org/Main_Page)
- [Joomla documentation](https://docs.joomla.org/)
- **SQL / MySQL** - "SQL" is a standardized language used by many different types of database management systems to manipulate data in the database. "MySQL" is one specific database management system which, by default, CiviCRM uses to store all its data. The query syntax that MySQL uses conforms [almost](http://troels.arvin.dk/db/rdbms/) entirely to the SQL standard, so learning SQL is basically synonymous to learning MySQL.
- [SQL tutorial](http://www.w3schools.com/sql/default.asp)
- [MySQL statement syntax](http://dev.mysql.com/doc/refman/en/sql-syntax.html)
- [MySQL Workbench](http://www.mysql.com/products/workbench/) -
an intuitively designed GUI tool for inspecting and interacting with a
MySQL database (great for learning more about the CiviCRM data model).
- [MySQL Workbench](http://www.mysql.com/products/workbench/) - an intuitively designed GUI tool for inspecting and interacting with a MySQL database (great for learning more about the CiviCRM data model).
# The codebase
This chapter provides a general overview of the codebase organisation.
!!! tip
In order to explore the directories inside the CiviCRM repository it is
generally quickest to to make a local clone of the CiviCRM from GitHub,
or better yet install the [buildkit](/requirements/#buildkit).
!!! tip
The CiviCRM codebase is object oriented. If you aren't familiar with object
oriented programming, spend a little time reading some beginner tutorials on
the net.
## Namespaces
Classes in CiviCRM must be placed in one of two folders:
***`CRM`*** (e.g.: `CRM_Core_Invoke`)
classes use PEAR-style class-naming conventions that were common up until
PHP 5.2. Class names include underscores and MUST NOT use the PHP
"[namespace](http://php.net/namespace)"
directive. Use `CRM` style when creating classes that plug into existing `CRM`
subsystems such
as payment processors (CRM_Core_Payment) and reports (CRM_Report).
***`Civi`*** (e.g.: `\Civi\API\Kernel`)
"Civi" classes use PHP 5.3 namespaces. They MUST use the
"[namespace](http://php.net/namespace)" directive.
Namespaces are designated with "\".
!!! note
At time of writing (May 2014, before Civi 4.5), some classes may not load
properly if they are in `Civi` – for example, the payment system will only load
payment-processor classes in `CRM_Core_Payment`. If you encounter problems like
this, please submit an issue or patch.
!!! tip
The `Civi` namespace uses composer's PSR-0 autoloader. This autoloader does not
support custom PHP overrides.
Use `Civi` when creating new object-oriented subsystems (like `\Civi\API`).
## Business logic
Most of the business logic of CiviCRM, is found in the CRM directory (`CIVICRM_ROOT/CRM`).
This logic is the part of CiviCRM that
defines what it does and how it behaves
(e.g. that allows people to register on events).
In this directory, you will find directories for core CiviCRM functions like
contacts, groups, profiles, deduping, importing, and the CiviCRM components
like CiviCampaign, CiviMail, Pledges, etc.
Each of these directories is slightly different depending on their purpose, but
there are some common subdirectories: BAO, DAO, Form and Page.
### DAO
The CiviCRM **data access objects** (DAOs) are PHP classes that
([e.g. `CRM/Pledge/DAO`](https://github.com/civicrm/civicrm-core/tree/master/CRM/Pledge/DAO))
expose the contents
of the database. The release script generates each DAO automatically based
on the matching XML file in the [data schema](#database-structure). DAO objects tend to be instantiated in BAO classes.
The DAO classes all extend the core
[DAO base class](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/DAO.php)
which itself is an extension of the external
[DataObject class](https://github.com/civicrm/civicrm-packages/blob/master/DB/DataObject.php).
These base classes provide standard CRUD (create, retrieve, update and delete)
methods, etc. <!--fixme why the etc? what else?? -->
The generated DAO object has:
* A property for each field using the actual field name, not the unique name
* A `links()` method which retrieves the links to other tables (off the foreign keys)
* An `import()` method and an `export()` method for ?
* A `fields()` method which returns an array of fields for that object keyed by the field's unique name.
* A couple of functions to define the labels for enumerated fields
Looking at the field 'pledge.amount' we see
```php
'pledge_amount' => array(
'name' => 'amount',
'type' => CRM_Utils_Type::T_MONEY,
'title' => ts('Total Pledged') ,
'required' => true,
'import' => true,
'where' => 'civicrm_pledge.amount',
'headerPattern' => '',
'dataPattern' => '',
'export' => true,
'bao' => 'CRM_Pledge_BAO_Pledge',
'table_name' => 'civicrm_pledge',
'entity' => 'Pledge',
),
```
The key is the unique name but the name field is the field's name and the 'where' field shows the MySQL description of it. We also see the data type and whether it is available for search or export.
Generally fields should be exportable unless there is a security reason or they are weird and confusing as the search builder is also driven by this setting.
Fields whose option values can be calculated will also have a `pseudoconstant` section.
### BAO
BAO stands for business access object
([example](https://github.com/civicrm/civicrm-core/blob/master/CRM/Event/BAO/Event.php)).
BAOs map to DAOs and extend them with
the business logic of CiviCRM. The core logic of CiviCRM belongs in the
BAOs, for example they have the code that creates follow up activities when an
activity is created, or create activities and populating custom fields when a
pledge is created.
!!! note
Historically some BAOs had both `add()` and `create()` methods. Current practice
is to favour a single `create()` method.
### Form
In general each form page in CiviCRM maps to a file in one of
the form directories. Form files contain a class that extends CRM_Core_Form.
This class has different methods that the core calls before display to
check permissions, retrieve information (`preProcess`), display
the form (`buildForm`), validate the form (`formRule`) and carry out tasks once the
form is submitted (`postProcess`). Forms can diplay information from the BAO
to users and then call the BAO on submission. Generaly each form has an
associated template (see below) which defines the form's html.
!!! Note
Logic in forms should support friendly user-interfaces but core application logic belongs in the BAO layer.
Moving logic to BAO layer facilitates unit testing and developing modernised front-end applications in the future.
!!! tip
Perhaps the best way to get to grips with the Forms is by experience and
experimentation.
### Page
If a CiviCRM screen is not a Form, it is probably a page. Pages files contain a
class that extend CRM_Core_Page. Similar to the form class, Pages have methods
that are called before the page is displayed to control access, set the title,
etc. (`preProcess`), and when the page is displayed (`run`). Pages tend to
take information from the BAO to be displayed to users. In general, each
page has an associated template (see below) which is used to create the
html of the page.
### xml
This directory contains a menu directory which maps urls to CRM form or page
classes and controls access to these URLs using permissions.
## Templates
The templates directory contains all the HTML for pages and forms. Directly
inside the templates directory is a CRM directory. In general, all templates
map to a form or page class. CiviCRM chooses a template for the form or page
based on the class name.
For example, the class CRM_Member_Form_MembershipRenewal looks for a template
in `templates/CRM/Member/Form/MembershipRenewal.tpl`.
Templates are written in smarty, a common PHP template engine. Variables can
be passed to smarty using the assign() method which is available to all Form
and Page classes.
Customising templates is discussed in more detail in 'Techniques'
## The API
The application programming interface (API) is stored in the `/api`
directory. Best practice for using the API is discussed in more detail in
'Techniques'
## bin scripts
The bin directory contains a variety of scripts that can be run to carry out
specific functions. Some of these scripts are run on a regular basis, for
example the CiviMail 'send' and 'process' scripts. Others are run on a one of
or occasional basis, e.g. update geo-coding.
## SQL
The SQL directory is automatically generated as part of a release. It contains
useful files like the SQL to create the database and insert demo data. Most
developers will not need to edit files in this directory.
## l10n
This directory contains lots of automatically generated localisation files.
You should not need to edit this directory directly. You should instead use
CiviCRM's online translation tool transifex.
## packages
CiviCRM makes use of a lot of 3rd party packages for things like the database,
form, javascript and pdf libraries, wysiwyg editors and so on. You
shouldn't need to edit these packages directory.
This diff is collapsed.
This diff is collapsed.
## When should I edit core CiviCRM?
!!! danger
Most of the time, editing the core codebase directly
is not the recommended way for developers to customise and extend CiviCRM.
CiviCRM has multiple releases per year so direct edits to the core codebase
will create upgrade issues for you.
There are other recommended ways for the majority of scenarios:
extensions, the APIs and hooks.
Most of the time, editing the core codebase directly is not the recommended way for developers to customise and extend CiviCRM. CiviCRM has multiple releases per year so direct edits to the core codebase will create upgrade issues for you. There are other recommended ways for the majority of scenarios: extensions, the APIs, and hooks.
To help you decide, here are a couple of principles:
- Bug fixes should always be applied to core. See [Contributing to Core](/core/contributing.md) for details.
- Some (but not all) enhancements to existing features may be best applied to
core, especially if they would be useful to others.
- New features should generally be packed as Extensions.
- If you aren't familiar with CiviCRM, by far the best way to get a sense if
you should be editing core is by talking with the CiviCRM developer community.
- Bug fixes should always be applied to core. See [Contributing to Core](contributing.md) for details.
- Some (but not all) enhancements to existing features may be best applied to core, especially if they would be useful to others.
- New features should generally be packed as [Extensions](../extensions/index.md).
- If you aren't familiar with CiviCRM, by far the best way to get a sense if you should be editing core is by [talking with the CiviCRM developer community](../basics/community.md#collaboration-tools).
This diff is collapsed.
# Review/Release Process
Releases are developed on a monthly cycle, with all but urgent bug and security fixes appearing in versions that are released the first Wednesday of each month. These iterate the second part of the version number. For example, 5.23.0 was released on Wednesday, March 4, 2020, 5.24.0 was released on Wednesday, April 1, 2020, and 5.25.0 was released on Wednesday, May 6, 2020.
There is no active plan to iterate the first part of the version number. The 4.7.x series generally followed the same monthly pattern as above except that the third part of the version number was iterated each month. CiviCRM 5.0.0 followed 4.7.31 in April 2018, and the change was solely for numbering purposes: there is no substantial difference between 5.0.0 and 4.7.31 except for routine changes similar to those introduced every month. The difference from 4.7.31 to 5.0.0 is comparable to that between 4.7.30 and 4.7.31 or that between 5.0.0 and 5.1.0.
During the month, regressions and security vulnerabilities may be fixed in releases that iterate the third part of the version number. Generally speaking, security changes will only be released on the first or third Wednesday of the month, and they will be announced through [the CiviCRM website and a security announcement email list](https://civicrm.org/security). Other point releases may appear at any time.
## Timing
Monthly versions require the participation of a variety of people around the world. While there is no firm time window, and new versions have been released at all times of day, you can typically expect monthly versions to be releases late in the day in the US/Pacific time zone, which may be early Thursday in many parts of the world.
## Branches
The main branch, termed `master`, is where pull requests are typically merged. These changes will appear in the version released up to 2 months in the future.
Each monthly release is based upon a branch of the codebase that is number for the first two parts of the upcoming release number. This is branched off of `master` soon after the prior monthly release and is often termed the "Release Candidate" even though no static, numbered release candidates are generally produced.
After the monthly release, the numbered branch is retained, and any changes that will appear in point releases for that version will be merged there.
Pull requests should typically be made against `master` unless they clearly should appear in the release candidate or a point release. Situations like this usually involve regressions that newly appear in the release candidate or a very recent version. In these cases, it's not uncommon for reviewers or release managers to request two or three pull requests in tandem: one for `master`, one for the release candidate, and potentially one for the most recent release.
## Example release cycle
This system is best illustrated by the 5.24.x release cycle. The `5.24` branch was cut from `master` soon after the 5.23.0 release on March 4, 2020. A number of late changes, including urgent fixes and intra-release-candidate regressions, were merged during March. On April 1, 2020, the first Wednesday of the month, 5.24.0 was released based upon the branch as it stood that day.
Soon thereafter, the `5.25` branch was cut from `master`.
Over the next couple of weeks, a number of urgent bug fixes were merged to `5.24`, and point releases 5.24.1 and 5.24.2 were released on the 4th and 9th of the month, respectively. Security changes were merged to the branch and released on April 15, 2020, the third Wednesday of the month, as 5.24.3. Further urgent bug fixes were merged to the branch and released as 5.24.4, 5.24.5, and 5.24.6 later in the month.
All of these changes were also merged to `master` and `5.25`. On May 6, 2020, the first Wednesday of the next month, 5.25.0 was released based upon the `5.25` branch. No further changes have been merged to `5.24`, and any bugs appearing in the 5.24.x series are to be addressed in later branches and versions. Meanwhile, the `5.26` branch was cut from `master`, denoting the start of the cycle for the release to appear in June.
## Release impact
Despite having a similar numbering pattern, CiviCRM *does not* use Semantic Versioning. Each monthly version may contain any number of bug fixes, new features, deprecations, database schema changes, and API changes. Each may require site administrators to reconfigure features, manually change things prior to or after upgrading, or address problems in past upgrades.
The release notes for each monthly version contain a synopsis to highlight these considerations. The following items are noted for each version:
- **Fix security vulnerabilities?** A new monthly release will not typically fix security vulnerabilities. This is as a courtesy to site administrators so that they won't have the pressure to upgrade for security reasons while potentially facing other changed due to the upgrade. A security release will more commonly contain only the security-specific changes.
- **Change the database schema?** This highlights releases that add or remove database tables, fields, indexes or triggers. The upgrade process will manage these changes, but site administrators with unusual circumstances or direct-to-database integrations will want to take note of these changes.
- **Alter the API?** The API is intended to be a relatively stable way to integrate other systems or build customizations. A change to the API, whether through adding or deprecating entities or methods, altering the output of a method, or changing permissions for a method, will often need more attention than other changes to the software.
- **Require attention to configuration options?** Changes in the CiviCRM code will often require site administrators to revisit configuration settings that would otherwise be left alone. One common example is system workflow message templates: when the standard template for a receipt is changed, any site administrator who has edited their site's copy of the template will need to merge their site-specific changes with the changes between versions. Another is when new permissions are added: a site administrator may need to check that the appropriate users have permission to do a task that may have previously required a different permission.
- **Fix problems installing or upgrading to a previous version?** Site administrators encountering a problem installing or upgrading may use a variety of workarounds. A version that resolves these problems may require the workarounds to be undone. This is also a useful flag for those who encounter problems and revert the upgrade: they can know that their problems may have been addressed.
- **Introduce features?** Most monthly versions introduce new features, which are defined as improvements that go beyond making an existing feature behave the way it purports to.
- **Fix bugs?** Practically all monthly versions and point releases fix bugs.
## Getting your changes documented accurately
Clear documentation multiplies the impact of any improvement in code. If your change is a new feature someone might want to use, you should explain that it's there and how to use it. If you're fixing a bug, you should make it easy for others experiencing it to know that it is fixed in the new version. If your changes require a site administrator's attention just to continue using CiviCRM as they had before, it's imperative that you catch their attention.
At the very least, be sure that the intent and impact of your changes are clear for the editors of the release notes. If you need the attention of site administrators as they upgrade, you should write an upgrade message and/or system check. Finally, the user, system administrator, and developer documentation allow you to explain in-depth how to make use of your changes.
### Descriptions for release notes
The release notes for each monthly version are built upon a list of all commits since the previous monthly version. The editors of the release notes will never be able to follow all changes as they are developed, deliberated, and merged. Instead, they need to be able to understand the effect of each pull request within a minute or two and convey that in the release notes.
In a long ticket, it's sometimes hard to distinguish the initial proposal from what ultimately happened. For the editors, and for their readers who follow the link to read the details, you can't assume they have followed anything that went on. Conversations sometimes split between GitLab and a pull request (sometimes a pull request that gets closed), and sometimes key things are resolved on email or Mattermost conversations, or in person at a sprint.
An accurate title goes a long way. If a pull request fixes a bug, and the title describes the bug concisely, that is completely sufficient. If the title is "Changes to CiviContribute Component Settings not saved", and in fact the bug is that submitting that form does not save the changes, and the fix makes it so that submitting the form saves the changes, that makes for a perfect description.
If the pull request can't be completely described in the title, it's best to add a modifier indicating what details to look for. "Changes to CiviContribute Component Settings not saved for some users" or "Changes to certain fields in CiviContribute Component Settings not saved" both point to the sort of detail that the editors--and anyone skimming the release notes--would want to read more about.
The second key thing is to link all related issues and pull requests. If there's a GitLab issue that the pull request addresses, link to it. If the pull request follows on an earlier, closed one where there was a lot of discussion, link to it. If there are corresponding pull requests for other branches, link to them. The first two provide context for describing the issue or solution. The latter helps the editors be sure that the change is indeed new in that version or already introduced in an earlier point release.
Finally, especially for larger issues, review the pull request and related issue, if there is one, to make sure it's clear what has actually changed. You can edit the pull request or issue description to add a couple sentences, or if you lack permission to do that, you can add a final comment. If the issue is only partially resolved or just has groundwork laid, make it clear what has been done and what has yet to come. You might also consider whether it's appropriate to have a massive issue that will just be partially complete for the foreseeable future: maybe it's better to split it into several.
### Upgrade messages and system checks
If a site administrator might need to do something following the upgrade, add a post-upgrade message. (These are in the version-specific file in the `CRM/Upgrade/Incremental/php` folder.) You may also add a pre-upgrade message, but know that by the time a site administrator sees it, the codebase has already been swapped out. It's not realistic to expect them to revert at that point, so the effect is similar to the post-upgrade messages.
The person applying upgrades will not necessarily be in a position to make configuration changes. Whether they lack the organizational authority or simply lack the context, the person reading a post-upgrade message may not understand something you're describing or may not feel confident doing it. This is separate from something simply being technically confusing: the IT director might run the upgrade but defer to the fundraising officer for managing edits to the online contribution receipt.
Of course, some other site administrators may simply blow past the pre- and post-upgrade messages, and there's no way to go back and view them later.
There are two strategies for addressing these issues. The first is to make the information easy to find and share. Add documentation in the user or administrator guide, a helper extension like Doctor When, or simply in a clear explanation on the issue or pull request. Make the post-upgrade message something straightforward for the person running the upgrade to share with their colleagues.
The second is to assume that many site administrators will ignore the message. Instead, think through the consequences and make sure that the issue is highlighted later through help text, form validation, or a system check.
System checks are a good way to flag problems on a site. When an administrator logs in for the first time in a day or visits the system status page, problems will be visible. You can write a system check that ensures that things are configured properly, and if they are not, administrator users will be reminded.
### User, administrator, or developer documentation
When you add a new feature or resolve a bug in a way that requires some thought on the part of site administrators, writing documentation in the User Guide, System Administrator Guide, or this Developer Guide will help people understand and use it properly.
It's common to open a pull request in a documentation repository and point out that it is dependent upon unmerged changes in the code repository. Add links in both directions, and reviewers can know to merge them together.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.