Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • documentation/docs/dev
  • totten/dev
  • bgm/dev
  • ivan_compucorp/dev
  • seamuslee/dev
  • artfulrobot/dev
  • ufundo/dev
  • wmortada/dev
  • lucky091588/dev
  • DaveD/dev
  • jtwyman/dev
  • rukkykofi/dev
  • JonGold/dev
  • jaapjansma/developer-docs
  • alainb/dev
  • noah/dev
  • justinfreeman/dev
  • pradeep/dev
  • larssg/dev
  • eileen/dev
  • darrick/dev
  • mattwire/dev
  • colemanw/dev
  • homotechsual/dev
  • JoeMurray/dev
  • maynardsmith/dev
  • kurund/dev
  • rocxa/dev
  • AllenShaw/dev
  • bradleyt/dev
  • chrisgaraffa/dev
  • martin.w/dev
  • herbdool/dev
  • MattTrim1/dev
  • Detlev/dev
  • ErikHommel/dev
  • brienne/devdocs
  • pminf/dev
  • SarahFG/dev
  • ayduns/dev
  • JKingsnorth/dev
  • ginkgomzd/dev
  • nicol/dev
  • almeidam/dev
  • arthurm/dev
  • damilare/dev
  • semseysandor/dev
  • major/devdocs
  • usha.makoa/dev
  • yurg/dev
  • shaneonabike/dev
  • andie/dev
  • mmyriam/dev
  • gngn/dev
  • florian-dieckmann/dev
  • jade/dev
  • luke.stewart/dev
  • vinaygawade/dev
58 results
Show changes
Hide whitespace changes
Inline Side-by-side
Showing
with 768 additions and 415 deletions
docs/testing/upgrades.md
View file @ 11817d0b
......@@ -6,6 +6,6 @@ They are suitable for checking issues in the DB upgrade logic –
but do not check for issues in the administrative experience or in the
CMS-integration.
Upgrade tests are run daily on the Jenkins [continuous integration](/testing/continuous-integration.md) server.
Upgrade tests are run daily on the Jenkins [continuous integration](continuous-integration.md) server.
Locally you can use [civi-test-run](/tools/civi-test-run.md) to run the same upgrade tests as Jenkins would.
Locally you can use [civi-test-run](../tools/civi-test-run.md) to run the same upgrade tests as Jenkins would.
docs/tools/buildkit.md
View file @ 11817d0b
# Buildkit
[Buildkit](https://github.com/civicrm/civicrm-buildkit) is a collection of ~20 tools for developing and testing CiviCRM, the most important of which is [civibuild](/tools/civibuild.md).
To develop for CiviCRM, you will need the [basic requirements (PHP, MySQL, etc)](../basics/requirements.md) and several specific development tools.
Many of these tools are commonly used by web developers, so you may have already installed a few. Even so, it's generally easier to install the full collection — installing each individually takes a lot of work.
The collection of typical development tools is called [civicrm-buildkit](https://github.com/civicrm/civicrm-buildkit). But how do you install it?
This is the same collection of tools which manages the test/demo/release infrastructure for civicrm.org.
| Installation Approach | Pros / Cons |
| -- | -- |
| __[Full Stack](#full-stack)__: Install one suite with everything (PHP, MySQL, `cv`, `civix`, etc). | __Pro__: You get a comprehensive suite. Everything is configured.<br/> __Con__: Highly opinionated configurations. You need to find a full stack that comes close to your needs. |
| __[Half Stack](#half-stack)__: Install the basic requirements (PHP, MySQL) on your own. Add `buildkit` for additional tools (`cv`, `civix`, etc). | __Pro__: You can choose how to address basic requirements, but you don't need to manage every tool.<br/> __Con__: Initial configuration may be tricky. Requires Unix-style environment. Somewhat opinionated. |
| __[Custom Stack](#custom-stack)__: Choose and configure every component yourself. | __Pro__: Maximum flexibility. Works on all platforms.<br/> __Con__: You will need to understand, configure, and maintain all tools. Your toolset may get out-of-sync with other developers' toolset. |
## Installation
## Full Stack
Buildkit supports several Unix-style environments. It may be installed on [a fresh workstation or virtual machine running Ubuntu/Debian](#ubuntu-debian). It can also be used in a [Vagrant VM](#vagrant) or [Docker container](#docker). If you wish to install in any other Unix-style system (such as macOS or RedHat), then follow the [generic instructions](#other-platforms).
For a new or inexperienced contributor, the safest bet is a [Full Stack: Virtual Desktop](buildkit/virtual.md). It provides broad functionality and broad compatibility
with very little configuration.
### Ubuntu / Debian
Of course, there are limitations with __Virtual Desktop__ - it's not as fast; it may not use your favorite IDE; and so on. Fortunately, there are other full-stack options
(*with different trade-offs*). Here are a few ways to assemble a full stack:
If you have a fresh system install of Ubuntu or Debian (with a [recent version](#operating-systems) like Ubuntu 18.04 LTS), then you can download everything using the `get-buildkit.sh` script.
| Full Stack Approach | Host Compatibility |
| -- | -- |
| __[Apt-Get](buildkit/apt-get.md)__: Install PHP, MySQL, etc as _local system-packages_ (`/usr`). | __Linux__: Native ([details](buildkit/apt-get.md#compatibility)) </br>__MacOS__: No<br/>__Windows__: No |
| __[Docker](buildkit/docker.md)__: Install PHP, MySQL, etc as _headless containers_ (`/var/lib`). | __Linux__: Native (x86<sup>&dagger;</sup>)</br>__MacOS__: Virtual (x86<sup>&dagger;</sup>)<br/>__Windows__: Virtual (x86<sup>&dagger;</sup>) |
| __[Nix](buildkit/nix.md)__: Install PHP, MySQL, etc as _isolated add-on packages_ (`/nix`). | __Linux__: Native (x86, arm64)</br>__MacOS__: Native (x86, arm64)<br/>__Windows__: No<sup>&dagger;&dagger;</sup> |
| ~~__[Vagrant](buildkit/vagrant.md)__~~: ~~Run a _headless virtual machine_ with PHP, MySQL, etc.~~ | __Linux__: Virtual (x86<sup>&dagger;</sup>)</br>__MacOS__: Virtual (x86<sup>&dagger;</sup>)<br/>__Windows__: Virtual (x86<sup>&dagger;</sup>) |
| __[Virtual Desktop](buildkit/virtual.md)__: Run a virtual machine with an _entire desktop OS_ and all other requirements. | __Linux__: Virtual (x86)</br>__MacOS__: Virtual (x86, arm64)<br/>__Windows__: Virtual (x86) |
```bash
sudo apt-get install curl
curl -Ls https://civicrm.org/get-buildkit.sh | bash -s -- --full --dir ~/buildkit
```
__<sup>&dagger;</sup>__: <small>Docker and Vagrant officially support x86 and arm64, but the buildkit configuration is only developed on x86.</small><br/>
__<sup>&dagger;&dagger;</sup>__: <small>Nix officially supports WSL2 virtualization, but the buildkit configuration is not tested on WSL2.</small>
This creates a personal workspace folder (`~/buildkit`) for helper scripts, caches, and builds. It also uses `--full` mode to download a complete set of system packages (PHP, MySQL, Apache, etc).
## Half Stack
!!! tip "Usage tips"
If the basic requirements are already provided by your operating system (Debian, Ubuntu, Fedora, RHEL, etc) or add-on stack (MAMP, XAMPP, brew, etc),
then consult the [generic installation guide for buildkit](buildkit/generic.md).
* You should *not* use `root`, `su`, or `sudo` except where specifically noted. Buildkit is generally designed to run as your regular user, and unnecessary `root` privileges *will* cause problems. If the installer needs elevated privileges, it will call `sudo` on a case-by-case basis.
* The install script will only execute `--full` mode on a supported release of Ubuntu / Debian. See also: [Appendix: Operating Systems](#operating-systems).
* The `--full` option is *very opinionated*; it specifically installs `php`, `apache`, and `mysql` (rather than `hhvm`, `nginx`, `lighttpd`, or `percona`). If you try to mix `--full` with alternative systems, then expect conflicts.
* If you use the Ubuntu feature for "encrypted home directories", then don't put buildkit in `~/buildkit`. Consider `/opt/buildkit`, `/srv/buildkit`, or some other location that remains available during reboot.
## Custom Stack
After running the above command, then proceed to the [post-installation configuration](#config).
### Vagrant
[Full Download: Vagrantbox](https://github.com/civicrm/civicrm-buildkit-vagrant) - Download a prepared virtual-machine with all system dependencies (mysql, etc). This is ideal if you work on Windows or OS X.
### Docker
If you have [Docker](https://www.docker.com/) running, you can use one of the following projects to run buildkit within a Docker container:
* <https://github.com/michaelmcandrew/civicrm-buildkit-docker>
* <https://github.com/progressivetech/docker-civicrm-buildkit>
* <https://github.com/ErichBSchulz/dcbk>
!!! Note
There are different versions of Buildkit on Docker. Michael McAndrew's seems to be the easiest to get started with on Linux.
#### Install buildkit on docker on ubuntu
Follow the official installation instructions from https://docs.docker.com/compose/install/ to install docker compose on your linux machine.
```bash
git clone https://github.com/michaelmcandrew/civicrm-buildkit-docker.git
cd civicrm-buildkit-docker
sudo docker-compose up -d
```
Now you are ready to go.
To create a new site with buildkit run the following command:
```bash
docker-compose exec -u buildkit civicrm civibuild create dmaster --url http://localhost:8080
```
Alternative you can login into the conatiner and run the commands from there:
```bash
docker-compose exec -u buildkit civicrm bash
```
More information is in the Readme: https://github.com/michaelmcandrew/civicrm-buildkit-docker/blob/master/README.md
### Generic {:#other-platforms}
You may install buildkit in other environments. The main pre-requisites are:
* Linux or OS X
* Git
* PHP 5.6+ (7.2+ recommended), including the following extensions: `bcmath curl gd gettext imap intl imagick json mbstring openssl pdo_mysql phar posix soap zip`
* NodeJS 8+
* NPM
[amp](https://github.com/totten/amp) and [civibuild](/tools/civibuild.md) also require:
* Apache 2.2 or 2.4, including the `mod_rewrite` module and, on SUSE, possibly `mod_access_compat` (This list may not be exhaustive.)
* MySQL 5.5+ (5.7+ recommended), including both client and server
All pre-requisites must support command-line access using the standard command names (`git`, `php`, `node`, `mysql`, `mysqldump`, etc). In some environments, you may need to enable these commands by configuring `PATH` -- this is especially true for MAMP, XAMPP, and other downloaded packages. (See, e.g., [Setup Command-Line PHP](/standards/php.md).)
Once the pre-requisites are met, download buildkit to `~/buildkit`:
```bash
$ git clone https://github.com/civicrm/civicrm-buildkit.git ~/buildkit
$ cd ~/buildkit
$ ./bin/civi-download-tools
```
## Post-install configuration {:#config}
### Configuring your path {:#path}
!!! note "Not needed for Vagrant/Docker installations"
If you set up buildkit using Vagrant or Docker, then you don't need to perform the configuration steps listed here.
Buildkit includes many CLI commands in the `bin/` folder.
You may execute the commands directly (e.g. `./bin/civix` or `/path/to/buildkit/bin/civix`). However, this would become very cumbersome. Instead, you should configure the shell's `PATH` to recognize these commands automatically.
!!! tip
Throughout this document, we will provide examples which assume that buildkit was downloaded to `/path/to/buildkit`. Be sure to adjust the examples to match your system.
If you want to ensure that the buildkit CLI tools are always available, then:
1. Determine the location of your shell configuration file. This is usually `~/.bash_profile`, or `~/.profile`. You may have to create one.
1. At the end of the file, add `PATH="/path/to/buildkit/bin:$PATH"`.
1. If you are on a mac, you can close and re-open your terminal. On other systems, you will need to log-out or source your `~/.profile` with `source ~/.profile`.
1. Enter the command `civibuild -h`. This should display a help screen for civibuild. If you get 'command not found', then check your path and retry the steps above.
!!! tip
For most installations with the standard buildkit install script the following lines in your shell configuration file will work.
``` bash
# Add ~/buildkit/bin to path if it exists.
if [ -d "$HOME/buildkit/bin" ] ; then
PATH="$HOME/buildkit/bin:$PATH"
fi
```
!!! note "More on bash `$PATH`"
On most OS's `~/.profile` is run only once when you login to your desktop. There is a distinction between "login shells" and "non-login shells" which you don't really need to worry about, except that the distinction is the reason that you should set your `$PATH` in your `~/.profile` and not your `~/.bashrc`.
When you open a terminal (non-login), `~/.bashrc` will be executed. The common idiom for changing the path is to add to the `$PATH`, not rebuild it, so if you update your `$PATH` every time a shell is invoked, your `$PATH` will continually grow. This is not really a problem, but you might want to be aware of this.
If you are on a mac, the situation is reversed. That is, your `$PATH` is not set when you login into your desktop and every terminal you open is a "login shell" and `~/.profile` will be executed every time.
You do not need to run `export PATH=...` because your system certainly has already exported the `$PATH` variable and you only need to update it.
References:
* <https://unix.stackexchange.com/a/26059>
* <https://superuser.com/questions/244964/mac-os-x-bashrc-not-working#244990>
* <https://askubuntu.com/questions/155865/what-are-login-and-non-login-shells#156038>
!!! note
Buildkit includes specific versions of some fairly popular tools (such as `drush`, `phpunit`, and `wp-cli`), and it's possible that you have already installed other versions of these tools.
By design, buildkit can coexist with other tools, but you must manually manage the `PATH`.
Whenever you wish to use buildkit, manually run a command like, e.g.:
```bash
export PATH=/path/to/buildkit/bin:$PATH
```
To restore your normal `PATH`, simply close the terminal and open a new one.
Each time you open a new terminal while working on Civi development, you would need to re-run the `export` command.
### Configuring `amp` {:#amp-config}
Buildkit provides a tool called `amp` which [civibuild](/tools/civibuild.md) uses when it needs to set up a new site. Before you can use `civibuild`, need to configure `amp` by telling it a bit about your system (e.g. what webserver you're using).
1. Run the interactive configuration tool.
```
$ amp config
```
!!! tip "tips"
* Run this as a non-`root` user who has `sudo` permission. This will ensure that new files are owned by a regular user, and (if necessary) it enables `civibuild` to restart your webserver and edit `/etc/hosts`.
* Pay close attention to any instructions given in the output of this command.
* To check which version of apache you have, run `apachectl -v`.
!!! caution
We strongly recommend using Apache as your webserver because support for nginx is limited.
1. Test amp's configuration
```
$ amp test
```
The test is successful if you see `Received expected response` at the end.
If the test produces any errors, you might try re-running the above config steps and/or asking for help in the [developer chat room](https://chat.civicrm.org/civicrm/channels/dev).
1. After `amp` is configured, you can move on to running [civibuild](/tools/civibuild.md) to build a local development installation of CiviCRM.
## Troubleshooting {:#troubleshooting}
### Node JS issues
Nodejs version too old or npm update does not work:
: Download the latest version from nodejs.org and follow their instructions
Nodejs problems
: It might be handy to run
```bash
npm update
npm install fs-extra
```
### Website login issues
If you find that when you try and login to a new buildkit build or similar and it doesn't seem to login just redirects to the same page. This may mean that the rewrite module for apache is not enabled. To enable it do the following
```bash
sudo a2enmod rewrite
```
After enabling the rewite module you will need to restart apache.
## Upgrading buildkit {:#upgrading}
New versions of buildkit are likely to include new versions of tools. The new tools will download automatically when you first run `civibuild`. If you prefer to download explicitly, then re-run `civi-download-tools`.
The configurations and tools in buildkit are periodically updated. To get the latest, simply run:
```bash
cd ~/buildkit
git pull
./bin/civi-download-tools
```
See the [buildkit changelog](https://github.com/civicrm/civicrm-buildkit/blob/master/CHANGELOG.md) for info about specific changes to buildkit.
!!! tip "When upgrading `civix`, check upgrade instructions."
If you see an upgrade to `civix` in the changelog, and if you maintain extensions with `civix`,
then check the general [civix upgrade documentation](/extensions/civix.md#upgrade-templates) and [UPGRADE.md](https://github.com/totten/civix/blob/master/UPGRADE.md).
## Appendix: Operating Systems {:#operating-systems}
Currently Buildkit includes specific, tested install steps for the following Ubuntu and Debian operating system releases. Note that recently removed versions are shown in this list for information and are marked in the final column.
There are no specific installer steps for macOS but Buildkit itself is fully usable on a Mac. Buildkit does not natively support running on Windows at this time but other options are available (e.g: Vagrant/Docker).
!!! note
Versions of Ubuntu and Debian running on Windows Subsystem for Linux (WSL) and WSL2 are not currently compatible with Buildkit.
### Ubuntu
Version | Codename | Release Date | EoL Date | Buildkit Removal |
--------- | ------------ | -------------- | ---------- | ------------------------- |
19.04 | Disco Dingo | April 2019 | January 2020 | June 2020 |
18.10 | Cosmic Cuttlefish | October 2018 | July 2019 <sup>&#x1F534;</sup> | January 2020 |
18.04 | Bionic Beaver | April 2018 | April 2023 | October 2023 |
17.10 | Artful Aardvark | October 2017 | July 2018 <sup>&#x1F534;</sup> | January 2019 <sup>&#x2705;</sup> |
17.04 | Zesty Zapus<sup>*</sup> | April 2017 | January 2018 <sup>&#x1F534;</sup> | July 2018 <sup>&#x2705;</sup> |
16.10 | Yakkety Yak<sup>*</sup> | October 2016 | July 2017 <sup>&#x1F534;</sup> | January 2018 <sup>&#x2705;</sup> |
16.04 | Xenial Xerus | April 2016 | April 2021 | October 2021 |
14.04 | Trusty Tahr | April 2014 | April 2019 <sup>&#x1F534;</sup> | October 2019 <sup>&#x2705;</sup> |
12.04 | Precise Pangolin | April 2012 | April 2017 <sup>&#x1F534;</sup> | October 2017 <sup> &#x2705;</sup> |
<sup>*</sup> = Reuses installation steps for Xenial Xerus.
<sup>&#x1F534;</sup> = Is currently EoL.
<sup>&#x2705;</sup> = Has been removed from Buildkit
### Debian
Version | Codename | Release Date | EoL Date | Buildkit Removal |
--------- | ------------ | -------------- | ---------- | ------------------ |
10 | Buster | July 2019 | 202x | Unknown |
9 | Stretch | June 2017 | 2022 | Unknown |
8 | Jessie | April 2015 | June 2020 | September 2020 |
!!! warning
Our current policy is that these specific install steps will be removed from Buildkit when they reach their End Of Life (EoL) date plus 6 months. See [this issue](https://github.com/civicrm/civicrm-buildkit/issues/432) for discussion/information.
If you are going to do a __custom stack__, then there is no specific guide. However, you should review [civicrm-buildkit](https://github.com/civicrm/civicrm-buildkit) to
understand the kinds of tools and configurations that are typically required.
docs/tools/buildkit/apt-get.md 0 → 100644
View file @ 11817d0b
In Debian and Ubuntu, the `apt-get` package-manager provides access to an extensive library of software. The script `get-buildkit.sh` will
use `apt-get` to install all major requirements.
??? info "Major features"
This process will install the following packages in `/usr`:
* __Interpreters__: PHP-CLI, NodeJS
* __Servers__: Apache HTTPD, MySQL, PHP-FPM
Additionally, this process will also install buildkit in `$HOME/buildkit`.
??? info "Trade-offs"
* __Pro__: Large, well-known repository of packages. Packages are shared and deeply integrated with the host/desktop.
* __Con__: Package versions are dictated by the distro. The desktop generally only supports one version of each package. When developing
for multiple projects, you may get boxed-in by conflicting dependencies.
## Requirements
* Debian or Ubuntu (*see also: [Appendix: Compatibility](#compatibility)*)
* Sudo privileges
* (*Recommended*) Install on a new/clean system.
??? warning "Warning: This process is only tested on new/clean systems."
If you previously used `apt-get` to install packages for web-hosting/PHP/JS on this system, then this creates some uncertainty. Stay alert for these kinds of complications:
* _Third-Party Sources_: Third-party APT repositories could override packages -- changing the versions or file-layouts.
* _Conflicting Services_: Services like `mariadb-server` and `mysql-server` may substantively conflict (*because they have similar port-numbers and similar command-names*).
In the same way, `nginx` or `caddy` may conflict with `apache2`.
* _Customized Configuration_: If you previously installed and modified Apache/PHP/MySQL, the custom configuration could unexpectedly interact with the new configuration.
## Download
If you are installing a new Debian/Ubuntu desktop, then you can download everything via `get-buildkit.sh`. The script will:
1. Download system requirements (PHP, MySQL, Apache, etc) via `apt-get`.
2. Create a working folder `~/buildkit` (for helper scripts, caches, site-builds).
You may run `get-buildkit.sh` as follows:
```bash
sudo apt-get install curl
curl -Ls https://civicrm.org/get-buildkit.sh | bash -s -- --full --dir ~/buildkit
```
??? warning "Warning: Do not use `root`, `su`, or `sudo` -- except where specifically noted."
Buildkit is generally designed to run as your regular user. If it needs elevated privileges, it will call `sudo` and ask for privileges.
If one mistakenly runs commands as `root`, then data-files and cache-files may be created with the wrong permissions and/or wrong locations.
This *will* cause problems later -- and will require finding and fixing the erroneous data.
??? warning "Warning: If you have "encrypted home directories", then change the `--dir`."
Ubuntu has optional support for "encrypted home directories". If your system uses this option, then system-services like Apache will have difficulty
reading from `~/buildkit`. The difficulty may not appear immediately - it would appear after the next reboot.
Instead, put buildkit in a reliable folder like `/srv/buildkit`. For example:
```bash
sudo apt-get install curl
sudo mkdir /srv/buildkit
sudo chown $USER /srv/buildkit
curl -Ls https://civicrm.org/get-buildkit.sh | bash -s -- --full --dir /srv/buildkit
```
Once the download is complete, you should be able to see the new folder `~/buildkit` (or `/srv/buildkit`).
## Next steps
* After you have downloaded all the tools, you must [Configure PATH](generic.md#path) and [Configure amp](generic.md#amp-config).
* Finally, you can move on to [using `civibuild`](../civibuild.md).
## Appendix: Compatibility {:#compatibility}
The script `get-buildkit.sh` includes installation steps that have been tested on specific releases of Debian and Ubuntu.
!!! note "Recently removed versions are shown in this list for information and are marked in the final column."
!!! note "Versions of Ubuntu and Debian running on Windows Subsystem for Linux (WSL1 and WSL2) are not currently compatible with `get-buildkit.sh`."
!!! warning "Our current policy is that these specific install steps will be removed from Buildkit when they reach their End Of Life (EoL) date plus 6 months. See [this issue](https://github.com/civicrm/civicrm-buildkit/issues/432) for discussion/information."
### Debian
Version | Codename | Release Date | EoL Date | Buildkit Removal |
--------- | ------------ | -------------- | ---------- | ------------------ |
11 | Bullseye | September 2021 | 202x | Unknown |
10 | Buster | July 2019 | 202x | Unknown |
9 | Stretch | June 2017 | 2022 | Unknown |
8 | Jessie | April 2015 | June 2020 | September 2020 <sup>&#x2705;</sup> |
### Ubuntu
Version | Codename | Release Date | EoL Date | Buildkit Removal |
--------- | ------------ | -------------- | ---------- | ------------------------- |
20.04 | Focal Fossa | April 2020 | April 2025 | October 2025 |
19.04 | Disco Dingo | April 2019 | January 2020 <sup>&#x1F534;</sup> | June 2020 <sup>&#x2705;</sup> |
18.10 | Cosmic Cuttlefish | October 2018 | July 2019 <sup>&#x1F534;</sup> | January 2020 <sup>&#x2705;</sup> |
18.04 | Bionic Beaver | April 2018 | April 2023 | October 2023 |
17.10 | Artful Aardvark | October 2017 | July 2018 <sup>&#x1F534;</sup> | January 2019 <sup>&#x2705;</sup> |
17.04 | Zesty Zapus<sup>*</sup> | April 2017 | January 2018 <sup>&#x1F534;</sup> | July 2018 <sup>&#x2705;</sup> |
16.10 | Yakkety Yak<sup>*</sup> | October 2016 | July 2017 <sup>&#x1F534;</sup> | January 2018 <sup>&#x2705;</sup> |
16.04 | Xenial Xerus | April 2016 | April 2021 | October 2021 <sup>&#x2705;</sup> |
14.04 | Trusty Tahr | April 2014 | April 2019 <sup>&#x1F534;</sup> | October 2019 <sup>&#x2705;</sup> |
12.04 | Precise Pangolin | April 2012 | April 2017 <sup>&#x1F534;</sup> | October 2017 <sup> &#x2705;</sup> |
<sup>*</sup> = Reuses installation steps for Xenial Xerus.<br/>
<sup>&#x1F534;</sup> = Is currently EoL.<br/>
<sup>&#x2705;</sup> = Has been removed from Buildkit<br/>
docs/tools/buildkit/docker.md 0 → 100644
View file @ 11817d0b
In the _Docker_ approach, you run all components (PHP, MySQL, `buildkit`) in Docker containers.
??? info "Trade-offs"
* __Pro__: The _Docker_ approach works well on Linux-x86. Configuration is packaged, editable, and disposable. It is somewhat compatible with Windows and MacOS (with virtualization).
* __Con__: Feels like using a remote server, so it may be difficult to integrate debuggers and other desktop tools. Only maintained on x86. For Windows/MacOS hosts, virtualization hinders performance (esp file I/O).
## Requirements
* [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/install/)
* Intel/AMD CPU (*The Docker core is compatible with other CPUs, but there are currently compatiblity issues with specific containers.*)
## Download
You can download one of the following projects to run buildkit within a Docker container:
* <https://lab.civicrm.org/michaelmcandrew/civicrm-buildkit-docker>
* <https://github.com/progressivetech/docker-civicrm-buildkit>
* <https://github.com/ErichBSchulz/dcbk>
!!! Note
There are different versions of Buildkit on Docker. Michael McAndrew's seems to be the easiest to get started with on Linux.
#### Download buildkit on docker on ubuntu
After install docker and docker-compose, download the container descriptions:
```bash
git clone https://lab.civicrm.org/michaelmcandrew/civicrm-buildkit-docker.git
cd civicrm-buildkit-docker
sudo docker-compose up -d
```
Now you are ready to go.
To create a new site with buildkit run the following command:
```bash
docker-compose exec -u buildkit civicrm civibuild create dmaster --url http://localhost:8080
```
Alternative you can login into the conatiner and run the commands from there:
```bash
docker-compose exec -u buildkit civicrm bash
```
More information is in the Readme: https://lab.civicrm.org/michaelmcandrew/civicrm-buildkit-docker/-/blob/master/README.md
## Next steps
Once you are able to run commands on the `civicrm` container, you can move on to using [using `civibuild`](../civibuild.md).
docs/tools/buildkit/generic.md 0 → 100644
View file @ 11817d0b
In the _Generic_ approach, you install `buildkit` on top of an existing Unix-style environment.
??? info "Trade-offs"
* __Pro__: The _Generic_ approach is compatible with many different environments, styles, and versions. You may freely tune your environment and process without concern for how it affects other developers.
* __Con__: The _Generic_ approach requires configuring several subsystems (`bash`, `amp`, `mysql`, etc). You need to understand each subsystem, track requirements, manage updates. Your environment may not match other developers' environments.
## Requirements
* The system must already meet the [basic requirements](../../basics/requirements.md) for PHP, MySQL, etc.
??? warning "Apache HTTPD is _very strongly_ recommended. Why?"
Apache HTTPD is not _strictly_ required. Many tools in buildkit work with any web-server (*or no web server!*).
However, one major tool ([civibuild](../civibuild.md)) works better with Apache. `civibuild` facilitates testing across different PHP CMS's (*D7, D8, D9, WordPress,
Backdrop, etc*). Every CMS includes built-in Apache support (via `.htaccess`).
It is _theoretically_ possible to run `civibuild` and then serve sites with a different HTTPD. Or you could skip `civibuild` completely. However, in either case, you
would have to maintain the HTTPD configuration yourself.
Apache is the only option with automatic, canonical support in all CMS's.
* Additionally, the configuration process requires *sysadmin skills*. You will be asked to configure several components and adapt them to match the local environment.
!!! tip "If you are not comfortable with sysadmin tasks, then consider using a [Virtual Desktop](virtual.md) instead."
## Download
Clone the `civicrm-buildkit.git` repo and run `civi-download-tools`:
```bash
$ git clone https://github.com/civicrm/civicrm-buildkit.git ~/buildkit
$ cd ~/buildkit
$ ./bin/civi-download-tools
```
In the above example, all tools are downloaded under `~/buildkit`.
!!! note "Requirements checking"
`civi-download-tools` will attempt to identify and report common issues (such as missing/unknown commands).
If it reports a problem, you should resolve that problem and retry.
Depending on the specific problem and its resolution, you may find it necessary to close and reopen the terminal.
## Configure PATH {:#path}
Buildkit includes many CLI commands in the `bin/` folder.
You may execute the commands directly (e.g. `./bin/civix` or `/path/to/buildkit/bin/civix`). However, this would become very cumbersome. Instead, you should configure the shell's `PATH` to recognize these commands automatically.
!!! tip
Throughout this document, we will provide examples which assume that buildkit was downloaded to `/path/to/buildkit`. Be sure to adjust the examples to match your system.
If you want to ensure that the buildkit CLI tools are always available, then:
1. Determine the location of your shell configuration file. This is usually `~/.bash_profile`, or `~/.profile`. You may have to create one.
1. At the end of the file, add `PATH="/path/to/buildkit/bin:$PATH"`.
1. If you are on a mac, you can close and re-open your terminal. On other systems, you will need to log-out or source your `~/.profile` with `source ~/.profile`.
1. Enter the command `civibuild -h`. This should display a help screen for civibuild. If you get 'command not found', then check your path and retry the steps above.
!!! tip
For most installations with the standard buildkit install script the following lines in your shell configuration file will work.
``` bash
# Add ~/buildkit/bin to path if it exists.
if [ -d "$HOME/buildkit/bin" ] ; then
PATH="$HOME/buildkit/bin:$PATH"
fi
```
!!! note "More on bash `$PATH`"
On most OS's `~/.profile` is run only once when you login to your desktop. There is a distinction between "login shells" and "non-login shells" which you don't really need to worry about, except that the distinction is the reason that you should set your `$PATH` in your `~/.profile` and not your `~/.bashrc`.
When you open a terminal (non-login), `~/.bashrc` will be executed. The common idiom for changing the path is to add to the `$PATH`, not rebuild it, so if you update your `$PATH` every time a shell is invoked, your `$PATH` will continually grow. This is not really a problem, but you might want to be aware of this.
If you are on a mac, the situation is reversed. That is, your `$PATH` is not set when you login into your desktop and every terminal you open is a "login shell" and `~/.profile` will be executed every time.
You do not need to run `export PATH=...` because your system certainly has already exported the `$PATH` variable and you only need to update it.
References:
* <https://unix.stackexchange.com/a/26059>
* <https://superuser.com/questions/244964/mac-os-x-bashrc-not-working#244990>
* <https://askubuntu.com/questions/155865/what-are-login-and-non-login-shells#156038>
!!! note
Buildkit includes specific versions of some fairly popular tools (such as `drush`, `phpunit`, and `wp-cli`), and it's possible that you have already installed other versions of these tools.
By design, buildkit can coexist with other tools, but you must manually manage the `PATH`.
Whenever you wish to use buildkit, manually run a command like, e.g.:
```bash
export PATH=/path/to/buildkit/bin:$PATH
```
To restore your normal `PATH`, simply close the terminal and open a new one.
Each time you open a new terminal while working on Civi development, you would need to re-run the `export` command.
## Configure `amp` {:#amp-config}
Buildkit provides a tool called `amp` which [civibuild](../civibuild.md) uses when it needs to set up a new site. Before you can use `civibuild`, need to configure `amp` by telling it a bit about your system (e.g. what webserver you're using).
1. Run the interactive configuration tool.
```
$ amp config
```
!!! tip "tips"
* Run this as a non-`root` user who has `sudo` permission. This will ensure that new files are owned by a regular user, and (if necessary) it enables `civibuild` to restart your webserver and edit `/etc/hosts`.
* Pay close attention to any instructions given in the output of this command.
* To check which version of apache you have, run `apachectl -v`.
!!! caution
We strongly recommend using Apache as your webserver because support for nginx is limited.
1. Test amp's configuration
```
$ amp test
```
The test is successful if you see `Received expected response` at the end.
If the test produces any errors, you might try re-running the above config steps and/or asking for help in the [developer chat room](https://chat.civicrm.org/civicrm/channels/dev).
## Next steps
After configuring the `PATH` and `amp`, you can move on to [using `civibuild`](../civibuild.md).
## Appendix: Troubleshoot {:#troubleshooting}
### Node JS issues
Nodejs version too old or npm update does not work:
: Download the latest version from nodejs.org and follow their instructions
Nodejs problems
: It might be handy to run
```bash
npm update
npm install fs-extra
```
### Website login issues
If you find that when you try and login to a new buildkit build or similar and it doesn't seem to login just redirects to the same page. This may mean that the rewrite module for apache is not enabled. To enable it do the following
```bash
sudo a2enmod rewrite
```
After enabling the rewite module you will need to restart apache.
## Appendix: Upgrade buildkit {:#upgrading}
New versions of buildkit are likely to include new versions of tools. The new tools will download automatically when you first run `civibuild`. If you prefer to download explicitly, then re-run `civi-download-tools`.
The configurations and tools in buildkit are periodically updated. To get the latest, simply run:
```bash
cd ~/buildkit
git pull
./bin/civi-download-tools
```
See the [buildkit changelog](https://github.com/civicrm/civicrm-buildkit/blob/master/CHANGELOG.md) for info about specific changes to buildkit.
!!! tip "When upgrading `civix`, check upgrade instructions."
If you see an upgrade to `civix` in the changelog, and if you maintain extensions with `civix`,
then check the general [civix upgrade documentation](https://docs.civicrm.org/dev/en/latest/extensions/civix/#upgrade) and [UPGRADE.md](https://github.com/totten/civix/blob/master/UPGRADE.md).
docs/tools/buildkit/nix.md 0 → 100644
View file @ 11817d0b
`nix` is a cross-platform package-manager. `nix` runs on top of most Linux distributions (Debian, Ubuntu, Redhat, etc) as well as MacOS.
Nix packages are stored in a separate folder (`/nix`), and you can safely run multiple versions of any package.
`nix` is used by the CI test-infrastructure for `civicrm-core`.
??? info "Major features"
These instructions will install the following packages in `/nix`:
* __Interpreters__: PHP-CLI, NodeJS
* __Servers__: Apache HTTPD, MySQL, PHP-FPM, MailHog, Redis <!-- Memcached is installed but disabled by default -->
Additionally, they will install buildkit in `$HOME/buildkit`.
??? info "Trade-offs"
* __Pro__: `nix` runs locally at native speed on Linux+MacOS with x86+arm64. Configuration is packaged, editable, and disposable. Can be combined with desktop tools (IDEs/debuggers). Packages don't interfere with host. Closely matches official test environment.
* __Con__: No Windows support (except through WSL/virutalization). Customization requires learning new language. Major host OS updates sometimes have compatibility issues (but usually fixable).
## Requirements
* Linux or MacOS (*Note: Generally, you'll get the best results with an OS release that is 3-24 months old -- something recent but not brand new.*)
* x86 or arm64
* `git`
* Sudo privileges
## Download
```bash
## Download the configuration
git clone https://github.com/civicrm/civicrm-buildkit ~/buildkit
## Download and install the binaries
cd ~/buildkit
./nix/bin/install-developer.sh
```
!!! info "The "`install-developer.sh`" script will download a few gigabytes worth of data.<br/>It may compile some packages from source."
For further discussion about these commands and their variations, see [nix/doc/install-developer.md](https://github.com/civicrm/civicrm-buildkit/blob/master/nix/doc/install-developer.md).
## Startup
```bash
## Open a subshell
use-bknix dfl -s
## Start the services - HTTPD, MySQL, etc
cd ~/buildkit
loco run
```
!!! info "The "`loco run`" command will start HTTPD, MySQL, etc on alternative ports.<br/>This allows them to coexist with any other services you have."
For further discussion about these commands and their variations, see [nix/doc/usage-loco.md](https://github.com/civicrm/civicrm-buildkit/blob/master/nix/doc/usage-loco.md).
## Next steps
Finally, once you are able to work with `use-bknix` and `loco`, you can move on to [using `civibuild`](../civibuild.md) within the subshell.
docs/tools/buildkit/vagrant.md 0 → 100644
View file @ 11817d0b
In the _Vagrant_ approach, you install `buildkit` in a local virtual-machine. It looks and feels like a remote server.
!!! warning "The Vagrant integration has not been updated in several years."
??? info "Major features"
These instructions will create a virtual-machine with the following features:
* __Interpreters__: PHP-CLI, NodeJS
* __Servers__: Apache HTTPD, MySQL, PHP-FPM, Mailcatcher, Memcached
* __Web Applications__: phpMyAdmin
Additionally, it includes a copy of buildkit.
??? info "Trade-offs"
* __Pro__: The _Vagrant_ approach works equally well on Windows, MacOS, and Linux. Configuration is packaged, editable, and disposable.
* __Con__: Using _Vagrant_ feels like using a remote server, so it may be difficult to integrate debuggers and other desktop tools. File-sync can be a performance drag. At time of writing, may not work well with arm64 CPUs.
## Requirements
* [Vagrant](https://www.vagrantup.com/) and a compatible hypervisor
* Intel/AMD CPU (*The Vangrant core may be compatible with other CPUs, but there are currently compatibility issues with hypervisors and specific Vagrantboxes.*)
## Download
The `civicrm-buildkit-vagrant` project provides a virtual machine with all system requirements. For more instructions, see the README:
* [https://github.com/civicrm/civicrm-buildkit-vagrant](https://github.com/civicrm/civicrm-buildkit-vagrant)
## Next steps
Once you are able to login to the Vagrantbox, you can move on to [using `civibuild`](../civibuild.md).
docs/tools/buildkit/virtual.md 0 → 100644
View file @ 11817d0b
The __Virtual Desktop__ is an appliance. It includes a graphical environment ([XFCE](https://www.xfce.org/)), web services ([nix](nix.md)), and editor ([VS Code](https://code.visualstudio.com/)).
The virtual desktop is primarily intended to facilitate *education and training*. It provides broad functionality and broad compatibility with minimal
configuration work. However, it may not have the same speed or the same applications as your regular desktop.
??? info "Major features"
These instructions will create a virtual-machine with the following features:
* __Interpreters__: PHP-CLI, NodeJS
* __Servers__: Apache HTTPD, MySQL, PHP-FPM, MailHog, Redis <!-- Memcached is installed but disabled by default -->
* __Web Applications__: phpMyAdmin
* __Desktop Applications__: Firefox, Gedit, Visual Studio Code, XFCE4
Additionally, the virtual-machine includes a copy of `buildkit` in `$HOME/buildkit`.
??? info "Trade-offs"
* __Pro__: Comprehensive functionality and compatibility. Little configuration. Solid isolation from main desktop. Perform one big download rather than many small downloads.
Closely matches the official test-environment on civicrm.org.
* __Con__: Virtualization reduces performance and increases hardware requirements. Large download may take a long time. Hard to share applications or data with your regular desktop.
## Requirements
* Multi-core CPU (Intel/AMD/Apple Silicon)
* 8+ GB RAM
!!! info "By default, the virtual-machine is only configured to use 4 GB. However, the host machine should have extra RAM for its own purposes."
* 5 - 15 GB storage space
!!! info "The base virtual-machine is only 3-5 GB. However, you may need extra working-space to facilitate installation."
* Virtualization software
!!! info "The suggested software depends on your operating-system and CPU."
Here are some configurations that should work with this virtual-machine:
| OS | CPU | Suggested Software |
| -- | -- | -- |
| Linux | Intel/AMD (x86-64) | [Virtual Box](http://virtualbox.org/) or [VMWare Workstation Player](https://www.vmware.com/products/workstation-player.html) |
| Windows | Intel/AMD (x86-64) | [Virtual Box](http://virtualbox.org/) or [VMWare Workstation Player](https://www.vmware.com/products/workstation-player.html) |
| MacOS | Intel/AMD (x86-64) | [Virtual Box](http://virtualbox.org/) or [VMWare Fusion](https://www.vmware.com/products/fusion.html) |
| MacOS | Apple M1(*) (arm64) | [Mac UTM](https://mac.getutm.app/) |
Microsoft HyperV, Linux KVM, and Parallels Desktop may work, but they have not been tested, and they require an extra conversion step.
## Download
Choose the download based on your CPU:
| CPU | Download |
| -- | -- |
| Intel/AMD (x86-64) | [CiviDev-2.0.alpha4-x86.ova](https://storage.googleapis.com/civicrm/civicrm-buildkit-appliance/CiviDev-2.0.alpha4-x86.ova) |
| Apple M1(*) (arm64) | [CiviDev-2.0.alpha4-arm64.utm.zip](https://storage.googleapis.com/civicrm/civicrm-buildkit-appliance/CiviDev-2.0.alpha4-arm64.utm.zip) |
## Import/Open VM {:#import-x86}
You will need to import or open the virtual-machine. Here are instructions for each application:
| Software | Instructions |
| -- | -- |
| VirtualBox | Open the `CiviDev-*.ova` file using ["File: Import Appliance"](https://docs.oracle.com/en/virtualization/virtualbox/6.0/user/ovf.html) |
| VMWare Workstation Player | Open the `CiviDev-*.ova` file using ["File: Open a Virtual Machine"](https://docs.vmware.com/en/VMware-Workstation-Player-for-Linux/16.0/com.vmware.player.linux.using.doc/GUID-DDCBE9C0-0EC9-4D09-8042-18436DA62F7A.html) |
| VMWare Fusion | Open the `CiviDev-*.ova` file using ["File: Import"](https://docs.vmware.com/en/VMware-Fusion/12/com.vmware.fusion.using.doc/GUID-275EF202-CF74-43BF-A9E9-351488E16030.html) |
| Parallels | Convert and open the `CiviDev-*.ova` file using [Convert from OVF](https://kb.parallels.com/en/122835) |
| Mac UTM | In the MacOS "Finder", extract the `CiviDev-*.zip` file. This will create the `CiviDev-*.utm` folder. Double-click to run. |
!!! tip "Fine tune the virtual hardware"
When you import or open the virtual machine, several default values will be set. Consider updating these values for improved performance:
* __RAM__: Expand to 6 GB
* __CPU__: Expand to 6 cores
* __(*VirtualBox*) Video RAM__: Expand to 128 MB
* __(*VirtualBox*) Video Driver__: VMSVGA ([details](https://superuser.com/questions/1403123/what-are-differences-between-vboxvga-vmsvga-and-vboxsvga-in-virtualbox))
## Startup and Login
* Start the newly imported virtual-machine
* Login as user "cividev" with password "cividev"
## Next steps
After you login, there will be a `README.md` on the desktop. It has suggestions for next steps.
docs/tools/civi-test-run.md
View file @ 11817d0b
......@@ -4,7 +4,7 @@
## Installation
`civi-test-run` is included within [buildkit](/tools/buildkit.md).
`civi-test-run` is included within [buildkit](buildkit.md).
## Usage
......
docs/tools/civibuild.md
View file @ 11817d0b
......@@ -21,6 +21,10 @@ For a list of available build-types as well as documentation on writing build sc
For example, at time of writing, it includes:
* CiviCRM (standalone)
* `standalone-clean`: A clean "out of the box" installation of CiviCRM. (*Composer-first with private source-tree; more attuned to CiviCRM usage*)
* `standalone-dev`: A clean "out of the box" installation of CiviCRM. (*Git-first with public source-tree; more attuned to CiviCRM development*)
* __TIP__: _Both builds use a mix of `composer` and `git`. In the `dev` build, `git` is the __primary__ mechanism for switching versions/branches/patches. In the `clean` build, `composer` is primary. `composer` requires extra care for switching branches and publishing assets._
* CMS builds
* Backdrop
* `backdrop-clean`: A clean "out of the box" installation of CiviCRM.
......@@ -30,21 +34,17 @@ For example, at time of writing, it includes:
* `drupal-clean`: Drupal 7 CMS: A clean "out of the box" installation of CiviCRM.
* `drupal-demo`: Drupal 7 CMS: A demo site running CiviCRM.
* `drupal-empty`: Drupal 7 CMS: An empty site without CiviCRM. Useful for testing tarball installation.
* Drupal 8 - Standard file-layout, per tarball and drush (`vendor` **within** web root)
* `drupal8-clean`: A clean "out of the box" installation of CiviCRM.
* ~~`drupal8-demo`~~: A demo site running CiviCRM.
* `drupal8-empty`: An empty site without CiviCRM.
* Drupal 8 - Contributed file-layout, per `drupal-composer/drupal-project` (`vendor` **outside of** web root)
* `d8prj-empty`: An empty site without CiviCRM.
* Drupal 10
* `drupal10-clean`: A clean "out of the box" installation of CiviCRM. (Based on `drupal/recommended-project`)
* `drupal10-demo`: A demo site running CiviCRM. (Based on `drupal/recommended-project`)
* `drupal10-empty`: An empty site without CiviCRM. (Based on `drupal/recommended-project`)
* `drupal10-dod`: A minimalist build in the style of Drupal.org development. (Based on `drupal/drupal`)
* Joomla
* ~~`joomla-demo`~~: WIP/incomplete/broken.
* `joomla-empty`: Joomla CMS: An empty Joomla site (without CiviCRM). Useful for testing tarball installation.
* WordPress
* `wp-demo`: WordPress CMS: A demo site running WordPress and CiviCRM.
* `wp-empty`: WordPress CMS: An empty without CiviCRM. Useful for testing tarball installation.
* Other
* ~~`hrdemo`~~: A demo site running Drupal, CiviCRM, and CiviHR.
* ~~`symfony`~~: An experimental hybrid site running Drupal 7, Symfony 2, and CiviCRM.
* Special builds
* `cxnapp`: A self-signed CiviConnect app based on the reference implementation.
* `dist`: A website containing nightly builds akin to dist.civicrm.org. Useful for preparing CiviCRM tarballs.
......@@ -52,27 +52,27 @@ For example, at time of writing, it includes:
* `extdir`: A mock website akin to civicrm.org/extdir/ . Useful for testing the extension download process.
* `l10n`: WIP - A build environment for creating translation files.
* `messages`: A backend service for delivering in-app messages (eg "Getting Started").
* [`universe`](/tools/universe.md): A broad collection of publicly visible repos, extensions, infrastructure, etc.
* [`universe`](universe.md): A broad collection of publicly visible repos, extensions, infrastructure, etc.
Build types can be mixed/matched with different versions of Civi, e.g.,
```bash
$ civibuild create my-drupal-civi47 \
$ civibuild create my-drupal-civimaster \
--type drupal-demo \
--civi-ver master \
--url http://my-drupal-civi47.localhost
$ civibuild create my-drupal-civi46 \
--url http://my-drupal-civimaster.localhost
$ civibuild create my-drupal-civi5.38 \
--type drupal-demo \
--civi-ver 4.6 \
--url http://my-drupal-civi46.localhost
$ civibuild create my-wordpress-civi4719 \
--civi-ver 5.38 \
--url http://my-drupal-civi538.localhost
$ civibuild create my-wordpress-civi539 \
--type wp-demo \
--civi-ver 4.7.19 \
--civi-ver 5.39.0 \
--cms-ver 4.0 \
--url http://my-wp-civi4719.localhost
--url http://my-wp-civi539.localhost
```
The `--civi-ver` argument will accept any branch or version tag. *Note: the 4.7 version is in the `master` branch.*.
The `--civi-ver` argument will accept any branch or version tag. The `master` branch is the latest development branch.
You can also specify `--patch` with a pull request URL to apply those changes on top of your CiviCRM version.
......@@ -147,7 +147,7 @@ cd ~/buildkit/build/dmaster/sites/all/modules/civicrm/
### Check the status of all git repos {:#upgrade-site-git-scan}
There are multiple git repos in your build (`civicrm-core.git`, `civicrm-packages.git`, etal). Before making a major switch, first double-check that all of these repos are in sane condition &mdash; i.e. there shouldn't be any uncommitted changes, and the repos should be on normal branches. For this purpose, use [git-scan](https://github.com/totten/git-scan) (installed with [buildkit](/tools/buildkit.md)).
There are multiple git repos in your build (`civicrm-core.git`, `civicrm-packages.git`, etal). Before making a major switch, first double-check that all of these repos are in sane condition &mdash; i.e. there shouldn't be any uncommitted changes, and the repos should be on normal branches. For this purpose, use [git-scan](https://github.com/totten/git-scan) (installed with [buildkit](buildkit.md)).
```
git scan status
......@@ -168,10 +168,10 @@ git scan up
!!! tip
If you didn't cleanup earlier, then "fast-forward" may not be possible. It takes some judgment to decide what to do &mdash; e.g. a "merge" versus "rebase". Rather than risk a wrong decision, `git scan` will skip these repos and display warnings instead.)
Alternatively, if you'd like to hop to a specific tag, you can use `givi` (a tool included with [buildkit](/tools/buildkit.md)), but keep in mind that if you hop to a *previous* tag with code that expects a different database scheme, there will be no way to run database downgrades.
Alternatively, if you'd like to hop to a specific tag, you can use `givi` (a tool included with [buildkit](buildkit.md)), but keep in mind that if you hop to a *previous* tag with code that expects a different database scheme, there will be no way to run database downgrades.
```
givi checkout 4.7.17
givi checkout 5.39.0
```
### Update the generated code, config files, databases {:#upgrade-site-update}
......@@ -192,7 +192,7 @@ drush civicrm-upgrade-db
## Downgrading a site you installed with civibuild {:#downgrade-site}
If you are [reviewing a pull request](/core/pr-review.md) you may wish to *downgrade* a civibuild site in order to begin replicating the issue and testing the fix. Currently this is **not possible** with civibuild, so instead you will need to do a [rebuild](#rebuild) with the the `--civi-ver` argument to specify your target version of CiviCRM.
If you are [reviewing a pull request](../core/pr-review.md) you may wish to *downgrade* a civibuild site in order to begin replicating the issue and testing the fix. Currently this is **not possible** with civibuild, so instead you will need to do a [rebuild](#rebuild) with the the `--civi-ver` argument to specify your target version of CiviCRM.
## Rebuilds {:#rebuild}
......
docs/tools/civilint.md
View file @ 11817d0b
......@@ -20,8 +20,8 @@ git diff --name-only master | civilint -
See also:
- [CiviCRM Coding Standards](/standards/php.md)
- [CiviCRM Javascript Standards](/standards/javascript.md)
- [CiviCRM Coding Standards](../standards/php.md)
- [CiviCRM Javascript Standards](../standards/javascript.md)
- [Drupal Coding Standards](https://www.drupal.org/docs/develop/standards/coding-standards)
- [PHP_CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer)
- [coder](https://github.com/civicrm/coder)
......
docs/tools/debugging.md
View file @ 11817d0b
......@@ -32,13 +32,13 @@ After enabling debugging, append any of the following name-value pairs to the UR
### Displaying a backtrace
The backtrace can be enabled independently of debugging. If this option is selected, a backtrace will be displayed even if debugging is disabled. **A backtrace is normally displayed only when an error is encountered.** To manually trigger a backtrace in your custom code, see "Changing source code" (below).
The backtrace can be enabled independently of debugging. If this option is selected, a backtrace will be displayed even if debugging is disabled. **A backtrace is normally displayed only when an error is encountered.** To manually trigger a backtrace in your custom code, see *Changing source code* (below).
A backtrace is a list of all the functions that were run in the execution of the page, and the PHP files that contain these functions. It can be really useful in understanding the path that was taken through code, what gets executed where, etc.
## Viewing log files
CiviCRM's log files are stored in the `ConfigAndLog` directory within CiviCRM's local file storage (see [the File System documentation](/framework/filesystem.md) for details on your CMS). Most runtime errors are logged here, as well as data that you explicitly write to log using the `CRM_Core_Error::debug log=true` parameter.
CiviCRM's log files are stored in the `ConfigAndLog` directory within CiviCRM's local file storage (see [the File System documentation](/dev/en/latest/framework/filesystem) for details specific to your CMS). Most runtime errors are logged here, as well as data that you explicitly write to log using the `CRM_Core_Error::debug log=true` parameter.
One may also write to the log file without creating an error with:
`Civi::log()->debug(__FUNCTION__);`
......@@ -46,7 +46,8 @@ This example will write the function-name to ConfigAndLog using the PHP constant
## Changing file-based settings
The following values can be added to your site's settings file `civicrm.settings.php` to assist in debugging:
The following values can be added to your site's settings file `civicrm.settings.php` to assist in debugging. In addition some of them can be passed in via the `env` command - eg.
` env CIVICRM_DEBUG_LOG_QUERY=flush_log drush cvapi System.flush` will log the queries from that command (and only those) to a log file with the name 'flush_log' in the file name. Any value accessed by `CRM_Utils_Constant::value()` can be set on a command-specific basis using `env`
- `define('CIVICRM_MAIL_LOG', 1);` causes all outbound CiviCRM email to be written to a log file. No real emails are sent.
......@@ -56,6 +57,8 @@ The following values can be added to your site's settings file `civicrm.settings
- `define('CIVICRM_DEBUG_LOG_QUERY', 1);` outputs all SQL queries to a log file.
- `define('CIVICRM_DEBUG_LOG_QUERY', 'n');` outputs all SQL queries to a log file, includes 'n' in the file name (useful if you want to capture some portion of the queries in a separate log file). **Added in CiviCRM 5.30.**
- `define('CIVICRM_DEBUG_LOG_QUERY', 'backtrace');` will include a backtrace of the PHP functions that led to the query.
- `define('CIVICRM_DAO_DEBUG', 1);` writes out various data layer queries to your browser screen.
......@@ -64,15 +67,17 @@ The following values can be added to your site's settings file `civicrm.settings
- `define('CIVICRM_TEMP_FORCE_DURABLE', 1);` forces temp tables to be created as non temporary tables. Note that you may need to manually delete them very pro-actively while this is on as it will result in errors in some cases
!!! tip
When any sort of "logging stuff to a file" is enabled by one of the
above settings, check the `ConfigAndLog` directory within the local files
directory to find the log. (See [the File System
documentation](/framework/filesystem.md) for the location in your CMS.)
??? tip "Log file locations"
When any sort of "logging stuff to a file" is enabled by one of the above settings, check the `ConfigAndLog` directory within the local files directory to find the log.
(See [the File System documentation](/framework/filesystem.md) for the location in your CMS.)
!!! tip
??? tip "Sites built with CiviBuild"
If you're developing within a site built with [civibuild](/tools/civibuild.md), then you can to apply settings to *all your local sites* by using the [civicrm.settings.d folders](/tools/civibuild.md#settings-civicrm).
??? tip "Parsing logged queries with System Tools"
You can parse the output of the queries logged to the debug log into a csv using the [system tools extension](https://civicrm.org/extensions/system-tools) (in non-backtrace mode)
## Viewing a query log from MySQL
Outside of CiviCRM, the [MySQL General Query Log](https://dev.mysql.com/doc/refman/en/query-log.html) log allows MySQL to log all queries. This is a performance killer, so avoid using it in production!
......@@ -220,7 +225,7 @@ XDebug isn't the only PHP debugger, but it's the one we recommend for CiviCRM de
#### Debian / Ubuntu Linux (System Packages)
```bash
sudo apt-get install php5-xdebug
sudo apt-get install php-xdebug
```
#### Red Hat / CentOS Linux (System Packages)
......@@ -259,7 +264,7 @@ After installing NetBeans, open your local CiviCRM installation in NetBeans and
#### MacGDBp
[MacGDBp](http://www.bluestatic.org/software/macgdbp/) is a lighter-weight option, only available for OS X. After installing MacGDBp, launch it and make sure it says "Connecting" at the bottom in the status bar. If it doesn't, click the green "On"
[MacGDBp](http://www.bluestatic.org/software/macgdbp/) is a lighter-weight option, only available for OS X. After installing MacGDBp, launch it and make sure it says "Connecting" at the bottom in the status bar. If it doesn't, click the green **On**
button in the upper-right corner to enable it. The next time you access CiviCRM, the web browser will appear to hang. If you click on MacGDBp, you'll see that it has stopped on the first line of code in CiviCRM. From there you can step through the code to the part you're interested in. But it's probably a better idea to set a breakpoint in the part of the code you're interested in stopping at. See the MacGDBp documentation for more information.
## Error 500, white screen (WSoD), "Internal Server Error" errors
......@@ -337,7 +342,7 @@ Javascript (JS) is what makes web-apps interactive. Without it, nothing on a web
AJAX is a complicated-sounding acronym for the simple task of loading more stuff from the server on-demand. When you see a spinning triangle or have to wait for autocomplete results, the content is loading via AJAX. The J stands for Javascript, so if your site is having JS toubles, AJAX may not work either.
### Troubleshooting Tools
Most of the time the other debugging methods metioned earlier will solve 90% of the issues. However sometimes you need to use your browser to debug. Your web-browser (Firefox/Chrome/etc.) has a javascript console where you can view error messages. Press F12 to turn it on and then click over to the "Console" tab. With these tools enabled, try triggering the symptom again and watch for error messages on the screen or in the console. If you see an error message, try doing a google search for it to see if others have solved the same problem.
Most of the time the other debugging methods metioned earlier will solve 90% of the issues. However sometimes you need to use your browser to debug. Your web-browser (Firefox/Chrome/etc.) has a javascript console where you can view error messages. Press F12 to turn it on and then click over to the **Console** tab. With these tools enabled, try triggering the symptom again and watch for error messages on the screen or in the console. If you see an error message, try doing a google search for it to see if others have solved the same problem.
### Problems and Solutions
* Incompatible/buggy extensions for CiviCRM might crash javascript. Try temporarily turning them all off. If your problem is fixed, re-enable them one at a time to find the culprit.
......@@ -347,6 +352,7 @@ Most of the time the other debugging methods metioned earlier will solve 90% of
* Certain website plugins have been known to cause javascript errors. Check this list of known incompatibilities.
* Ensure you are using the latest version of CiviCRM.
* Clear your site's caches.
* WordPress users may experience some interference by WordFence - especially after CiviCRM or WordFence are newly installed. [Putting WordFence in learning mode may help](https://lab.civicrm.org/dev/core/-/issues/3887)
If none of the above works:
Create a new post on Stack Exchange. Indicate that you have already read this and list what solutions you tried and what happened. List exact steps to reproduce the problem, and paste in the complete error message(s) you are seeing.
......
docs/tools/git.md
View file @ 11817d0b
......@@ -3,7 +3,7 @@
CiviCRM uses git, GitHub, and GitLab to manage changes to the code. A solid understanding of these tools (especially git) is important for developing CiviCRM. This page provides some information about *how CiviCRM uses these tools* &mdash; but, due to the wealth of resources already available elsewhere online, this page does *not* attempt to teach you everything you need know about how to use these tools for CiviCRM development.
!!! tip
If you are new to git, a great way to get started using it within the CiviCRM community is to [contribute to documentation](/documentation/index.md). The editing workflow involves git in the same was that core coding does &mdash; but the stakes are much lower!
If you are new to git, a great way to get started using it within the CiviCRM community is to [contribute to documentation](../documentation/index.md). The editing workflow involves git in the same way that core coding does &mdash; but the stakes are much lower!
## External resources {:#resources}
......@@ -42,24 +42,24 @@ CiviCRM uses git, GitHub, and GitLab to manage changes to the code. A solid unde
Whether you are contributing to civicrm-core or an ancillary project (using GitHub or GitLab) the process generally goes somewhat like this:
1. Consider [opening an issue on GitLab](/tools/issue-tracking.md#gitlab) to describe the change you'd like to make. Not all changes need GitLab issues, but opening an issue is recommended if you are making significant changes, expect discussion, or expect your changes to be grouped into more than one [pull request](#pr).
1. Consider [opening an issue on GitLab](issue-tracking.md#gitlab) to describe the change you'd like to make. Not all changes need GitLab issues, but opening an issue is recommended if you are making significant changes, expect discussion, or expect your changes to be grouped into more than one [pull request](#pr).
1. Find the page on GitHub or GitLab for the project to which you would like to contribute. We will call this repository the **upstream repository**.
1. **Clone** the upstream repository to your local machine. (If you are working on core, you should use [civibuild](/tools/civibuild.md) for this step.)
1. **Clone** the upstream repository to your local machine. (If you are working on core, you should use [civibuild](civibuild.md) for this step.)
1. On the web page for the upstream repository, **fork** the upstream repository to your personal user account.
1. Within your local repository **add your fork** as a second git *remote*. *[Learn more...](#remotes)*
1. **Choose the correct base branch** in the upstream repository as the starting point for your changes. (Usually this will be `master`.) *[Learn more...](#base-branch)*
1. (If it's been some time since you've cloned) **pull or fetch** the latest changes from the *upstream repository* into the appropriate branch of your local repository. *(You might also need to [upgrade your civibuild site](/tools/civibuild.md#upgrade-site).)*
1. (If it's been some time since you've cloned) **pull or fetch** the latest changes from the *upstream repository* into the appropriate branch of your local repository. *(You might also need to [upgrade your civibuild site](civibuild.md#upgrade-site).)*
1. Create (and checkout) a **new branch** for your changes, based on the correct branch (chosen above) in the upstream repository. *[Learn more...](#branching)*
1. Make your changes. (Take care to follow the guidelines in [contributing to core](/core/contributing.md).)
1. Make your changes. (Take care to follow the guidelines in [contributing to core](../core/contributing.md).)
1. **Commit** your changes. *[Learn more...](#commiting)*
1. **Push** your changes *to your fork*.
1. **Open a pull request**. *[Learn more...](#pr)*
1. Wait for someone else to [review your pull request](/core/pr-review.md).
1. Wait for someone else to [review your pull request](../core/pr-review.md).
1. If you need to make more changes later, commit them on the same branch and push your new commits to your fork. The new commits the will automatically appear in the pull request.
1. If other people commit changes to the upstream repository which create *merge conflicts* in your pull request, then **rebase** your branch. *[Learn more...](#rebasing)*
1. Once your changes are merged, delete your local branch
See also: [reviewing someone else's pull request](/core/pr-review.md)
See also: [reviewing someone else's pull request](../core/pr-review.md)
## Pull requests {:#pr}
......@@ -69,11 +69,13 @@ See also: [reviewing someone else's pull request](/core/pr-review.md)
### Creating a pull request {:#pr-submit}
1. In the web browser, navigate to the web page for your fork (e.g. `https://github.com/myuser/civicrm-core` )
1. In the web browser, navigate to the web page for your fork (e.g. `https://github.com/myuser/civicrm-core` ).
1. Click **Pull Request**
1. There will be two branches specified – the (first) left should be "civicrm" (i.e. where the code is going to). The second (right) should be your branch.
1. Add a [good subject](#pr-subject) and explanation, and submit.
To note: it's not possible to create a pull request for the civicrm-core repo on GitLab ─ use GitHub for this.
### Writing a pull request subject {:#pr-subject}
Pull request titles don't need to be identical to issue titles, and in particular, you may want to focus more positively on the changes in code than on the broader feature changes. Here are some guidelines for writing a good subject line:
......@@ -89,8 +91,8 @@ When filing a pull-request, use a descriptive subject. These are good examples:
A few elements to include:
* **Acronyms** - You're welcome to use the [acronyms](#acronyms) below to flag your PR with certain characteristics.
* **dev/_project_#_XXXX_** - This is a [GitLab issue reference](/tools/issue-tracking.md#gitlab-reference).
* **CRM-_XXXXX_** - This is a reference to the now-deprecated [Jira issue tracker](/tools/issue-tracking.md#jira). A bot will set up crosslinks between JIRA and GitHub.
* **dev/_project_#_XXXX_** - This is a [GitLab issue reference](issue-tracking.md#gitlab-reference).
* **CRM-_XXXXX_** - This is a reference to the now-deprecated [Jira issue tracker](issue-tracking.md#jira). A bot will set up crosslinks between JIRA and GitHub.
* **Description** - Provide a brief description of what the pull-request does.
### Acronyms within PR subjects {:#acronyms}
......@@ -124,17 +126,19 @@ Here are some examples and counter-examples of NFC:
* Modify whitespace in PHP code.
* Update a code comment.
* Fix a typo or grammatical error in a help dialog.
* (*Maybe*) Add a new unit-test where there was no coverage before.
* (*Maybe*) Add a new unit-test where there was no coverage before (see below).
* (*Maybe*) Update a PHP doc block (see below).
* _Functional Change_:
* Refactoring
* Replace 20 lines of redundant code with a call to a helper function.
* Refactoring, e.g. replacing 20 lines of redundant code with a call to a helper function.
* (__Why?__ A reviewer would consider whether the helper is truly equivalent, better, or worse.)
* Fix a typo in a *symbol* (PHP class-name, PHP function-name, HTML field name, etc).
* (__Why?__ A reviewer would consider dangling references to the symbol.)
* Change the general wording of a help dialog or menu item.
* (__Why?__ A reviewer would consider impact on the user's comprehension.)
* Alter the substance of an existing unit-test.
* Add or alter an existing unit-test.
* (__Why?__ A reviewer would consider whether the change improves the correctness of the test.)
* Update annotations in a PHP doc block.
* (__Why?__ Some annotations affect functionality, e.g. the `@required` annotation on an APIv4 param.)
* Alter the build process.
* (__Why?__ A reviewer would consider whether the new build will work correctly.)
......@@ -146,7 +150,7 @@ There is no size limit for PRs as long as they are focused on completely solving
### Reviewing a pull request
See [How to review a core pull request](/core/pr-review.md)
See [How to review a core pull request](../core/pr-review.md)
### Who merges pull requests? {:#pr-merge}
......@@ -233,13 +237,17 @@ $ git checkout upstream/master -b mail-111
When creating a new branch, you should explicitly declare a starting point.
Most of the time, your base branch should be `master`. In special circumstances, a patch may be accepted for the *Release Candidate*, or *Stable* branch. The table below summarizes the policies for each branch.
Most of the time, your base branch should be `master`. In special circumstances, a patch may be accepted for the *Release Candidate* (RC), or *Stable* branch. The table below summarizes the policies for each branch.
| Name | Git branch (example) | Version number (example) | Acceptable patches |
| --- | ---- | --- | --- |
| Master | master | 5.99.alpha1 | This is primary target for most patches, including typical bugfixes, cleanups, and minor features. |
| Release Candidate | 5.98 | 5.98.beta1 | Fixes for critical, recent regressions. The regression should be traced to a specific, recent change. In a typical cycle, only 1-10 RC patches are accepted.|
| Stable | 5.97 | 5.97.1 | Backports of fixes for very critical issues. |
| Stable | 5.97 | 5.97.1 | Backports of fixes for very critical issues. This is usually only done by the project maintainers. Make your PR against the RC instead and they will backport if necessary. |
If you are unsure about which is the current RC or master branch, you can refer to https://download.civicrm.org/latest/, and look at the numbers in brackets to the right of the filename, e.g. if you see `civicrm-RC-drupal.tar.gz (5.24.beta1-...)` it means the current release candidate branch is the 5.24 branch.
Don't make multiple PRs against multiple branches for the same thing even if the bug is present in both versions, e.g. both the RC and master. Consult the table above and just pick one branch.
### Committing {:#committing}
......
docs/tools/index.md
View file @ 11817d0b
......@@ -2,26 +2,26 @@
## Tools included with buildkit {:#with-buildkit}
When you install [buildkit](/tools/buildkit.md) you'll get all these tools.
When you install [buildkit](buildkit.md) you'll get all these tools.
*This list of tools is also maintained [in the buildkit readme file](https://github.com/civicrm/civicrm-buildkit/blob/master/README.md).*
### CiviCRM-specific tools {:#civicrm-specific}
* `civibuild` - Build a complete source tree (with CMS+Civi+addons), provision httpd/sql, etc.
* *[documentation](/tools/civibuild.md)*
* *[documentation](civibuild.md)*
* *[repository](https://github.com/civicrm/civicrm-buildkit)*
* `cv` - command is a utility for interacting with a CiviCRM installation
* *documentation: run `cv list`*
* *[repository](https://github.com/civicrm/cv)*
* `civix` - Generate skeletal code for CiviCRM extensions
* *[documentation](/extensions/civix.md)*
* `civix` - CiviCRM extension code generator
* *[documentation](../extensions/civix.md)*
* *[repository](https://github.com/totten/civix)*
* `civistrings` - Scan code for translatable strings (*.pot)
* *documentation: run `civistrings --help`*
* *[repository](https://github.com/civicrm/civistrings)*
* `cividist` - Generate tarballs from a series of git branches/tags
* *[documentation](/tools/cividist.md)*
* *[documentation](cividist.md)*
* *repository: [within civicrm-buildkit](https://github.com/civicrm/civicrm-buildkit/blob/master/bin/cividist)*
* `gitify` - Convert a CiviCRM installation to a git repo
* *documentation: run `gitify` with no arguments*
......@@ -35,16 +35,16 @@ When you install [buildkit](/tools/buildkit.md) you'll get all these tools.
* `civicrm-upgrade-test` - Scripts and data files for testing upgrades
* *[documentation& repository](https://github.com/civicrm/civicrm-upgrade-test)*
* `civi-test-run` - Run one or more test suites
* *[documentation](/tools/civi-test-run.md)*
* *[documentation](civi-test-run.md)*
* *repository: [within civicrm-buildkit](https://github.com/civicrm/civicrm-buildkit/blob/master/bin/civi-test-run)*
* Coder - Configure phpcs for CiviCRM's [coding standards](/standards/php.md)
* Coder - Configure phpcs for CiviCRM's [coding standards](../standards/php.md)
* *[documentation & repository](https://github.com/civicrm/coder)*
* *(Derived from [Drupal's coder project](https://www.drupal.org/project/coder))*
### External tools installed with buildkit {:#external}
These tools are not specific to CiviCRM, so you may already have some of them installed on your system. If you install [buildkit](/tools/buildkit.md) you'll get all these tools at once, in addition to the CiviCRM-specific tools listed above.
These tools are not specific to CiviCRM, so you may already have some of them installed on your system. If you install [buildkit](buildkit.md) you'll get all these tools at once, in addition to the CiviCRM-specific tools listed above.
* Dependency management
* [composer](http://getcomposer.org/) - Manage dependencies for PHP code.
......@@ -72,13 +72,13 @@ These tools are not specific to CiviCRM, so you may already have some of them in
* [git](https://git-scm.com/) - version control system
* [psysh](http://psysh.org/) - a reply-echo-print-loop for PHP (like `php -a`, but better)
* [MySQL Workbench](https://www.mysql.com/products/workbench/) - A graphical interface to your local (or remote) MySQL server
* [MkDocs](http://www.mkdocs.org) - for [editing documentation](/documentation/index.md)
* [MkDocs](http://www.mkdocs.org) - for [editing documentation](../documentation/index.md)
### Text editors
If you already have a text editor you love, then stick to that. If you're new and need some recommendations, here are some of the most popular text editors among CiviCRM developers:
* [PhpStorm](https://www.jetbrains.com/phpstorm/) *(See [CiviCRM-specific notes on PhpStorm](/tools/phpstorm.md))*
* [PhpStorm](https://www.jetbrains.com/phpstorm/) *(See [CiviCRM-specific notes on PhpStorm](phpstorm.md))*
* [NetBeans](https://netbeans.org/)
* [Sublime](https://www.sublimetext.com/)
* [Atom](https://atom.io/)
......
docs/tools/issue-tracking.md
View file @ 11817d0b
......@@ -26,7 +26,7 @@ Issues about CiviCRM itself are opened within the [Development Team's projects](
#### Referencing GitLab issues {:#gitlab-reference}
Use text like `dev/core#1` to reference specific issues in your [commit messages](/tools/git.md#commit-messages), [pull request subjects](/tools/git.md#pr-subject), and other communication.
Use text like `dev/core#1` to reference specific issues in your [commit messages](git.md#commit-messages), [pull request subjects](git.md#pr-subject), and other communication.
In the above example:
......@@ -37,7 +37,7 @@ In the above example:
!!! tip
GitLab displays the issue reference at the bottom of the right-hand column along with a link that copies the reference to your clipboard:
![Screenshot of GitLab issue reference](/img/gitlab-reference.png)
![Screenshot of GitLab issue reference](../img/gitlab-reference.png)
### GitHub {:#github}
......@@ -68,7 +68,7 @@ Used as an issue-tracking system until early 2018 for:
### When to create an issue {:#when-to-create}
If you are ready to make a change to CiviCRM, you can [submit a pull request](/tools/git.md#pr) *without* creating an issue first.
If you are ready to make a change to CiviCRM, you can [submit a pull request](git.md#pr) *without* creating an issue first.
Otherwise, you'll want to create an issue in order to track work that might need to be done at some point in the future.
......@@ -146,17 +146,17 @@ Better yet, describe the issue distinctly and coherently yourself. If you find
A rule of thumb is that if an issue has more than 2 or 3 pull requests in GitHub (described below), something is wrong. It may be a series of false starts, and that's okay, but if it's a bunch of pull requests against the same repository, you probably should have opened new issues to describe the separate features or bugs&mdash;or to document a regression or feature gap.
See also: [pull request scope](/tools/git.md#pr-scope)
See also: [pull request scope](git.md#pr-scope)
#### Categorization {:#categorization}
Categorization is useful for finding issues, and it also determines how issues appear in the release notes.
Use the Labels field to apply any relevant labels to your issue.
Use the Labels field to apply any relevant labels to your issue. You need to join the Development Group on Gitlab in order to be able to apply labels.
#### Confidentiality {:#confidentiality}
Issues can be marked Confidential, this is a useful "middle ground" status for issues which perhaps shouldn't be public but which are **not** security issues. If you have a security issue please report it as described at [Reporting a Security Vulnerability](/security/reporting/).
Issues can be marked Confidential, this is a useful "middle ground" status for issues which perhaps shouldn't be public but which are **not** security issues. If you have a security issue please report it as described at [Reporting a Security Vulnerability](../security/reporting/).
In GitLab a user can be either anonymous (no account), have a GitLab account but not be a project member, or be a project member. The confidential checkbox on issues will hide it from anonymous users and non-project members.
......@@ -183,25 +183,25 @@ Triage is done from this screen https://lab.civicrm.org/groups/dev/-/boards and
### Categorisation
During triage the following labels may be added:
In Gitlab, issues are categorized by "Label", and multiple labels may be applied. A few of the most important labels are standalone terms:
_Critical_
If the bug is a critical bug this label will be added. UPDATE- we don't really add this any more since by definition if a bug has been present for the entire 5.x series it's not critical. It would need to be something really big to get this
* __`regression`__ - A problem which was demonstrably introduced by a recent change (*last few months*).
* __`triaged`__ - The issue has received an initial evaluation/categorization.
* __`prioritised`__ - The issue was designated by the product maintenance team as a priority.
_Regression_
If the bug is a recent regression this label will be added
Additionally, there are [a number of specific labels](https://lab.civicrm.org/dev/core/-/labels) organized with prefixes. This list is easier to navigate if you know the prefixes:
_type: proposal | type:request_
This is intended to identify whether the submitter is proposing to do something (if agreed it would be acceptable) or requesting that a change be made for them
* __`comp:{$X}`__ - The *component* or *subsystem* or [functional area](https://stackoverflow.com/questions/16475979/what-is-the-difference-between-functional-and-non-functional-requirement).
* __Example__: The CiviMail component addresses the functional requirements to compose, send, and track email-blasts. Any bug, feature, or improvement relating to email-blasts would be `comp:CiviMail`.
* __`sig:{$X}`__: The *signficance* or *quality* or [non-functional requirement](https://stackoverflow.com/questions/16475979/what-is-the-difference-between-functional-and-non-functional-requirement).
* __Example__: If a screen unexpectedly terminates after pressing the letter "p", then it is not functioning correctly. That screen has a bug (`sig:bug`).
* __Example__: If a screen runs too slowly or consumes too many resources (CPU/RAM/disk), then the functionality is correct, but the screen has a `sig:performance` issue.
* __`needs:{$X}`__ - The next step that needs work/action.
* __Example__: Suppose an issue has been investigated and a patch was approved for the next release. The patch adds a new field, and this field needs to be explained in the "CiviCRM User Guide". The issue `needs:documentation`.
* __`type:{$X}`__ - The *type* indicates how this issue is being shepherded.
* __Example__: Suppose someone filed an issue because they believed it was important - but they did not have any specific capacity or resources allocated to fixing it. They're looking for other people to help with understanding/resolving it. This is a `type:request`.
_needs:Concept approval| Concept approved_
In most cases the Concept needs approval label will be added in triage as the triager will normally only spend a brief time on each issue and may not be the correct person to assess the proposal. The triager will ping people from the issue or on chat who might be able to comment and / or approve if they have an idea as to who that might be.
_sig:accessibilty|sig:bug|sig:compatibility_
If the bug is significant due to it relating to an area we are trying to focus on (currently bugs, compatibility with Wordpress, Drupal8, and accessibility html compliance) then a label will be added.
_triaged_
Indicates the issue has been triaged & removes it from the triage queue
For the most complete, current list of labels, see [Gitlab: Development > Core > Labels](https://lab.civicrm.org/dev/core/-/labels).
### Escalate any new critical bugs or regressions
......
docs/tools/jenkins.md
View file @ 11817d0b
......@@ -16,9 +16,12 @@ Pull-requests are tested automatically with build-bot software called [Jenkins](
* The pull-request is based on an old version of the codebase, and it cannot be applied cleanly with `git apply` or `git scan am`. Rebasing the PR branch should resolve this.
* On the test node, a local resource (such as disk-space, RAM, or inode count) was exhausted. If the failure was recent (past few hours), seek help on [the infrastructure channel](https://chat.civicrm.org/civicrm/channels/infrastructure). If the failure occurred a few hours or days ago, try running the test again.
* Some part of the build/test toolchain needs attention. For example, a test script may have been changed without supporting an edge-case; or a tool like `bower` or `npm` may need to be upgraded. Seek help on `infrastructure`.
* If the test appears to have failed randomly and you have been added to the whilelist by an admin, you can comment `test this please` to restart the test.
* Code-style tests are executed first. If the code-style in this patch is inconsistent, the remaining tests will be skipped.
* The primary tests may take 20-120 min to execute. This includes the following suites: `api_v3_AllTests`, `CRM_AllTests`, `Civi\AllTests`, `civicrm-upgrade-test`, and `karma`
* There are a handful of unit tests which are time-sensitive and which fail sporadically. See: https://forum.civicrm.org/index.php?topic=36964.0
* If needed, you can also manually [generate a test site](https://test.civicrm.org/job/CiviCRM-Manual/) with the PHP and
MYSQL versions, CMS and proposed patch you need or you can [manually run the test suites](https://test.civicrm.org/job/CiviCRM-Manual-Test/) with the same options.
* The web test suite (`WebTest_AllTests`) takes several hours to execute. [It runs separately -- after the PR has been merged.](https://test.civicrm.org/job/CiviCRM-WebTest-Matrix/)
For detailed discussion about automated tests, see [Testing](/testing/index.md)
For detailed discussion about automated tests, see [Testing](../testing/index.md)
docs/tools/phpstorm.md
View file @ 11817d0b
# Using PhpStorm for CiviCRM Development
[PhpStorm](https://www.jetbrains.com/phpstorm) is a commercial IDE which is popular among CiviCRM developers.
[PHPStorm](https://www.jetbrains.com/phpstorm) is a commercial IDE which is popular among CiviCRM developers.
## General project setup
* Use the root directory of your CMS as your project root (otherwise in-app debugging won't work)
* You can speed up PhpStorm's indexing process by taking the following steps to ignore most files:
1. Settings > Directories
1. Mark all directories as "Excluded"
1. Drill down to find the "civicrm" folder and mark it as "Sources"
1. **Settings > Directories**
1. Mark all directories as **Excluded**
1. Drill down to find the **civicrm** folder and mark it as **Sources**
## Code style setup
Create the 'CiviCRM' code styling preference:
Create the CiviCRM code styling preference:
1. Open you current project's properties: File > Settings
1. Create the CiviCRM code styling preference: Code Style > Manage > Save as, then indicate 'CiviCRM'
1. In the 'Code style' sub-menu, select 'PHP', then 'Set from ...' at the far right, Predefined Style > Drupal (this sets options across all tabs of the dialog)
1. In the 'Code style' sub-menu, select 'Javascript', then 'Set from ...' at the far right, PHP (this replicates all PHP settings to the Javascript section)
1. Click 'Apply' at the bottom of the screen
1. Open **Settings > Editor > Code Style**
1. Select **Scheme > Default** or **Project** if you want these settings to apply only to CiviCRM
1. In the **Code style** sub-menu, select PHP, then **Set from ...** at the far right, **Drupal** (this sets options across all tabs of the dialog), then click **Apply**
1. In the **Code style** sub-menu, select Javascript, then **Set from ...** at the far right, PHP (this replicates all PHP settings to the Javascript section), then click **Apply**
That's it. You can now use this code style on all future CiviCRM-related projects. If you are only developing for CiviCRM, you can also copy this style to the 'Default' style.
That's it. You can now use this code style on all future CiviCRM-related projects.
## XDebug integration
You need to configure XDebug on the webserver, phpstorm on the development machine and a debugger helper in the browser.
To configure XDebug and PHPStorm see: https://www.jetbrains.com/help/phpstorm/configuring-xdebug.html
For browser helpers see: https://confluence.jetbrains.com/display/PhpStorm/Browser+Debugging+Extensions
## Running automated tests from within PhpStorm {:#testing}
!!! note
These instructions assume you already have a working instance of CiviCRM running locally with [buildkit](/tools/buildkit).
From the `Run > Edit Configurations` menu add a new PHPUnit configuration.
## Running automated tests from within PHPStorm {:#testing}
Check "Use alternative configuration file" and choose `phpunit.xml.dist` in your CiviCRM root directory.
*PHPStorm provides convenient tools for running and debugging unit tests in-app.*
Expand "Environment variables" and add the following two:
!!! note
These instructions assume you already have a working instance of CiviCRM running locally with [buildkit](buildkit).
- `TEST_DB_DSN`: Use the DSN to connect to your test database. It is displayed on completion of civibuild. You should also be able to find it by running
`cv vars:show`.
- `CIVICRM_UF`: Use "UnitTests" unless you're running WebTests or End-to-end tests.
1. On the CLI from a directory inside your build, run the command `civibuild phpunit-info`. This outputs configuration settings you will need to paste into PHPStorm.
2. From the `Run > Edit Configurations` menu edit the default PHPUnit configuration for your project.
3. Check "Use alternative configuration file" and choose the one output by the `civibuild` command.
4. Set the custom working directory as specified by the `civibuild` output.
5. Expand "Environment variables" and add the variables shown in the `civibuild` output.
For step by step instructions (with screenshots!) see this [StackExchange answer](https://civicrm.stackexchange.com/questions/16489/how-do-i-run-php-unit-tests-w-xdebug-from-within-phpstorm-on-mac/16497#16497).
For more detailed instructions (with screenshots!) see this [StackExchange answer](https://civicrm.stackexchange.com/questions/16489/how-do-i-run-php-unit-tests-w-xdebug-from-within-phpstorm-on-mac/16497#16497).
### Adding external libraries
It can be frustrating when writing tests the PHPStorm complains about missing classes or undefined methods. This happens because PHPUnit is not included in the CiviCRM codebase.
To remedy this you can add an external content root. You'll need to clone the [phpunit] library locally. Then from that directory check out the latest supported version of phpunit (4.x right now).
After that you just add the directory to your project include paths by [following the instructions on the Jetbrains site][phpstorm-include-paths].
To remedy this you can add an external content root. You'll need to clone the [PHPUnit](https://phpunit.de/) library locally. Then from that directory check out the latest supported version of PHPUnit (4.x right now).
[phpstorm-include-paths]: https://www.jetbrains.com/help/phpstorm/configuring-include-paths.html
[phpunit]: https://github.com/sebastianbergmann/phpunit
After that you just add the directory to your project include paths by [following the instructions on the Jetbrains site](https://www.jetbrains.com/help/phpstorm/configuring-include-paths.html).
Alternatively: always use buildkit to generate you CiviCRM development environment; it ships with many tools - including phpunit4
Alternatively: always use buildkit to generate you CiviCRM development environment; it ships with many tools - including PHPUnit 4
docs/tools/universe.md
View file @ 11817d0b
......@@ -15,7 +15,7 @@ source tree for `getContactDetails` to see how it's being used.
## Create the universe
If your system is configured to support [civibuild](/tools/civibuild.md), then simply run:
If your system is configured to support [civibuild](civibuild.md), then simply run:
```bash
$ civibuild create universe
......
docs/translation/database.md
View file @ 11817d0b
......@@ -43,7 +43,7 @@ While there are many cities where street names can officially be in multiple lan
Similarly, the first and last name of an individual may be written in different alphabets (ex: Latin and Cyrillic), but this is not a frequent use-case worth the complexity. Administrators can workaround this by creating custom fields.
In order to define a field as localizable, the [XML schema definition](/framework/database/schema-definition.md) for that field must have the following tag:
In order to define a field as localizable, the [schema definition](../framework/entities/index.md) for that field must have the following tag:
```
<localizable>true</localizable>
......@@ -143,7 +143,7 @@ VALUES (
Two use-cases:
1. An existing field in CiviCRM was not tagged in the [XML schema](/framework/database/schema-definition.md) as `<localizable>` (ex: the `title` in `civicrm_survey`, before CiviCRM 4.5). After adding the `<localize>` tag in the XML file, you must also add an upgrade snippet for existing databases. Example, from `sql/4.1.0.mysql.tpl`:
1. An existing field in CiviCRM was not tagged in the [schema](../framework/entities/index.md) as `'localizable' => TRUE` (ex: the `title` in `civicrm_survey`, before CiviCRM 4.5). After adding the `<localize>` tag in the XML file, you must also add an upgrade snippet for existing databases. Example, from `sql/4.1.0.mysql.tpl`:
```smarty
{if $multilingual}
......@@ -155,4 +155,4 @@ Two use-cases:
{/if}
```
2. If a localized field was removed or added, the schema does odd things during the upgrade to figure out which fields are mutli-lingual. Rebuilding the multi-lingual schema will check in `CRM/Core/I18n/SchemaStructure.php` for the fields used by the database views. If the schema is changed, copy the `SchemaStructure.php` from the master branch to, for example, `SchemaStructure_4_5_alpha1.php`. The 4.5 alpha1 will then read this file when rebuilding the schema, see `CRM/Core/I18n/Schema.php` for more information (`getLatestSchema`). i.e. during an upgrade, we may be upgrading from 4.0 to 4.5, and when rebuilding the views at each stage, we need to load the correct schema version. Since we do not have a schema file for each minor version, CiviCRM will attempt to load the most relevant schema version to the version of the upgrade step being run.
2. If a localized field was removed or added, the schema does odd things during the upgrade to figure out which fields are mutli-lingual. Rebuilding the multi-lingual schema will check in `CRM/Core/I18n/SchemaStructure.php` for the fields used by the database views. If the schema is changed, copy the `SchemaStructure.php` from the master branch to, for example, `SchemaStructure_4_5_alpha1.php` and rename the class as appropriate. The 4.5 alpha1 will then read this file when rebuilding the schema, see `CRM/Core/I18n/Schema.php` for more information (`getLatestSchema`). i.e. during an upgrade, we may be upgrading from 4.0 to 4.5, and when rebuilding the views at each stage, we need to load the correct schema version. Since we do not have a schema file for each minor version, CiviCRM will attempt to load the most relevant schema version to the version of the upgrade step being run.
docs/translation/index.md
View file @ 11817d0b
# Translation for Developers
When writing new application code, developers should organize their code in a way which is amenable to internationalization, so that it may be localized to various languages and regions of the world.
Developers should write their code so that it may be localized to various languages and regions of the world.
If you are an extension developer, there is additional documentation in the [Extension translation](/translation/extensions.md) page.
If you are an extension developer, there is additional documentation in the [Extension translation](../extensions/translation.md) page.
## PHP
......@@ -11,23 +11,29 @@ If you are an extension developer, there is additional documentation in the [Ext
```php
$string = ts('Hello, World!');
$group = array('' => ts('- any group -')) + $this->_group;
$options = [
1 => ts('One'),
2 => ts('Two'),
3 => ts('Three'),
];
```
* You can also use placeholders for variables:
```php
$string = ts("A new '%1' has been created.", array(1 => $contactType));
$string = ts("A new '%1' has been created.", [1 => $contactType]);
```
Note that variables should themselves be translated by your code before passing in, if appropriate.
* If the string might be singular or plural, use the following syntax:
* If the string needs to be pluralized, use the singular form as the main string, and provide the count (integer) and plural (string) in the 2nd argument along with any placeholder values:
```php
$string = ts('%count item created',
array('count' => $total, 'plural' => '%count items created')
);
$string = ts('%count item was created by %1', [
'count' => $total,
'plural' => '%count items were created by %1',
1 => $userName,
]);
```
## Javascript
......@@ -65,14 +71,13 @@ When translating strings in an extension, ts scope needs to be declared. The `CR
## Best practices
The general rules for avoiding errors may be summed up like this:
The general rules for avoiding errors may be summed up as:
* If the string needs to be parsed (i.e. is in double quotes) then there's probably an error there.
* No string concatenation in the `ts()` calls.
* The second parameter of the `ts()` call must be an array.
* You must pass a literal string into `ts()`, not a variable.
* The first argument to `ts()` must be a single, literal string.
* The string must not contain variables, concatenation, line-breaks, or leading/trailing spaces.
* The second parameter of the `ts()` call is an array of variables to swap for placeholders in the string.
### Avoid variables inside strings
### Use placeholders instead of variables inside strings
!!! failure "Bad"
......@@ -83,7 +88,7 @@ The general rules for avoiding errors may be summed up like this:
!!! success "Good"
```php
$string = ts("The date type '%1' has been saved.", array(1 => $name));
$string = ts("The date type '%1' has been saved.", [1 => $name]);
```
### Avoid tags inside strings
......@@ -99,7 +104,52 @@ The general rules for avoiding errors may be summed up like this:
```smarty
<p>{ts}Hello, world!{/ts}</p>
```
Hyperlinks within larger blocks of text are an exception to this rule, where you should place the `<a>` tags within the `ts`. Any link parameters should be provided as arguments to the ts. For example:
!!! failure "Bad"
```smarty
{ts}Here is a block of text with a link to the <a href="https://www.civicrm.org" target="_blank">CiviCRM Web Site</a>.{/ts}
```
!!! success "OK"
```smarty
{ts 1='href="https://www.civicrm.org" target="_blank"'}Here is a block of text with a link to the <a %1>CiviCRM Web Site</a>.{/ts}
```
Smarty doesn't evaluate within single quotes, so if you are capturing an URL for a link, capture it with the `href="` and optionally `target="_blank"`.
!!! success "OK"
```smarty
{capture assign=something}href="{crmURL p='civicrm/admin/something' q='reset=1'}" target="_blank"{/capture}
{ts 1=$something}Here is a block of text with a link to a <a %1>specific URL in CiviCRM</a>.{/ts}
```
For `title` attributes in `<a>` links, within CiviCRM these usually only appear in links that aren't within a larger block of text or where there is no clickable text, such as a datepicker icon. In this situation, the title text needs to be translated:
!!! failure "Bad"
```smarty
{ts}<a href="https://www.example.org/civicrm/something?reset=1" title="List participants for this event (all statuses)">Participants</a>{/ts}
```
!!! success "Good"
```smarty
<a href="https://www.example.org/civicrm/something?reset=1" title="{ts}List participants for this event (all statuses){/ts}">{ts}Participants{/ts}</a>
```
If there is no clickable text, just translate the title attribute:
!!! success "Good"
```smarty
<a title="{ts}Select Date{/ts}"><i class="crm-i fa-calendar"></i></a>
```
### Avoid multi-line strings
Even if your code editor may not like it, long strings should be on a single line since a change in indentation might change where the line breaks are, which would then require re-translating the string.
......@@ -130,21 +180,21 @@ Even if your code editor may not like it, long strings should be on a single lin
!!! success "Good"
```php
$string = ts('%1 has been created.', array(1 => $labelFormat['label'])),
$string = ts('%1 has been created.', [1 => $labelFormat['label']]),
```
### Avoid escaped quotes
### Avoid escaped quotes whenever possible
!!! failure "Bad"
```php
$string = ts('A new \'%1\' has been created.', array(1 => $contactType));
$string = ts('A new \'%1\' has been created.', [1 => $contactType]);
```
!!! success "Good"
```php
$string = ts("A new '%1' has been created.", array(1 => $contactType));
$string = ts("A new '%1' has been created.", [1 => $contactType]);
```
### Use separate strings for plural items
......@@ -152,15 +202,17 @@ Even if your code editor may not like it, long strings should be on a single lin
!!! failure "Bad"
```php
$string = ts('%1 item(s) created', array(1 => $count));
$string = ts('%1 item(s) were created by %2', [1 => $count, 2 => $userName]);
```
!!! success "Good"
```php
$string = ts('%count item created',
array('count' => $total, 'plural' => '%count items created')
);
$string = ts('%count item was created by %1', [
'count' => $total,
'plural' => '%count items were created by %1',
1 => $userName,
]);
```
### Ensure that strings have *some* words in them
......@@ -169,21 +221,21 @@ Another common error is to use `ts()` to aggregate strings or as a "clever" way
!!! failure "Bad"
Incorrect aggregation. This will be extremely confusing to translations and might give some really bad results in some languages.
Incorrect aggregation. This will be extremely confusing to translators and might give some really bad results in some languages.
```php
$operation = empty($params['id']) ? ts('New') : ts('Edit'));
$string = ts("%1 %2", array(1 => $operation, 2 => $contactType));
$string = ts("%1 %2", [1 => $operation, 2 => $contactType]);
```
!!! success "Less bad"
!!! success "OK"
```php
if (empty($params['id'])) {
$string = ts("New %1", array(1 => $contactType));
$string = ts("New %1", [1 => $contactType]);
}
else {
$string = ts("Edit %1", array(1 => $contactType));
$string = ts("Edit %1", [1 => $contactType]);
}
```
......