Skip to content
Snippets Groups Projects
Unverified Commit e142c865 authored by Sean Madsen's avatar Sean Madsen Committed by GitHub
Browse files

Merge pull request #472 from seamuslee001/testing_docs

Update Jenkins Documention to resolve issue #458
parents 4a2dd6db e72d6511
No related branches found
No related tags found
No related merge requests found
## Jenkins
# Jenkins
To prevent defects from entering the system or remaining in the system,
the tests are executed automatically by Jenkins - our continuous integration server.
Jenkins is CiviCRM's [continuous integration](https://en.wikipedia.org/wiki/Continuous_integration) server.
The test system runs twice. Firstly, when a change is proposed through Github
(as a "pull-request" or "PR"), Jenkins tests it immediately. (This takes 5-90min.)
To prevent defects from entering the system or remaining in the system, the tests are executed automatically by Jenkins. Jenkins first runs php and javascript style checks through the usage of `civilint` before running unit tests.
Unfortunately, the full test suite would take several hours longer, so Jenkins
only runs some tests at this stage.
Jenkins runs two different types of jobs a PR test job and a matrix Job. The results of the tests are published at [test.civicrm.org](https://test.civicrm.org) There is some additional documenation in the [testing-readme](https://github.com/civicrm/civicrm-core/blob/master/tests/README.md)
The results of the tests are published at [test.civicrm.org][jenkins-test-results]
## Jenkins Whitelist
If the tests have failed for something that we suspect is a random failure, we
can ask Jenkins to run the tests again by commenting in the PR "Jenkins, test
this please"
For new (unrecognised by Jenkins) contributors, Jenkins will automatically respond "can an admin verify this patch?", and a Github user with admin permissions may approve running the tests by commenting on the PR "ok to test".
Jenkins only builds a Drupal site that is built against the branch that your
PR is modifying. If you need to test a patch against another CMS than you
will want to test the patch in your own environment.
If the user is trusted, CiviCRM administrators can add the person to the whitelist by commenting "add to whitelist".
## Jenkins Whitelist
## PR Test Jobs
For new (unrecognised by Jenkins) contributors, Jenkins will automatically
respond "can an admin verify this patch?", and a Github user with admin
permissions may approve running the tests by commenting on the PR "ok to test".
Jenkins runs a "PR Test" job which is triggered whenever a pull request is created or updated in GitHub. PR test jobs can take anywhere from 5 - 90 min to complete. This works for the following repos:
If the user is trusted, CiviCRM administrators can add the person to the
whitelist by commenting "add to whitelist".
* `civicrm-core`
* `civicrm-packages`
* `civicrm-drupal`
* `civicrm-backdrop`
## Build Schedule
Jenkins runs a mix of [PHPUnit](/testing/phpunit.md#suites) [PHPWebTests](/testing/selenium.md) and [Javascript UnitTests](/testing/karma.md). The details of the types of suites and purpose can be found on the index of this [chapter](/testing/index.md)
Jenkins periodically runs the full suite (once every 4-24hr) on the official
codebase.
Code-style is not checked on this build, but all upgrade and web tests are run.
To run all the tests in one of the suites locally, you can use [civi-test-run](/tools/civi-test-run.md).
The tests that are run are variable depending on the repository that is triggered.
* `civicrm-core`: Upgrade, karma, CRM, api_v3, E2E and Civi tests
* `civicrm-packages`: same as civicrm-core
* `civicrm-drupal`: Upgrade, Karma and E2E tests
* `civicrm-backdrop`: same as civicrm-drupal
For PR test jobs against `civicrm-core` and `civicrm-packages`, Jenkins only builds a Drupal site to run the tests against. If you are fixing an issue with another CMS you may need to build yourself a local test environment with that CMS
If the tests have failed for something that we suspect is a random failure, we can ask Jenkins to run the tests again by commenting in the PR "Jenkins, test this please"
## Matrix Test Jobs
The other type of Job that jenkins runs is what is described as a "matrix" job. This is a much more extended version of the PR job and is usually run against multiple different webserver configurations.
## Running the test suite locally
The main difference between the `CiviCRM-Core-Matrix` job and a PR test job is that it runs more upgrade tests than the `CiviCRM-Core-PR`.
The other matrix job is one that runs the webtests.
The matrix jobs operate on two main combinations, a PHP5.5 + MySQL5.5 server and a PHP7.0 + MySQL5.7 test seerver.
Due to the size of the test matrix, jobs can take from 2 to 24 hours to complete depending on the job.
### Build Schedule
Jenkins periodically runs the full suite (once every 4-24hr) on the official codebase.
Code-style is not checked on this build, but all upgrade and web tests are run.
You can use [civi-test-run](/tools/civi-test-run.md) locally to run a full standard CiviCRM Test suite.
[jenkins-test-results]: https://test.civicrm.org/
[testing-readme]: https://github.com/civicrm/civicrm-core/blob/master/tests/README.md
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment