phpunit.md - Multiple refinements

docs/testing/phpunit.md

@@ -8,8 +8,8 @@ ensuring that the `Contact.create` API actually creates a contact.
 
 ## Suites
 
-PHPUnit tests are written as *PHP classes* (such as `api_v3_ContactTest`) and grouped together into *suites* (such as `api_v3`).  Each suite
-may follow different coding conventions.  For example, all tests in the `api_v3` suite extend the base class `CiviUnitTestCase` and execute on the
+PHPUnit tests are written as *PHP classes* (such as `CRM_Core_RegionTest`) and grouped together into *suites* (such as `CRM`).  Each suite
+may follow different coding conventions.  For example, all tests in the `CRM` suite extend the base class `CiviUnitTestCase` and execute on the
 headless database.
 
 The `civicrm-core` project includes these suites:
@@ -17,38 +17,58 @@ The `civicrm-core` project includes these suites:
 
 | Suite   | Type | CMS | Typical Base Class | Comment |
 | ------- | ---- | --- | ------------------ | ----------- |
-|`api_v3` |`headless`|Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
-|`Civi`   |`headless`|Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
-|`CRM`    |`headless`|Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
-|`E2E`    |`e2e`|Agnostic|`CiviEndToEndTestCase`|Useful for command-line scripts and web-services|
-|`WebTest`|`e2e`|Drupal|`CiviSeleniumTestCase`|Useful for tests which require a full web-browser|
+|`api_v3` | [Headless](/testing/index.md#headless) |Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
+|`Civi`   | [Headless](/testing/index.md#headless) |Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
+|`CRM`    | [Headless](/testing/index.md#headless) |Agnostic|`CiviUnitTestCase`|Requires `CIVICRM_UF=UnitTests`|
+|`E2E`    | [E2E](/testing/index.md#e2e) |Agnostic|`CiviEndToEndTestCase`|Useful for command-line scripts and web-services|
+|`WebTest`| [E2E](/testing/index.md#e2e) |Drupal-only|`CiviSeleniumTestCase`|Useful for tests which require a full web-browser|
 
 Each extension may have its own suite(s).
 
 ## Running tests
 
-To execute all tests in a suite, run the corresponding `AllTests.php` file. For example, this command runs the full `CRM` suite:
+PHPUnit provides a command-line tool.  In [buildkit](/tools/buildkit.md), this tool is named `phpunit4`.  (In other environments, it might be
+`phpunit` or `phpunit.phar`.)
+
+Generally, to run any PHPUnit test, you should `cd` into the relevant project; then note the relative file-path of the test file; and run it with
+`phpunit4`.  A typical command might look like this:
 
 ```bash
-$ cd /path/to/civicrm
-$ env CIVICRM_UF=UnitTests phpunit4 ./tests/phpunit/CRM/AllTests.php
+$ cd /path/to/my/project
+$ phpunit4 ./tests/MyTest.php
 ```
 
-To execute a single test class, specify that file, as in:
+For the main `civicrm-core` project, you'll navigate to the main `civicrm` folder. Test files are stored under `./tests/phpunit`. For example,
+in Drupal 7, the canonical `civicrm` folder is `sites/all/modules/civicrm`, and a typical test is `CRM_Core_RegionTest`. You might run:
 
 ```bash
-$ cd /path/to/civicrm
-$ env CIVICRM_UF=UnitTests phpunit4 ./tests/phpunit/CRM/Core/RegionTest.php
+$ cd ~/buildkit/build/dmaster/sites/all/modules/civicrm
+$ phpunit4 ./tests/phpunit/CRM/Core/RegionTest.php
+```
+
+*However*, this command would only work on a [minimal unit test](/testing/index.md#unit).  `CRM_Core_RegionTest` is actually
+[headless](/testing/index.md#headless) (as are all tests in `CRM`).  Consequently, you may see an error message like this:
+
+```
+PHPUnit 4.8.21 by Sebastian Bergmann and contributors.
+
+EEEEEEEEE
+
+Time: 450 ms, Memory: 17.75Mb
+
+There were 9 errors:
+
+1) CRM_Core_RegionTest
+exception 'RuntimeException' with message '_populateDB requires CIVICRM_UF=UnitTests'...
 ```
 
-Generally, note how each example breaks down into a few pieces of information. These
-are the elements you would need to run PHPUnit tests in `civicrm-core`, but the formula
-would be the same for an extension:
+Headless tests are designed to run with a fake CMS, and you must activate the fake CMS by setting the environment variable `CIVICRM_UF`. This revised
+command should correct the issue:
 
- * Path to codebase. (Ex: `/path/to/civicrm`)
- * If required, any environment variables. (For `headless` tests, use `CIVICRM_UF=UnitTests`)
- * Name of the PHPUnit command. (Ex: `phpunit4`, which is bundled with [buildkit](/tools/buildkit.md))
- * Name of the test file. (Ex: `tests/phpunit/CRM/Core/RegionTest.php` or `tests/phpunit/CRM/AllTests.php`)
+```bash
+$ cd ~/buildkit/build/dmaster/sites/all/modules/civicrm
+$ env CIVICRM_UF=UnitTests phpunit4 ./tests/phpunit/CRM/Core/RegionTest.php
+```
 
 !!! tip "PhpStorm"
 
@@ -60,6 +80,15 @@ would be the same for an extension:
 
     [civi-test-run](/tools/civi-test-run.md) is a grand unified wrapper which runs *all* CiviCRM test suites. It's particularly useful for *continuous-integration*.
 
+!!! tip "Select tests using `AllTests.php`"
+
+    In `civicrm-core`, there are several suites (`CRM`, `api_v3_`, etc). Each suite has a file named `AllTests.php` which can be used as follows:
+
+    ```bash
+    $ cd /path/to/civicrm
+    $ env CIVICRM_UF=UnitTests phpunit4 ./tests/phpunit/CRM/AllTests.php
+    ```
+
 !!! tip "Select tests using `--filter`, `--group`, etc"
 
     The PHPUnit CLI supports a number of [filtering options](https://phpunit.de/manual/current/en/textui.html). For example,