Merge pull request #496 from totten/master-console-tips

tools/jenkins.md - Add tips for reviewing console output

docs/tools/jenkins.md

@@ -10,6 +10,12 @@ Pull-requests are tested automatically with build-bot software called [Jenkins](
 * If the automated test fails, click on the red dot to investigate details. Check for information in:
     * The initial summary. Ordinarily, this will list test failures and error messages.
     * The console output. If the test-suite encountered a significant error (such as a PHP crash), the key details will only appear in the console.
+        * __Tip__: Sometimes, the console output is pretty long (several hundred KB). This usually means that a majority of tests ran, but there's a failure mixed in somewhere. View the full log and search for the word `EXITCODE`; this should normally appear as blank or 0. The first non-empty `EXITCODE` should be close to the problem.
+        * __Tip__: Sometimes, the console output is relatively short (a page or two). You might look for evidence of one of these typical problems:
+            * A network service (such as `github.com` or `packagist.org`) was unavailable. These are usually corrected quickly without any action. Try running the test again.
+            * The pull-request is based on an old version of the codebase, and it cannot be applied cleanly with `git apply` or `git scan am`. Rebasing the PR branch should resolve this.
+            * On the test node, a local resource (such as disk-space, RAM, or inode count) was exhausted. If the failure was recent (past few hours), seek help on [the infrastructure channel](https://chat.civicrm.org/civicrm/channels/infrastructure). If the failure occurred a few hours or days ago, try running the test again.
+            * Some part of the build/test toolchain needs attention. For example, a test script may have been changed without supporting an edge-case; or a tool like `bower` or `npm` may need to be upgraded. Seek help on `infrastructure`.
 * Code-style tests are executed first. If the code-style in this patch is inconsistent, the remaining tests will be skipped.
 * The primary tests may take 20-120 min to execute. This includes the following suites: `api_v3_AllTests`, `CRM_AllTests`, `Civi\AllTests`, `civicrm-upgrade-test`, and `karma`
 * There are a handful of unit tests which are time-sensitive and which fail sporadically. See: https://forum.civicrm.org/index.php?topic=36964.0