xbps-src/libexec: add make_check option.

Allows a template to define in which circumstances its do_check phase
should run, without requiring custom do_check definitions in each
template (also makes it easier to change how build styles do things
without checking as many templates).

Add to Manual and CONTRIBUTING as well.

CONTRIBUTING.md

@@ -87,11 +87,15 @@ When you make changes to your pull request, please *do not close and reopen your
 
 #### Continuous Integration
 
-Pull requests are automatically submitted for Continuous Integration (CI) testing to ensure packages build on various combinations of libc and architecture.
+Pull requests are automatically submitted for Continuous Integration (CI) testing to ensure packages build and pass their tests (on native builds) on various combinations of C library and architecture.
 Packages that take longer than 120 minutes or need more than 14G of storage to complete their build (for example, Firefox or the Linux kernel) will fail CI and should include `[ci skip]` in the PR title or body (the comment field when the PR is being opened) to avoid wasting CI builder time.
 Use your best judgment on build times based on your local building experience. If you skip CI when submitting a PR, please build and cross-build for a variety of architectures locally, with both glibc and musl, and note your local results in PR comments.
 Make sure to cover 64-bit and 32-bit architectures.
 
+If you notice a failure in CI that didn't happen locally, that is likely because you didn't run tests locally.
+Use `./xbps-src -Q pkg <package>` to do so.
+Some tests won't work in the CI environment or at all, and their templates should encode this information using the `make_check` variable.
+
 Continuous Integration will also check if the templates you have changed
 comply with the our guidelines. At the moment not all packages comply with the rules, so if you update a package, it may report errors about places you haven't touched. Please feel free to fix those errors too.
 

Manual.md

@@ -166,7 +166,12 @@ can be used to perform other operations before configuring the package.
 
 - `build` This phase compiles/prepares the `source files` via `make` or any other compatible method.
 
-- `check` This optional phase checks the result of the `build` phase for example by running `make -k check`.
+- `check` This optional phase checks the result of the `build` phase by running the testsuite provided by the package.
+If the default `do_check` function provided by the build style doesn't do anything, the template should set
+`make_check_target` and/or `make_check_args` appropriately or define its own `do_check` function. If tests take too long
+or can't run in all environments, they should be run only if `XBPS_CHECK_PKGS` is `full`, which means they should either
+be under a `[ "$XBPS_CHECK_PKGS" = full ]` conditional (especially useful with custom `do_check`) or `make_check=extended`
+should be set in the template.
 
 - `install` This phase installs the `package files` into the package destdir `<masterdir>/destdir/<pkgname>-<version>`,
 via `make install` or any other compatible method.
@@ -589,6 +594,11 @@ patches to the package sources during `do_patch()`. Patches are stored in
 - `disable_parallel_build` If set the package won't be built in parallel
 and `XBPS_MAKEJOBS` has no effect.
 
+- `make_check` Sets the cases in which the `check` phase is run. Can be `yes` (the default) to run if
+`XBPS_CHECK_PKGS` is set, `extended` to run if `XBPS_CHECK_PKGS` is `full` and `no` to never run.
+This option should usually be accompanied by a comment explaining why it was set, especially when
+set to `no`.
+
 - `keep_libtool_archives` If enabled the `GNU Libtool` archives won't be removed. By default those
 files are always removed automatically.
 

common/xbps-src/libexec/xbps-src-docheck.sh

@@ -32,6 +32,17 @@ if [ -z "$XBPS_CHECK_PKGS" ]; then
     exit 0
 fi
 
+if [ "$make_check" = no ]; then
+    msg_normal "${pkgname}-${version}_${revision}: skipping check (make_check=no) ...\n"
+    exit 0
+fi
+
+if [ "$make_check" = extended -a "$XBPS_CHECK_PKGS" != full ]; then
+    msg_normal \
+        "${pkgname}-${version}_${revision}: skipping check (make_check=extended and XBPS_CHECK_PKGS is not 'full') ...\n"
+    exit 0
+fi
+
 for f in $XBPS_COMMONDIR/environment/check/*.sh; do
     source_file "$f"
 done