TruUcde Rabbithole 09/12 : Setting up for integration testing

In this post we’ll set up and configure what is needed for our WordPress integration tests. As mentioned previously in this article series, integration testing is all about testing code we’ve written integrated with other code from its operating environment. So in this case we will test our code as it’s running in WordPress.

Sounds a lot like checking the tire pressure on a moving vehicle doesn’t it? We will use what is referred to as the WordPress Unit Test Scaffold to accomplish this. What this system does is spin up a clean install of WordPress with a clean database for each test. The advantage of this is that our WordPress environment and database are not contaminated with other extra code or the detretius of previous tests.

So, as you can imagine if you are following the series, there are things to install and configure and then a sanity test to conduct. Let’s get started by looking at some documentation. Note: a frequently mentioned topic on the interwebs is that WordPress refers to this type of testing and its frameworks as “unit testing”, but we know it is really “integration testing.” And no, I don’t know why they do that, actually I do. The framework was initially developed to test WordPress core code itself, it just happens to be useful for testing themes, plugins and other code that extends WordPress.

Okay, so here’s the relevant documentation breakdown. Some of what we need to install is installed via a tool called WP-CLI, short for WordPress command line. If you’ve never heard of it poke around the documentation a little. You can basically do a ton of admin stuff from the command line that you normally would have have to do inside of WordPress’s interface (after logging in and getting to the right page).

https://make.wordpress.org/cli/

We will use this tool and one of its commands to accomplish the first part of our installation/setup. Here is the documentation on that command:

wp scaffold plugin-tests

And some further documentation on getting setup (Running Tests Locally section):

Plugin Unit Tests

Prerequisites

We need to have the following in place before we begin:

• Local Server Environment: apache, php and mysql. (Discussed in the third article, Development Setup, of this series)
• Git (Discussed in the fourth article, Installing the Tools, of this series)
• SVN (installation instructions)
• PHPUnit (Also discussed in the fourth article, Installing the Tools, of this series.)

Installing WP-CLI

The wp-cli file is downloaded as a phar file which is a format for bundling php files together so that they can be run as a single command. Composer, which we saw in previous articles, can also be installed this way. What we will do is download the file, test it and then move it to somewhere on our system where it is accessible via a path statement. Basically this will be a global install. Here are the commands and output.

curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5437k  100 5437k    0     0   537k      0  0:00:10  0:00:10 --:--:--  596k

php wp-cli.phar --info
OS:	Darwin 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64
Shell:	/usr/local/bin/zsh
PHP binary:	/usr/local/Cellar/php/7.4.7/bin/php
PHP version:	7.4.7
php.ini used:	/usr/local/etc/php/7.4/php.ini
WP-CLI root dir:	phar://wp-cli.phar/vendor/wp-cli/wp-cli
WP-CLI vendor dir:	phar://wp-cli.phar/vendor
WP_CLI phar path:	/Users/twelch/www/sites/multibox/wwwroot/wp-content/plugins/truucde-blog
WP-CLI packages dir:	/Users/twelch/.wp-cli/packages/
WP-CLI global config:
WP-CLI project config:
WP-CLI version:	2.4.0

chmod +x wp-cli.phar

sudo mv wp-cli.phar /usr/local/bin/wp
Password:

wp --info
OS:	Darwin 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64
Shell:	/usr/local/bin/zsh
PHP binary:	/usr/local/Cellar/php/7.4.7/bin/php
PHP version:	7.4.7
php.ini used:	/usr/local/etc/php/7.4/php.ini
WP-CLI root dir:	phar://wp-cli.phar/vendor/wp-cli/wp-cli
WP-CLI vendor dir:	phar://wp-cli.phar/vendor
WP_CLI phar path:	/Users/twelch/www/sites/multibox/wwwroot/wp-content/plugins/truucde-blog
WP-CLI packages dir:	/Users/twelch/.wp-cli/packages/
WP-CLI global config:
WP-CLI project config:
WP-CLI version:	2.4.0

Okay, so on line 1 is our curl command and download address that we use to download the phar file. We test if it’s working okay on line 6. Note that we have to put php in front of the rest of the command to run it as a php file. The output of this gives us some information and file paths.

Next we use the system command chmod +x to make this file executable (line 20). This means that we don’t have to put php in front of it any more. Then we use the system command mvto both rename (to the simpler WP) and move the file to a place among other commands and one that is on the path (line 22). Note that we preface this command with sudo which means superuser do. This is a linuxy way of protecting system files and you have to use it to do things in directories owned by the root user.

Install the plugin unit test scaffold

Doesn’t that sound cool. Sometimes when I want to seem impressively nerdy I’ll use this as an answer to “what are you up to today?” Actually, this is also fairly straightforward. We need to make sure we are in the plugin project root directory (refer to the previous articles in this series to refresh your memory on project/directory set up) and then run the following command:

wp scaffold plugin-tests truucde-blog
Success: Created test files.

Here we see the command and its response. Note: we use the folder name that our project files are in as part of the command. This command added 6 files to our project, I’ll just make some brief notes about them first and then we’ll tuck into what we need to do with them.

phpunit.xml.dist is a phpunit configuration file. The default scaffold installation is not hip to the way we’ve organized our folder with real unit tests in one directory and integration tests in the other, each supported by their own php xml config file and bootstrap file. Probably the easiest thing for us to do is compare this scaffold created file with our phpunit_unit.xml file from article 5 of this series and determine what needs to be in our phpunit_integration.xml file. We’ll do that below.

.travis.yml is a configuration file for Travis CI, a continuous integration system which we are not using. We have two courses of action: delete it or ignore it. I generally ignore it in case I decide to use Travis CI someday. You can find more information about Travis here: https://travis-ci.org/.

bin/install-wp-tests.sh This is a script that does a bunch of stuff, it installs a clean copy of WordPress and configures WordPress to use a local database (always use a separate database for testing as the database is wiped before each test). We’ll run this after we have our project configuration files sorted out.

tests/bootstrap.php This is, of course, the phpunit bootstrap file that will fire everything up before running our phpunit tests. We’ll want to tuck this file into our tests/integrationTests folder and call it bootstrap_integration.php. Also we’ll compare it to our working bootstrap_unit.php file from article 5 this series in case anything in there applies.

tests/test-sample.php The file name spills the beans here, this is a sample test file that we’ll temporarily move to our tests/integrationTests/tests folder. We can likely use it for our sanity test.

.phpcs.xml.dist Contains some additional PHP_CodeSniffer rules. These are probably helpful and need to be in our project root, so we’ll leave them there.

PHP Unit config xml and bootstrap files

Okay, I’m going to start by comparing our phpunit_unit.xml file with the one installed by the WP scaffold command. The two files are very similar except that the WP Scaffold one doesn’t have the <filter> and <logging> sections to support code coverage reports. It may not really be necessary, but I’m going to add them anyway.

Also, the directory paths and bootstrap filename need to be changed, so I’ll take care of that too. (Don’t tell anyone, but it’s going to be easier for me to copy from the phpunit_unit.xml and make the changes that way.) Also I’m going to make sure the file is called phpunit_integration.xml so that we clearly know which config file is which. As a final little touch I’m going to add the phpunit command that we need to run in a comment on line 2 of the file for convenience and as a reminder.

Finally, I remember a useful item mentioned in one of Codetabs posts in their plugin development series. https://www.codetab.org/tutorial/wordpress-plugin-development/unit-test/multisite-plugin-unit-tests/ They provide compelling logic for the inclusion of a multisite constant which you can see in the file listing below on lines 13-15.

Here’s what all that looks like:

<?xml version="1.0" encoding="UTF-8"?>
<!-- vendor/bin/phpunit -c phpunit_integration.xml -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.1/phpunit.xsd"
         bootstrap="./tests/integrationTests/bootstrap_integration.php"
         backupGlobals="false"
         colors="true"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
>

    <php>
        <const name="WP_TESTS_MULTISITE" value="1" />
    </php>

    <testsuites>
        <testsuite name="truucde-int">
            <directory>./tests/integrationTests/tests</directory>
        </testsuite>
    </testsuites>

    <filter>
        <whitelist addUncoveredFilesFromWhitelist="true">
            <file>./truucde.php</file>
        </whitelist>
    </filter>

    <logging>
        <log type="coverage-html" target="./tests/integrationTests/coverage" showUncoveredFiles="true"/>
    </logging>
</phpunit>

Okay, now let’s have a look at getting the bootstrap file in place. First, I’m going to move the WP scaffold bootstrap file into the integrations directory and while I’m at it tuck the sample-test.php file into the integration tests directory. I’m going to do the same thing as for the xml config file above, compare the one installed by WP Scaffold with our previous unit test bootstrap file.

Hrmm, completely different. Most of the paths and what not apply to the WP Scaffold test WordPress instance. So mostly I’m going to leave this alone, the only thing I see is that, because our plugin and directory are different (because I’m building up a second version for this blog post series) I’ll correct the name of the plugin.

As a side note, I created empty PHPunit xml config and bootstrap files for the integration when I was configuring for unit test a few blog posts back. So what I’m going to do is leave the WP Scaffold ones alone for later reference should I need it and put the working code in the blank versions I pre-created.

Oh, I did find something in here I want to point out. Take a look at this line:

require dirname( dirname( __FILE__ ) ) . '/truucde.php';

This is from a function (full file code coming in a minute) that loads our plugin into the WordPress test instance prior to running the tests. Now, WordPress has a lot of older php code in it, so I like to be careful in my application of new php techniques. What the dirname() function call does here is determine the parent path information of it’s parameter. In other words __FILE__ is a shortcut for the current file, bootstrap_integration.php. So, the dirname() command using as its argument __FILE__ returns integrationTests. Now notice, that there are two nested dirname() function calls. The outer one will return the parent directory of the inner one. So tests, because that is the parent directory of integrationTests.

But that is not where our truucde.php file is and it is this file the full line is trying to get to. That’s because the bootstrap file was created to be in the tests directory, so that sequence of dirname() calls would have worked. But I moved it down the hierarchy a further level into integrationTests. So I now need three nested dirname() function calls to adjust for moving the file. Luckily my IDE told me this so that I didn’t have to wait until my tests didn’t work.

A little side ramble, the first time I encountered this I went to the documentation on the dirname() command, because it seemed really stupid to nest 3 function calls. Also I wanted to get my head around what this function was doing. In PHP Version 7.0, which WordPress just recently specified as the minimum php version for WordPress, the dirname() function had a levels parameter added to it. So, this nesting can now be eliminated by some like: dirname( __FILE__, 3). But, just because WordPress core has specced this does not mean other things have updated yet. In particular, the WordPress Scaffold still requires an older version of PHPUnit because Scaffold is not yet hip to new PHP versions. For this reason, I’m going to go with the 3 nested dirnames (two turtle doves and a…yep, this whole side ramble was set up for that little bit of lameness.)

Anyway, here is the bootstrap_integration.php file.

<?php
/**
 * PHPUnit bootstrap file
 *
 * @package Truucde_Blog
 */

$_tests_dir = getenv( 'WP_TESTS_DIR' );

if ( ! $_tests_dir ) {
	$_tests_dir = rtrim( sys_get_temp_dir(), '/\\' ) . '/wordpress-tests-lib';
}

if ( ! file_exists( $_tests_dir . '/includes/functions.php' ) ) {
	echo "Could not find $_tests_dir/includes/functions.php, have you run bin/install-wp-tests.sh ?" . PHP_EOL; // phpcs:ignore WordPress.Security.EscapeOutput.OutputNotEscaped
	exit( 1 );
}

// Give access to tests_add_filter() function.
require_once $_tests_dir . '/includes/functions.php';

/**
 * Manually load the plugin being tested.
 */
function _manually_load_plugin() {
	require dirname( dirname( dirname( __FILE__ ) ) ) . '/truucde.php';
}
tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );

// Start up the WP testing environment.
require $_tests_dir . '/includes/bootstrap.php';

install-wp-tests.sh

The install-wp-tests.sh script that we need to run requires 5 values to run, I’ve included my values:

1. Database name: wptest-blog
2. Database user: root
3. Database user password: ‘*****’ (I didn’t actually use asterisks, you can pick your own password)
4. Database server host: localhost
5. WordPress version for testing (can be number or latest): latest

A quick note on #5. You can use this to install previous versions of WordPress if backwards compatibility is important to your project.

So the full command looks like this (run from the project root):

bash bin/install-wp-tests.sh wptest-blog root '*****' localhost latest

The output of this command looks like so (frightening isn’t it):

bash bin/install-wp-tests.sh wptest-blog root 'HJAc77DLQQmyACnm' localhost latest
+ install_wp
+ '[' -d /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress/ ']'
+ mkdir -p /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress/
+ [[ latest == \n\i\g\h\t\l\y ]]
+ [[ latest == \t\r\u\n\k ]]
+ '[' latest == latest ']'
+ local ARCHIVE_NAME=latest
+ download https://wordpress.org/latest.tar.gz /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress.tar.gz
++ which curl
+ '[' /usr/bin/curl ']'
+ curl -s https://wordpress.org/latest.tar.gz
+ tar --strip-components=1 -zxmf /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress.tar.gz -C /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress/
+ download https://raw.github.com/markoheijnen/wp-mysqli/master/db.php /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress//wp-content/db.php
++ which curl
+ '[' /usr/bin/curl ']'
+ curl -s https://raw.github.com/markoheijnen/wp-mysqli/master/db.php
+ install_test_suite
++ uname -s
+ [[ Darwin == \D\a\r\w\i\n ]]
+ local ioption=-i.bak
+ '[' '!' -d /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib ']'
+ mkdir -p /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib
+ svn co --quiet https://develop.svn.wordpress.org/tags/5.4.2/tests/phpunit/includes/ /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/includes
+ svn co --quiet https://develop.svn.wordpress.org/tags/5.4.2/tests/phpunit/data/ /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/data
+ '[' '!' -f wp-tests-config.php ']'
+ download https://develop.svn.wordpress.org/tags/5.4.2/wp-tests-config-sample.php /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
++ which curl
+ '[' /usr/bin/curl ']'
+ curl -s https://develop.svn.wordpress.org/tags/5.4.2/wp-tests-config-sample.php
++ echo /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress/
++ sed 's:/\+$::'
+ WP_CORE_DIR=/var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress/
+ sed -i.bak 's:dirname( __FILE__ ) . '\''/src/'\'':'\''/var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress//'\'':' /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
+ sed -i.bak s/youremptytestdbnamehere/wptest-blog/ /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
+ sed -i.bak s/yourusernamehere/root/ /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
+ sed -i.bak s/yourpasswordhere/HJAc77DLQQmyACnm/ /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
+ sed -i.bak 's|localhost|localhost|' /var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests-lib/wp-tests-config.php
+ install_db
+ '[' false = true ']'
+ PARTS=(${DB_HOST//\:/ })
+ local PARTS
+ local DB_HOSTNAME=localhost
+ local DB_SOCK_OR_PORT=
+ local EXTRA=
+ '[' -z localhost ']'
++ echo
++ grep -e '^[0-9]\{1,\}$'
+ '[' ']'
+ '[' -z ']'
+ '[' -z localhost ']'
+ EXTRA=' --host=localhost --protocol=tcp'
+ mysqladmin create wptest-blog --user=root --password=HJAc77DLQQmyACnm --host=localhost --protocol=tcp

It is worth noting where these files were installed (line 33). They are typically installed in whatever folder the system uses for temporary files. In my case:

/var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress
/var/folders/_n/rft3ncgj1r165v0rx7wxtkj1_q1v8f/T/wordpress-tests

This Unit Test Scaffold installs once for your whole system, so if you started another project tomorrow it would use this same set of files but according to the specifications of the project PHPUnit config xml and bootstrap files in your new project. On occasion if you are trying to reinstall the the wp-test framework it will complain that files already exists and you may need to delete existing ones.

Anyway we are ready to go. I’m going to rename the test-sample.php file to sampleTest.php to match the naming convention of our install. This is very simple sanity-check test file.

<?php
/**
 * Class SampleTest
 *
 * @package Truucde_Blog
 */

/**
 * Sample test case.
 */
class SampleTest extends WP_UnitTestCase {

	/**
	 * A single example test.
	 */
	public function test_sample() {
		// Replace this with some actual testing code.
		$this->assertTrue( true );
	}
}

We see a couple of things here. First, the class declaration on line 11 extends WP_UnitTestCase. Second the actual assertion on line 18 simply tests if true is true (which it is except in politics.)

Let’s see what happens when we run the test. Our command with results is now:

vendor/bin/phpunit -c phpunit_integration.xml
Installing...
Installing network...
Running as multisite...
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

.                                                    1 / 1 (100%)

Time: 2.66 seconds, Memory: 38.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in HTML format ... done

A couple of items are worth noting. Installing on line 2 refers to installing WordPress in the background (which it will do each and every time you run tests. Installing network on line 3 refers to making the WordPress instance a multisite. (Remember our constant definition in the PHPUnit XML config file.) It confirms it is running as multisite. Then it runs our test okay. And finally it generates the code coverage report. Of course if you look at the report we have a coverage of 0% because we have no tests covering our plugin code yet.

We’ll tackle that in the next article.