Update 1 June 2017: Read my second post on acceptance testing for an easier way to set up the test environment – Acceptance Testing WordPress plugins
As more and more organisations depend on Gravity Flow for business workflow automation the responsibility of ensuring reliability between releases is becoming increasingly important. One of the ways of ensuring that reliability is through automated acceptance testing.
The goal of acceptance tests, however, is to ensure the application works as expected in the browser. Here’s a video of the Gravity Flow acceptance tests. You don’t need to sit through all the tests to get an idea of how they work.
There are a number of acceptance testing frameworks and I’m sure most of them would do the job pretty well. After a relatively unscientific selection process I decided to use Codeception. There seemed to be a good deal of support for WordPress and I liked the idea of writing the tests in PHP.
Writing tests in Codeception is child’s-play. Literally – a child could do it. The test scripts remind me of the block programs my five year old son builds to make his little robot to go from the living room to the kitchen without bumping into anything.
Here’s what a test looks like:
* Test summary: submit vacation request then approve or reject it.
* Test details:
* - Fill in the form fields: First and Last name,
* Department (drop down), Extra text field,
* Start and End dates, Comments
* - Login to admin, go to Inbox
* - Click on the 'Vacation Request' Workflow
* - Click on Approve button
$I = new AcceptanceTester( $scenario );
$I->wantTo( 'Test a simple approval process' );
// Submit the form
$I->amOnPage( '/vacation-request' );
$I->see( 'Vacation Request' );
$I->selectOption( 'Department', 'Marketing' );
$I->fillField( 'Date from', '08/17/2016' );
$I->fillField( 'Date to', '08/18/2016' );
$I->fillField( 'Comments', 'Some text' );
$I->click( 'Submit' );
// Login to wp-admin
// Go to Inbox
$I->amOnWorkflowPage( 'Inbox' );
// Click on the vacation request
$I->click( 'Vacation Request' );
$I->click( 'Approve' );
The file name is important. It has to end in Cept.php so for example this test might have a file name of ApprovalCept.php. More structured tests can be written using a class but I’ve not found the need for that so far.
The tests are easy to write but setting up the test environment to actually run the tests is not quite so simple. Conceptually it’s reasonably straightforward, but it can be a bit frustrating piecing everything together for the first time. This is an account of how I set it up. It all went pretty smoothly for me but it appears that these steps don’t work out of the box for absolutely everyone – if you run into a problem and manage to solve it please add it to the comments below.
I wanted to see the tests actually running in a real browser rather than a headless browser like PhantomJS, so I chose to use Selenium with Chromedriver. Selenium listens for the commands from the while Chromedriver tells Chrome what to do. Both Selenium and Chromedriver need to be running at the same time when Codeception runs the tests.
You may well have a lot of this already set up and working on your system. If you do, you’ll find the process very simple. Here are the prerequisites:
I’ve found that Codeception wouldn’t run properly on versions of PHP lower than 5.6.
If you’re not running Composer, now’s a great time to get it installed. Composer takes care of installing Codeception and its dependencies.
Download and install the Java SDK.
Selenium & Chromedriver
Download the latest stable version of Selenium and put it in a folder in your home directory. Download the latest version of Chromedriver and put it in the same folder.
A dedicated WordPress site for Acceptance Testing
This is not strictly necessary but I found it useful. The tests will reset the database at the beginning of every run so don’t use a site you want to keep.
Gravity Flow Source Code (Development Version)
Clone development version of Gravity Flow from GitHub in your plugins folder and follow the installation instructions in the readme. You’ll also need Gravity Forms installed.
This is where composer comes in. Run the following command in a terminal window from the Gravity Flow plugin folder.
This will install Codeception and the dependencies.
Configure the Tests
Copy either codeception-sample-vvv.yml or codeception-sample-pressmatic.yml and rename it to codeception.yml. Adjust the URLs and database details as appropriate.
Run the tests
Once you’ve got everything installed you just need to start Selenium + Chromedriver and the run the tests.
Here’s the command I use – adjust according to the versions you have:
java -Dwebdriver.chrome.driver=./chromedriver -jar selenium-server-standalone-2.53.1.jar
Then run the tests like this:
Running all the tests every time can be annoying when you’re working on just one of them. Run individual tests by passing the file path of the test. e.g.
./vendor/bin/codecept run tests/acceptance-tests/acceptance/TotalDependingOnQuantityCept.php
If you’d like to add your test to the bank of Gravity Flow acceptance tests just clone the repository, export your form and add it to the _data folder, add your test file and submit a pull request. The bootstrap script will import your form automatically and add a page for it with a slug in the following format:
sanitize_title_with_dashes( $form[‘title‘] )
A couple of recommendations:
- Test the workflow not the settings. The workflow settings UI may be subject to change but the workflow from the point of view of the assignees is unlikely to change.
- Don’t include any sensitive data in your form as it will be made public under the GPL.
Acceptance tests provide a way for developers to ensure that applications work as expected and Codeception makes this trivial. By contributing tests to the Gravity Flow repository developers can be confident that their workflows will continue to work as expected after upgrading.
If you have any tips for other readers getting started, please leave them in the comments below.
Leho Kraav @lkraav saysOctober 17, 2016 at 10:42 pm
Kick ass (y)