CEO of comm-press GmbH and total Drupal evanglist.

Softwaretest Drupal

Simple Test with Drupal

Tests are an important issue in modern software development. With Drupal, we have the ability to use Simple Test. It's become standard for Drupal 7, and is used to test the entire Drupal Core, plus a few of the modules. The whole thing is managed by Boombatower, who's Blog incidentally, provides a huge amount of interesting information on the subject of testing.

Simple Test is a mixture of Web Crawler and DOM Parser with some highly useful functions. We use a collection of objects in a completely empty Drupal, configure these as we need them, load the sites, then see whether the data has been transmitted in the way we intended. What we have in Simple Test is a 'deterministic' test. From the same sequence of logical steps, we'll always obtain the same answers. When we don't, we know somethings' gone wrong.

So then, how exactly do I go about creating a test, what can be tested, and what are the limitations?

Conditions

In order to test in Drupal 6, you first have to Core Patch from the appropriate SimpleTest module. A lot of people would like them to patch the Core from the outset to spare us the job. I'm sure that will happen pretty soon. In Drupal 7, everything is already done for you. You simply enable the module and start work.

Drupal 7 is a joy to work with, but there are some caveats. It's easy to use, but incredibly slow at running tests, so you have to use a few tricks, like MyIsam, and not using InnoDB Tables in MySQL, to ensure that tests take seconds to run instead of minutes.

Otherwise we need CURL for PHP, and a PHP from 5.1 upwards because of the DOM Parsers.

Data

".test" data belongs in the /tests directory of your module. "hook_test", which is still described on lullabot.com, is no longer necessary - the files are detected automatically. In these files, we write the Class of our tests. As far as I know, I can create any number of files I like to help structure my tests.

You'll find a range of examples in the: Simple Test Tutorial "My Module" at drupal.org

DrupalWebTestCase

Our test Class is always derived from the Drupal Web Test Case. Take note of the fact that all the required classes are in the folder drupal_web_test_case.php. It contains all the most important aspects you'll need for a real test. The setUp() Line 1068 method is particularly interesting.

In this folder, you'll find all the Assertions and Auxillary functions as they' re called, and from time to time, document updates. For this reason alone, its well worth a visit to have a look at an Object to see if an update has been posted.

<?php
class MymoduleTestCase extends DrupalWebTestCase {
?>

getInfo()

To enable SimpleTest to work on your Class, you'll have to implement the static method, and provide information about your test to the SimpleTest module. The data from the Array should be self-explanatory.

<?php
 
public static function getInfo() {
    return array(
     
'name' => 'mymodule',
     
'description' => 'Ensure that the mymodule content type provided functions properly.',
     
'group' => 'Mymodule Demonstration Tests',
    );
  }
?>

setUp()

The actual entry in the test is the setUp() method. In the basic method, we install our Drupal, then put our tests into a completely empty Drupal in English with a new, random Table-Prefix. The tests are run in a Sandbox, which is installed in all modules, and contains no data or files. We create a derivative of the setUp() method and call up the original function. We then have to specify all the modules to be installed. For our tests, we also have to check our module, and see that all the dependencies are enabled. If Drupal is installed on default settings, we can assume that the Taxonomy module is enabled, but the Color module won't be.

In the setUp() Method we have to enable everything we'll need during the class lifetime. The MyModule-Test for example, is user-generated, and contains certain user-rights that need to be registered.

<?php
 
public function setUp() {
   
parent::setUp('mymodule');  // Enable any modules required for the test
    // Create and log in our user
   
$privileged_user = $this->drupalCreateUser(array('create mymodule', 'edit own mymodule'));
   
$this->drupalLogin($privileged_user);
  }
?>

testMymoduleCreate()

This is an actual test. Our Drupal is installed and waiting for a request. We fill in the SimpleTest-Browser, in which we use the auxiliary functions drupalPost or drupalGet . In this example, with drupalPost, the Array $edit sends an HTTP POST query,  then checks it to see if the reply contains the desired text.

<?php
 
// Create a mymodule node using the node form
 
public function testMymoduleCreate() {
   
$edit = array(
     
'title' => $this->randomName(32),
     
'body' => $this->randomName(64),
    );
   
$this->drupalPost('node/add/mymodule', $edit, t('Save'));
   
$this->assertText(t('mymodule page @title has been created.', array('@title' => $edit['title'])));

   
// For debugging we can output the page so it can be opened with a browser
    // Remove this line when the test has been debugged
   
$this->outputScreenContents('After page creation', 'testMymoduleCreate');
  }
?>

Assertions / Assumptions

The Assertions - or Assumptions as they're also referred to - are methods in the DrupalWebTestCase Class, that serve - as the name suggests - to assert something. When I first got started, I found it difficult to work out what I was supposed to do with them. so I'd always install if() and exit(), and structure my tests accordingly. But you should use Assertions when you're constructing a test. So now, whenever I want to check a Variable, I use, e.g. $this->assertTrue or e.g. $this->assertEqual, and not if(). Our tests must have a clear and simple structure. If they don't, and just one thing goes wrong, they'll all topple over together like a house of cards, but arguably you'll understand them better in the end. A test should just run quietly in the background, and shouldn't disturb us.

There are three Groups of Assertions we can use:

Comparisons

We can check Variables- that is, whether they're TRUE or FALSE, equal or unequal and so forth - using this type of Assertion.

<?php
$this
->assertTrue($result, $message = FALSE, $group = 'Other')
?>

$result is the Variable, which we check. $message is the result, which is shown in red or green. $group is a Group of Assertions, and to the best of my knowledge, is hardly ever used.

To use this one, you have to enter meaningful test that actually makes sense. means something. If you use an ! = Test, you'll get green text that reads "A is not equal to B", which sounds like a mistake, but is simply true for that Assertion.

Content Check

If you're using $this->drupalGet() or $this->drupalPost() to load a site, the DrupalWebTestCase presents the result of the cUrl request as a SimpleXML object. With Assertions like $this->assertText(), we're dealing with the DOM. Assert Text for example, deletes all of the HTML Tags and searches in what's left. $this->assertRaw searches the entire result, including the HTML.

Form Elements

They've recently added a few Assertions for checking Form Elements. This is very convenient, and reduces the effort involved in checking what each field contains, or whether every checkbox is ticked. The whole thing works through an XPath query. We can also search for a field using assertFieldByXPath() query, and with $this->xpath(), we can even search the DOM. Its' important to note that this test doesn't work for checking multiple selection in field groups. Hopefully, there will be a patch for this soon.

API Functions

SimpleTest has lots of useful Auxiliary Functions. I won't repeat the drupal.org documentation here, I'll just give you a few Tips about how to use them.

POST / GET

The most important are $this->drupalGet() and $this->drupalPost(). $this->drupalGet() reloads a site using HTTP GET, so we can check it. $this->drupalPost() uses HTTP POST and sends a specified Array Variable. We can do the vast majority of our tests with this method. You have to fill out a form for most things in Drupal - here's how it's done. The Keys in an Array are the names of the fields you fill in. You create these using the POST Header method. It should be noted that field groups can all be filled in using an Array. You can proofread the result with xpath, albeit only by hand, but the API for the input works quite neatly. Firebug is best for finding field names. It's very hard work looking for all the field names, but there's no alternative way round it for the moment.

Example from drupal.org

<?php
$name
= $this->randomName();
$mail = "$name@example.com";
$edit = array(
 
'name' => $name,
 
'mail' => $mail,
 
'status' => FALSE, // checkboxes must be set with TRUE/FALSE rather than 1/0
);
$this->drupalPost('user/register', $edit, 'Create new account');
?>

randomString / randomName

These functions generate random strings. randomName() uses only Standard ASCII characters. You can use these functions for all the content you want to check. So don't set static strings, always use random strings with this function. In this way, we increase the reliability of our tests, and avoid errors creeping in.

drupalCreateUser

This method makes us a User. You should note that every User gets their own Role, and has certain rights transferred to them. It seems a bit strange that every User has an individual Role, but its' generally irrelevant for a test.

Emails Test

In Drupal, you can use the drupal_mail_wrapper() function to implement your own dispatch method for Emails. This is the actual dispatch function called drupal_mail_send() after a few global variables have been set. SimpleTest does all of this, and provides the ability to send and receive Emails,  and to test them.

<?php
function drupal_mail_wrapper(array $message) {
 
$captured_emails = variable_get('drupal_test_email_collector', array());
 
$captured_emails[] = $message;
 
variable_set('drupal_test_email_collector', $captured_emails);
  return
TRUE;
}
?>

The data in the System variable "drupal_test_email_collector" is cached, then loaded into the test and can be verified with Assertions. Here's an example from the ecard module-test.

<?php
$mails
= variable_get('drupal_test_email_collector', array());

   
$this->assertEqual(count($mails), 2, '2 emails found.');

   
// Check original email.
   
$this->assertEqual($mails[0]['id'], 'ecard_ecard-mail', 'Correct email id found.');
   
$this->assertEqual($mails[0]['from'], $from_email, 'From address is correctly set.');
   
$this->assertEqual($mails[0]['to'], $to_email, 'To address is correctly set.');
?>

If your module is using drupal_mail_wrapper(), you can see that the phpmailer module prevent errors, whereas in a PHP Test, the function would have declared itself twice.

<?php
/**
* Determine if PHPMailer is used to deliver e-mails.
*/
function phpmailer_enabled() {
  return
strpos(variable_get('smtp_library', ''), 'phpmailer');
}

if (
phpmailer_enabled() && !function_exists('drupal_mail_wrapper')) {
 
/**
   * Implementation of drupal_mail_wrapper().
   */
 
function drupal_mail_wrapper($message) {
   
module_load_include('inc', 'phpmailer', 'includes/phpmailer.drupal');
    return
phpmailer_send($message);
  }
}
?>

Testing Hooks with Mock-Modules

If you want to test a Hook, its best to use a Mini-module (Mock-module in English). The trick here is that in the .info file you can simply enter the module as its invisible:

; $Id$
name = &quot;XML-RPC Test&quot;
description = &quot;Support module for XML-RPC tests according to the validator1 specification.&quot;
package = Testing
version = VERSION
core = 7.x
files[] = xmlrpc_test.module

hidden = TRUE

Then you simply enable the module and the tests in setup().

Unit Tests

The current Backport of SimpleTest from Drupal 7 comes with a Unit Test Functionality. The Unit Test runs in the background, like all the other tests. The Class that's used here is called the "DrupalUnitTestCase". The big difference is that no database is available, and no functions to access files. We're testing one completely isolated function, in the manner described above.

Debug Tests

True, it's annoying when a test doesn't do what we want it to. First, where appropriate under 'admin/config/development/testing/settings', we need to 'Provide verbose information when running tests'. Therefore, for every drupalPost and drupalGet the result is saved, and you can see the site together with the given Variables. This is priceless. I used to forget a few things- like User rights - but now you're shown immediately whether you're logged in or not, or if rights are missing. This was vastly different in the early days, but now there's just a really simple setting you can use. I prefer to use an Assertion for debugging, and devote more of my time to the string.

Also highly useful is the debug() function from common.inc. This writes the test output directly in a practical way - via print_r().

<?php
/**
* Debug function used for outputting debug information.
*
* The debug information is passed on to trigger_error() after being converted
* to a string using _drupal_debug_message().
*
* @param $data
*   Data to be output.
* @param $label
*   Label to prefix the data.
* @param $print_r
*   Flag to switch between print_r() and var_export() for data conversion to
*   string. Set $print_r to TRUE when dealing with a recursive data structure
*   as var_export() will generate an error.
*/
function debug($data, $label = NULL, $print_r = FALSE) {
 
// Print $data contents to string.
 
$string = $print_r ? print_r($data, TRUE) : var_export($data, TRUE);
 
trigger_error(trim($label ? "$label: $string" : $string));
}
?>

OOP Tricks

For PHP Developers who may be unfamiliar with OOP, there are several OOP-things that you may find useful for organizing your tests. I suggest you create your own variant of the DrupalWebTestCase, and then import your tests from it. If, for example, I need some vocabulary for my tests , and I'm using a private method, I can create them by simply using any derivative of the Class, rather than having to implement the method every time. You can just call up the Basic method, and create your own versions.

Just take a look in the System.test. This creates one ModuleTestCase, then uses it repeatedly. This Class contains the Variable "$admin_user" in the setup() User scale, which is provided for all tests.

What's important is that there are various different Classes, even for the setup() method itself, and each runs in its own Sandbox. This can be necessary if you have unique static variables in functions that that need to be reset for each call. I had this problem with the NAT module.

Just type OOP when you need it. Use it, and spare yourself a huge amount of hassle.

Simple Test Clone

The SimpleTest Clone module is a derivative of the DrupalWebTestCase with some very useful functions. It's easy to use to test an existing installation. The data has to be migrated to the Sandbox and that's basically what makes this module work. I don't recommend that you align any tests to existing data. Any test has to be capable of functioning on its own. But it can help to clone an existing system.

Selenium

Selenium is principally a Macro Generator for Firefox. We have to click through our website and record everything. SimpleTest isn't a good way to test a business process. In the end we end up having to fill out forms, check them, and then check to see whether they've gone through. The trick with Selenium is that you only have to record things once, then the automatic repeats can just whiz through it. Its important that the machine doesn't forget anything and isn't' sloppy. If the test is well-designed as well, it's a great help. Selenium can do much more. The tests can be quite similar and adapted, as was the case with SimpleTest, and they can be reached via random inputs, which increases security. With different extensions, you can run tests on different browsers, or even a whole server-grid. Personally thought I've never attempted this. Selenium provides us with a useful complement to SimpleTest for ensuring the actual functionality of a site.

Tests for the Core

If you're patching something for Drupal 7, one of your first tasks will be to create a test for it. It's annoying, but it makes complete sense. The most frequent questions are:"Why bother and where do I put them?"My Tip is "Keep At It!"". You can learn a lot , and that' s good for the Community. You should also read the Guidelines. They're interesting, and include a few of the tests included in the Simple Test module. For me, the most frequent problem was working out the right place to put a test. You should always think twice about what you're doing, and ask around. The system-module is a good place for general system tests.

What tests can't do

We will probably never be able to test what something will look like automatically. You could probably obtain a representation of the extent to which a CSS Style had actually influenced sections of the HTML, like Firebug does, but there isn't an available Browser-Engine. Perhaps we could achieve something like a Firebug-Plugin, but that's probably a long way off.

Themes, to the best of my knowledge, have no tests. There is one for an Issue, but unfortunately, I've forgotten where.

Conclusion: It's worth it!

Writing a test can be both exhausting and drive you mad. Just like documentation. But in the end, it can save you a lot of time. If we change anything, we might have to rewrite the entire test, but we'll never create a test that can replace taking a deterministic approach to all our work from the outset. And if we write tests more often, we'll get faster at everything, and can perhaps raise all our working methods to a new level. And if you do master the art of test writing, you'll be shielded from comments like "That will never work" or "What have you done, nothing works now" forever!

I hope to see a few at my SimpleTest Session in Essen!