Writing Tests

PHPUnit tests

PHPUnit tests are related to a unit you are testing. The path of the tests is identical to the path of the unit inside the tests/…​ folder. Each Unit test class has a suffix …​Test and should extend the SuiteCRM\StateCheckerPHPUnitTestCaseAbstract class. Test methods prefix is test…​.

E.g. if the /path/to/Example.php contains a class Example then the related test should be in the tests/path/to/ExampleTest.php:

Example for a PHPUnit test

contents of tests/path/to/ExampleTest.php:

use SuiteCRM;

// note: SuiteCRM\StateCheckerPHPUnitTestCaseAbstract extends PHPUnit_Framework_TestCase;

class ExampleTest extends StateCheckerPHPUnitTestCaseAbstract {

    public function testDoingSomething() {
        $example = new Example();
        $results = $example->doSomething();
        $this->assertSame('expected value', $results);
    }

}

See more about PHPUnit tests at https://phpunit.readthedocs.io

Codeception Acceptance Tests

Implementation of state-safe acceptance tests.

Example for Acceptance tests

use SuiteCRM;

class SigninCest extends StateCheckerCestAbstract
{
    public function tryToTest(AcceptanceTester $I)
    {
        $I->wantTo('test my page');
    }
}

Test classes:

  • abstract class SuiteCRM\StateCheckerPHPUnitTestCaseAbstract

Implementation of state checker Codeception tests, class StateCheckerPHPUnitTestCaseAbstract extends PHPUnit_Framework_TestCase and override setUp() and tearDown() methods so if you want to call these methods in the inheritance test classes you should call parent::setUp() and parent::tearDown() in it.


  • abstract class SuiteCRM\StateCheckerCestAbstract

Implementation of state checker Codeception tests, it uses _before() and _after() methods so if you want to call these methods in the inheritance test classes you should call parent::_before() and parent::_after() in it.


  • trait SuiteCRM\StateCheckerCodeceptionTrait

Used in state checker Codeception tests.


  • trait SuiteCRM\StateCheckerTrait

Used in state checker tests.

  • Configuration of state checker tests - see SuiteCRM\StateCheckerConfig class.

State-Safe Tests

StateSaver and StateChecker

  • State check library for SuperGlobals, FileSystem and DataBase etc. (Implemented for: PHP Unit tests and Codeception Cests.)

  • State saver library - helper classes for developers.

Keep the system global environment state clean, especially in tests.
If a test leaves some extra data in the database, file system or super globals etc. this could change the behaviour of the other test processes. When you write a new test make sure it is state safe, which means tests should not leave any 'garbage' data in the test environment state such as database, file system, superglobals etc..

  • Check System State

Unit test should be stateless, developers can change class StateCheckerConfig properties in local instance to make sure Unit tests don’t change the system state.

PHPUnit tests have to extend SuiteCRM\StateCheckerPHPUnitTestCaseAbstract instead of PHPUnit_Framework_TestCase.

Codeception Cests should extend SuiteCRM\StateCheckerCestAbstract class. (note: don’t forget to call parent::_before() and parent::_after() if you want to override these methods in your tests)

The following classes help you to write state-safe code (tests), of course you can use some of these classes in any case where you have to change and restore anything in the global system state:


  • Class SuiteCRM\StateSaver

Saves and checks the system state and reports any state change in the following:

  • Database

  • File system

  • Super globals

  • PHP error reporting level

  • PHP configuration options

See also SuiteCRM\StateChecker class.

  • Examples for storing superglobals:

use SuiteCRM;

$_POST['foo'] = 'bar';

// create a new instance of StateChecker:
$stateSaver = new StateSaver();

// save all superglobals
$stateSaver->pushGlobals();

$_POST['foo'] = 'bazz';

// restore super globals
$stateSaver->popGlobals();

echo $_POST['foo']; // output: bar
  • Examples for storing database tables:

use SuiteCRM;

// create a new instance of StateChecker:
$stateSaver = new StateSaver();
$stateSaver->pushTable('stufftable');

// modify your database table here. (insert/update/delete..)
...

$stateSaver->popTable('stufftable');

// restore your database table 'stufftable' here.
  • Example usage for System State Saver in test scripts:

StateSaver class is a helper library, typically for test scripts but usable everywhere:

	// Save state

        // Create an instance of StateChecker
        $state = new \SuiteCRM\StateSaver();
        $state->pushGlobals();    // saving superglobals
        $state->pushTable('your_module_stuffs'); // saving a database table
        $state->pushFile('your_file.txt');

        // Tests

	// Do some test changes in superglobals
        $_POST['foo'] = 'bar';

        // Test changes in database tables (example only)
        $stuff = BeanFactory::getBean('YourModuleStuff', '{your-module-stuff-id}');
        $stuff->your_property = 'baz';
        $stuff->save();

        // Some changes in your test file:
        file_put_contents('your-file.txt', 'New contents here: ' . rand(1, 10000));


        // Clean up

        $state->popFile('your_file.txt');
        $state->popTable('your_module_stuffs');  // restore table
        $state->popGlobals();  // restore globals

        // ... here you should get the restored super globals, database tables and files.
  • Available methods:

Error Collection:

    /**
     * Retrieve if any error occurred in storing/restoring processes.
     *
     * @return array
     */
    public function getErrors();
    /**
     * Clear all collected error information about latest storing/restoring processes.
     */
    public function clearErrors();
    /**
     * Retrieve if any error occurred in storing/restoring processes and
     * clear all collected error information about latest storing/restoring processes.
     *
     * @return array
     */
    public function getErrorsClear();

Push/pop stack storage:

    /**
     * Save any value into state store at a key and namespace.
     *
     * @param mixed $value
     * @param string $key
     * @param string $namespace
     */
    public function push($value, $key, $namespace);
    /**
     * Restore any value from state store at a key and namespace.
     *
     * @param string $key
     * @param string $namespace
     * @return mixed
     */
    public function pop($key, $namespace);
    /**
     * Save a global variable into storage at an optional namespace.
     *
     * @param string $key
     * @param string $namespace
     */
    public function pushGlobal($key, $namespace = 'GLOBALS');
    /**
     * Restore a global value from storage at an optional namespace.
     *
     * @param string $key
     * @param string $namespace
     */
    public function popGlobal($key, $namespace = 'GLOBALS');
    /**
     * Save all super globals which are specified in configuration.
     * @see StateCheckerConfig
     *
     * pushGlobals
     */
    public function pushGlobals();
    /**
     * Restore all super globals which are specified in configuration.
     * @see StateCheckerConfig
     *
     * popGlobals
     */
    public function popGlobals();
    /**
     * Save all defined global variable name.
     * (note: this function does not store the values, so use it carefully)
     *
     * pushGlobalKeys
     */
    public function pushGlobalKeys();
    /**
     * Restore all defined global variable name.
     * (note: this function does not restore the values, so use it carefully)
     *
     * popGlobalKeys
     */
    public function popGlobalKeys();
    /**
     * Save Error Reporting Level into the store at an optional key and namespace.
     * (note: error level should not be changed for any reason, so use it for own risk)
     *
     * @param string $key
     * @param string $namespace
     */
    public function pushErrorLevel($key = 'level', $namespace = 'error_reporting');
    /**
     * Restore Error Reporting Level from the store at an optional key and namespace.
     * (note: error level should not be changed for any reason, so use it for own risk)
     *
     * @param string $key
     * @param string $namespace
     */
    public function popErrorLevel($key = 'level', $namespace = 'error_reporting');
    /**
     * Save all data from a database table into store at an optional namespace.
     *
     * @param string $table
     * @param string $namespace
     * @throws StateSaverException
     */
    public function pushTable($table, $namespace = 'db_table');
    /**
     * Restore all data into a database table from store at an optional namespace.
     *
     * @param string $table
     * @param string $namespace
     */
    public function popTable($table, $namespace = 'db_table');

File system

    /**
     * Save a file contents.
     *
     * @param string $filename
     * @throws StateSaverException
     */
    public function pushFile($filename);
    /**
     * Restore a file contents.
     *
     * @param string $filename
     * @return boolean
     * @throws StateSaverException
     */
    public function popFile($filename);

PHP Configuration options:

    /**
     * Getter for PHP Configuration Options
     * @see more at StateCheckerConfig::$phpConfigOptionKeys
     *
     * @return array
     */
    public static function getPHPConfigOptions();
    /**
     * Setter for PHP Configuration Options
     * @see more at StateCheckerConfig::$phpConfigOptionKeys
     *
     * @param array $configOptions
     * @throws StateSaverException
     */
    public static function setPHPConfigOptions($configOptions);
    /**
     * Store PHP Configuration Options
     * @see more at StateCheckerConfig::$phpConfigOptionKeys
     *
     * @param string $key
     * @param string $namespace
     */
    public function pushPHPConfigOptions($key = 'all', $namespace = 'php_config_options');
    /**
     * Restore PHP Configuration Options
     * @see more at StateCheckerConfig::$phpConfigOptionKeys
     *
     * @param string $key
     * @param string $namespace
     */
    public function popPHPConfigOptions($key = 'all', $namespace = 'php_config_options');

  • class SuiteCRM\StateSaverException (Exception)

Simple Exception to catch and handle the state changes.


  • class SuiteCRM\StateChecker

Saves and checks the system state and reports any state change in the following: - Database - File system - Super globals - PHP error reporting level - PHP configuration options

See more about the SuiteCRM\StateChecker configuration in the SuiteCRM\StateCheckerConfig class.

Examples:

use SuiteCRM;

// saving a hash of the current state at this point.
$stateChecker = new StateChecker();

// ... do something to change the current system state

try {

    // getting a hash of the current state or
    // throws an exception if the state doesn't match with the previously saved state.

    $hash = $stateChecker->getStateHash();

} catch (StateCheckerException $e) {

    // state is changed!
    // use the following exception to detect / debug the problem: (optional)

    $info = $e->getMessage();

}

  • class SuiteCRM\StateCheckerException (extends PHP standard Exception)

Simple Exception to catch and handle the state changes.


  • class SuiteCRM\StateCheckerConfig

Configuration of SuiteCRM\StateChecker and StateChecker Tests classes such as:

SuiteCRM\StateCheckerPHPUnitTestCaseAbstract
SuiteCRM\StateCheckerUnitAbstract
SuiteCRM\StateCheckerCestAbstract

SuiteCRM\StateCheckerConfig configuration options have default values and each is available in
$sugar_config['state_checker'][$key].

Each configuration value is available with a getter method:
SuiteCRM\StateCheckerConfig::get($key) where $key could be any of the following:


globalKeys

The SuperGlobals Collection determines which super globals are stored and restored.

Usage:

$value = SuiteCRM\StateCheckerConfig::get('globalKeys')

Default return value:

array('_POST', '_GET', '_REQUEST', '_SESSION', '_SERVER', '_ENV', '_FILES', '_COOKIE')

SuiteCRM config overrides:

$sugar_config['state_checker']['global_keys']

fileExludeRegexes

Array of regexp for excluding files from state checking.
Tests won’t check hash for these files so won’t fail.

Usage:

$value = SuiteCRM\StateCheckerConfig::get('fileExludeRegexes')

Default value:

array(
    '/\/\.git\//',
    '/\/cache\//',
    '/\.log$/',
    '/\/tests\/_output\//',
    '/\/blowfish\//',
    '/\/upload\//',
    '/\/vendor\//',
    '/\/sugarfield_jjwg_maps_/',
    '/\/vardefs.ext.php$/',
    '/\/modules\/AOD_Index\/Index\/Index\//',
    '/\/travis\/build\//',
)

SuiteCRM config overrides:

$sugar_config['state_checker']['file_exclude_regexes']

autoRun

Automatically run state collection in StateChecker constructor.

Usage:

$value = SuiteCRM\StateCheckerConfig::get('autoRun')

Default value:

true

SuiteCRM config overrides:

$sugar_config['state_checker']['auto_run']

saveTraces

Saves trace info on state-hash mismatch
(Slow running but gives more information about the error location, use in development only)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('saveTraces')

Default value:

false

SuiteCRM config overrides:

$sugar_config['state_checker']['save_traces']

redefineMemoryLimit

Redefine memory limit
(For more memory expensive tasks, for e.g collection stack trace information when $saveTraces is ON,
use in development only)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('redefineMemoryLimit')

Default value:

false

SuiteCRM config overrides:

$sugar_config['state_checker']['redefine_memory_limit']

storeDetails

Stores more information about hash-mismatch, which part has state of globals/filesys/database.
(Slow working but gives more information about the error location, use in development only)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('storeDetails')

Default value:

true

SuiteCRM config overrides:

$sugar_config['state_checker']['store_details']

testStateCheckMode

Enum specified that tests need to check system state for Test Cases behaviour, possible values:

SuiteCRM\StateCheckerConfig::RUN_NEVER: State check and save never runs.
SuiteCRM\StateCheckerConfig::RUN_PER_TEST: State check runs after each test methods.
SuiteCRM\StateCheckerConfig::RUN_PER_CLASSES: State check runs after each test class.

Note: Mode RUN_PER_CLASSES affects PHPUnit Test Cases only
Note: Developer mode overrides this value.

Usage:

$value = SuiteCRM\StateCheckerConfig::get('testStateCheckMode')

Default value:

SuiteCRM\StateCheckerConfig::RUN_PER_CLASSES

SuiteCRM config overrides:

$sugar_config['state_checker']['test_state_check_mode']

testsUseStateChecker

Test using StateChecker
(Slow working but gives more information about the error location, use in development only)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('testsUseStateChecker')

Default value:

true

SuiteCRM config overrides:

$sugar_config['state_checker']['tests_use_state_checker']

testsUseAssertionFailureOnError

Test shows up an assertion failure when there is a hash-mismatch,
use $testsUseStateChecker also, $testsUseAssertionFailureOnError applied only if $testsUseStateChecker = true (use in development only)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('testsUseAssertionFailureOnError')

Default value:

true

SuiteCRM config overrides:

$sugar_config['state_checker']['tests_use_assertion_failure_on_error']

testsFailureExcludeKeys

Tests won’t check hash for these keys so won’t fail
(It should be empty)

Usage:

$value = SuiteCRM\StateCheckerConfig::get('testsFailureExcludeKeys')

Default value:

array()

SuiteCRM config overrides:

$sugar_config['state_checker']['tests_failure_exclude_keys']

phpConfigOptionKeys

State saver needs to know which PHP configuration options to save/restore.

Usage:

$value = SuiteCRM\StateCheckerConfig::get('phpConfigOptionKeys')

Default value:

array('max_execution_time', 'display_errors', 'display_startup_errors')

SuiteCRM config overrides:

$sugar_config['state_checker']['php_configuration_option_keys']

How to: Testing Imap

Developers able to write IMAP test (unit and acceptance tests).

If developer mode set and logger level is debug the ImapHandler class logs each imap calls, then each imap method call logged with called parameter and return values so that the developers and testers can see exactly whats going on in the background. If imap_test set, the system use fake calls by pre defined method parameters and return values in ImapHandlerFakeCalls.php so that the developers able to add more tests for any email functionality even if it needs a valid imap resource.

  • ImapHandler: Wrapper class for functions of IMAP PHP built in extension.

  • ImapHandlerFacotry: Retrieves an ImapHandlerInterface. It could be ImapHandler or ImapHandlerFake. Use $sugar_config['imap_test'] = true in config_override.php to set test mode on.

  • ImapHandlerFake: Wrapper class for functions of IMAP PHP built in extension. (tests only)

  • ImapHandlerFakeCalls.php: describes the fake imap functions return values for each function calls with a specific parameters in every test scenario.

  • ImapHandlerFakeData: For tests only, it deals fake return values for fake calls on an IMAP wrapper.

  • ImapHandlerInterface: IMAP wrappers need to implements so that the system can use it as an IMAP handler.

  • ImapTestSettingsEntry.php: for an entry point to set the current test scenario in any acceptance test can call it. (entry point example: index.php?entryPoint=setImapTestSettings&imap_test_settings=[index of array in ImapHandlerFakeCalls.php])

Example usage in Unit Tests:

first needs to include the following files:

include_once __DIR__ . '/../../../../../include/Imap/ImapHandlerFakeData.php';
include_once __DIR__ . '/../../../../../include/Imap/ImapHandlerFake.php';

Example unit test for imap connection (using fake imap data)

     public function testConnectMailserverUseSsl()
     {
        // saving state
         $state = new SuiteCRM\StateSaver();
         $state->pushGlobals();

         // using fake imap handler behaviour in test
         $fake = new ImapHandlerFakeData();

         // set up the fake handler behaviour
         $fake->add('isAvailable', null, [true]);
         $fake->add('setTimeout', [1, 60], [true]);
         $fake->add('setTimeout', [2, 60], [true]);
         $fake->add('setTimeout', [3, 60], [true]);
         $fake->add('getErrors', null, [false]);
         $fake->add('getConnection', null, [function () {
             // the current crm code needs a valid resource to an imap server
             // but also will accept a file resource
             return fopen('fakeImapResource', 'w+');
         }]);
         $fake->add('getMailboxes', ['{:/service=/notls/novalidate-cert/secure}', '*'], [[]]);
         $fake->add('ping', null, [true]);
         $fake->add('reopen', ['{:/service=}', 32768, 0], [true]);

         // instantiate a fake imap handler
         $imap = new ImapHandlerFake($fake);

         $_REQUEST['ssl'] = 1;

         // using fake imap in InboundEmail class (only for testing)
         $ie = new InboundEmail($imap);

         // test connection, it should pass
         $ret = $ie->connectMailserver();
         $this->assertEquals('true', $ret);

         // restore state
         $state->popGlobals();
     }

useful config variables:

$sugar_config['imap_test'] = true;
$sugar_config['logger']['level'] = 'debug';
$sugar_config['stack_trace_errors'] = false; // set to true for more details
$sugar_config['developerMode'] = true;
$sugar_config['show_log_trace'] = false; // set to true for more details

References

Content is available under GNU Free Documentation License 1.3 or later unless otherwise noted.