TDOG is a lightweight, portable and open source C++ xUnit Testing Framework. It features:
TDOG is maintained for g++ for Linux, MSVC and MinGW on Windows. It can be found at: http://bigangrydog.com
TDOG is distributed under the Apache 2.0 open source license, as follows:
Copyright (c) 2016 Andy Thomas
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
The TDOG Framework comprises a static library file and related header files. Makefiles are provided which will build both the static library and a self-testing application. See the README notes in the install directory for more information.
To link your project against TDOG:
Below is an example of a simple test and main function:
In this example, we have (presumably) written a function called "is_prime()" which we want to test.
We should include the header file "tdog.hpp", and define a simple test case to exercise our function with the help of the TDOG_TEST_CASE() macro. The test can then be executed with the macro TDOG_RUN_ALL(), or TDOG_RUN("*"), in the main function. The return result will be the number of test failures, or RAN_NONE (-1) if no tests were executed.
Behind the scenes, when the test case is declared, the test implementation will "auto-register" with a global instance of a class called tdog::runner. Normally, there is no need to concern yourself with this, as macros are available which "encapsulate" all commonly needed functionality.
The output of the above example will be written to the console, and will look like:
If we want to a different output and, say, write a JUnit compatible report to STDOUT as well as a HTML file for human consumption, our main function should look like this:
In our example, we have defined the test case in the same file as the main function. In reality, however, it would be normal practice to define them in header files, as they will be easier to manage this way. However, it is important to include test definitions in only one compilation unit (usually the one containing main()), otherwise "redefinition" linker errors will occur.
Below, are three test reports generated by the TDOG self-test application in the support output formats:
Use the browser back button to return.
TDOG supports the following report types:
Any one of the above report type may be written to STDOUT during execution, plus any combination written to file at the end of the test run.
Test cases are declared using the TDOG_TEST_CASE() macro, as follows:
The test name should not contain spaces, as it will be used internally to form the basis of a C++ class type. The test code should not return any result.
A test suite allows test cases to be run or disabled in groups. The suite name will be used directly as a C++ namespace around its test cases, so it must not contain spaces. Declaring tests within namespaced suites also allows the same test name to be used in multiple contexts.
Tests within the suite will automatically "self-register" with a global instance of the tdog::runner class. This means that they can simply be defined and run using the TDOG_RUN() macro, without having to deal with a runner class directly. However, if you need the flexibility to register tests explicity with your own code, the macro TDOG_EXPLICIT_SUITE() is provided to allows you to do this.
When declaring a test suite, it is important to "close" the suite with a TDOG_CLOSE_SUITE statement at the end of namespace scope. This is needed to allow the runner to properly track where one suite namespace ends, and a new one begins.
In the following example, a unit test is defined inside a test suite:
Because a suite is effectively a namespace, it may also declare data and functions for use in test implementation. This may be used to provide a mechanism to inject parameters into tests, and retrieve data set by them.
Suites may also be nested, and the following demonstrates a number of things:
Here we have a test case inside a nested suite that uses a data value declared within the suite. We set this to the required value prior to running the test.
Furthermore, to demonstrate an important point, we have declared the suites inside the ordinary namespace, "project_namespace". This is allowed. However, to run the test, you must refer to it with the string name "example_suite::beast_suite::number_test", rather than "project_namespace::example_suite::beast_suite::number_test", as TDOG cannot track the namespace name without the use of the TDOG_SUITE() macro.
Typically, suites and test cases would be defined in header files to be included in the compilation unit containing the main() function. Test suite names may also be contiguous, i.e. defined across multiple header files, provided the compiler supports the commonly available "__COUNTER__" macro. (See TDOG_COUNTER_ID for more information on this.)
Within a test case implementation, one or more assertions should be called using the TDOG assert macros.
The assert macro checks that the condition is true, and records a failure condition in the report if it is not. A test case is deemed to have passed if all its assert conditions are true, but failed if one or more assertions fail.
When an assert condition fails, a failure message will be displayed in the test report output, along with the filename and line number. If the reporting style is "verbose", test pass events and informational "PRINT" messages will also be displayed in the report.
If the test code throws an unhandled exception, the test case will be placed into an "error state". In this event, the test has effectively failed, but the state is considered distinct as it allows for possible test implementation errors to be easily distinguished.
TDOG assert macros include:
Additionally, the following negative (or "not equal") variants are available:
Variants allowing a custom message are also supported.
Text fixtures implement setup and teardown routines which ready the environment for a test, or series of tests. Setup and teardown routines can be implemented on a test level, suite level and globally.
For a test level fixture, the TDOG_TEST_FIXTURE() macro is provided, with which a test case is derived from a class or struct type provided which must implement the following methods:
Additionally, it must have a default constructor, and be instantiatable.
When the test is executed, the setup() method will be called first. It should return true on success, otherwise neither the test itself or the the teardown() routine will be called. If setup() returns false, or either setup() or teardown() throw an exception, the test will be set to the "error" state. You may also choose to put setup and teardown functionality in the constructor and destructor, but note that if the destructor throws an exception, behaviour is undefined.
Alternatively you may wish to implement setup and teardown functionality at the suite level. This is easy to do, and all that is needed is to create test cases prefixed with "setup" and "teardown" (ASCII case variations are accepted, as are leading underscores). TDOG will recognise these tests, and execute any "setup" named tests at the start of the suite, and "teardown" at the end, irrespective of where they are declared. These may call assert macros, like any normal test case and, should a setup test fail, the following tests in the suite will be skipped, including the teardown. If the suite contains nested suites (sub-suites) the teardown will be executed after those.
As an alternative to TDOG_TEST_FIXTURE(), you may prefer to wrap single unit tests in a suite, and provide them with "setup" and "teardown" named tests, as above.
You may also define "setup" and "teardown" tests in the global suite namespace simply by not enclosing them in a suite. For example:
Here, irrespective of where it appears, the teardown routine will be executed at the end of the test run, after all other tests, suites and suite level teardowns.
Let's assume, for a moment, that we have written several pseudo random number generation classes, each derived from the same base class which has an "interface" which looks like this:
What we would not want to do is write the same test code, multiple times, in order to test each of our concrete random generator classes. Ideally, we would want to define a single test implementation, and "instantiate" the same test with all concrete types to test them the same way.
Here's how to do it:
Unlike TDOG_TEST_PROTECTED() or TDOG_TEST_FIXTURE(), the test is not derived from the supplied "user_type", but is implementated internally as template class. In the above, we define the implementation using TDOG_DEFINE_REPEATED(), and give it a repeated "test typename". The typedef "USER_TYPE" will defined for use in the test implementation, as shown above. The test is not instantiated, however, until we declare TDOG_TEST_REPEATED(), which we must supply with a unique test name (i.e. "mt32_test"), the repeated test type ("rand_test"), and the concrete "user_type" with which to test.
Moreover, a repeated test can be used to fulfil a different test requirement.
Here, we will assume that we have written a function overloaded for both std::string std::wstring types, which looks like this:
When executed, the test names shown in the reports will be "narrow_test" and "wide_type" respectively, but file and line number information will refer to where the implementation is defined. Use the fully qualified "test names" when running or disabling test, rather than the repeated test typename.
To write out a formatted value to the report, use TDOG_PRINTF() instead. This is macro is similar to TDOG_PRINT(), except that it is possible to supply an additional value which will appear in a formatted message. The format is similar to that used in the C printf() function, although more restrictive.
The value parameter may be an any integer type upto 64-bit, or one of the following types:
To identify the location and format of the value in the output, the message string should contain one of the format specifiers below.
It is recommended that you use "\%g" generally, unless a specific format is required, as it will act as the placeholder for all supported value types.
Unlike the C printf() function, the TDOG_PRINTF() macro does not support a variable number of arguments, and the available specifiers are restricted to the exact forms above.
Additionally, the "%b" and "%k" specifiers have no equivalent in the C printf() function. The "%b" is used to represent a boolean value, and is substituted with either "true" or "false". The "\%k" applies to strings, but unlike "\%s", ASCII characters less than 0x20 (space) in value will be substitued with "escaped" characters including the replacements: "\t", "\n", "\v", "\f" and "\r".
For example, the tab character (0x09) will be replaced with "\t". For other characters, hex digits will be used instead, such as "\x08" for backspace.
Where the input message contains multiple specifiers, all will be substituted with the same value.
Test setup and teardown code can be implemented at either the test case level, or test suite level.
To implement individual test case setup and teardown, use a "test fixture", as described above. Setup code can be defined in the constructor of the associated data type, and teardown code in its destructor.
At the suite level, any test case name beginning with "setup" (or "_setup"), will always be executed at the start of the suite. Whereas, a test name beginning with "teardown" (or "_teardown") will be executed at the end. It is, therefore, possible to declare and manage suite-wide data this way.
Note. Case variations of the "setup" and "teardown" names are also permitted, and the variations "SETUP_MYSUITE" or "SetupMyTest" can be used with the same effect.
To execute all tests, use the following macro:
To run a single test case in the default suite, for example, call:
The input string may contain multiple names separated with a space or comma (or both). Test case names should be qualified with the suite name, and are case sensitive. If a test case is not part of a suite (i.e. it's in the default suite), then the name may optionally be prefixed with the default namespace "::". Any name referring to a test or suite which does not exist will be ignored.
Both test case and suite names may be specified, and will match according to the following rules:
To specify all tests in the default test suite (i.e. those declared outside of a suite) use "::". For example, the following will run the default suite, plus a single test in another suite:
Test cases which have been disabled will be not be executed, even if they are explicity specified in the "names" string.
When running tests, suite names will always be sorted, with the default suite executed first. Names are sorted according to an ordinal* comparison between their ASCII lower case values, and if equal, their case sensitive values. This will execute names with case variations together in a consistent way.
By default, test names within suites will be executed in the order they are registered with the runner, with a few exceptions:
If TDOG_SET_RUN_SORTED() is called to fully sort the run order, test cases inside suites will also be sorted during execution. It is good practice to ensure that tests do not depend on the run order, other than for explicitly declared setup() and teardown() operations.
The return value of TDOG_RUN() gives the number of test cases which either failed or raised an error. The result will be 0 if all test cases ran successfully without asserting failures or test errors. If, however, no tests were run, the return value will be tdog::RAN_NONE (-1), rather than 0. This distinguishes a successful test run from one that didn't actually test anything. This may occur, for example, if all test cases are disabled, if no tests have been registered with the runner, or the name input refers to non-existent tests.
A project name and version string can be associated with the tests using:
This information will be displayed in test reports provided it is set before running the tests.
Additionally, an author name can be associated with each test case. In this case, this must be set within the test implementation itself, as follows:
There are several ways to disable tests so that they are not performed during the test execution.
Tests can be disabled prior to execution, as follows:
Here, the input string may contain multiple names separated with a space or comma (or both as above). Entire suites can easily be disabled using the naming convention described in: Running tests and execution order
Additionally, a test case can be skipped from within its own implementation with a call to TDOG_SKIP_TEST(), as follows:
This can be used by the test author to skip the test. Skipping a test effectively disables it, however, skipped and disabled states are considered disctinct. If used, the skip macro should be placed at or near the start of the test code, as execution will occur up to the point it is called.
The TDOG_RUN_CMD() macro is provided to parse command line arguments and make running and disablising tests from the command line simple. This macro takes the command line arguments directly from the main() function, and runs the tests according to:
Other parameters will be ignored.
To run all tests, the command "--trun *" should be given or, alternatively, "--trall". The use of "--tdis" to disable tests is optional.
The following will run all with two tests disabled:
Windows style input, with a command prefix of "/", is also accepted and the following command is equivalent to that above:
Moreover, this macro is overloaded with the wchar_t type for use with a Windows main function.
As a demonstration, run the self-test application with the following:
This will run only the tests in the "message_suite". (Note the name of the application binary is platform dependent). See TDOG_RUN_CMD() for more information.
Test reports will display the hostname of the machine used to run the test. TDOG will look for the hostname under the environment variable "TDOG_HOSTNAME". If this doesn't exist, it will use "HOSTNAME" instead. If this is unspecified, the hostname will default to "localhost".