Google Test

Embed Google Test into my work as a formal Unit Testing environment

It's time to include Google test into my code base and make my development work more efficiently, or more cumbersome, let's see:
(This is just my learning note, which includes a lot of original content from the primer of the google test. Thank you the development team of the Google Test.)

Installation

It seems that there is no installation required. the google test module seems not too expensive to compile. Maybe it's a good idea to just keep a portable version into my source tree.

I believe this is the design point, portability, that is emphasised by the prime at the beginning.

<But it is still need a small amount of work to link google test into Make, which is my current build tool. But I am planning to change to CMake.>

Concept

Google test has a different definition of the 'Test Case' or 'Test' from the ISTQB (International Software Testing Qualification Board) for historical reason. But the legacy is kept because of the difficulty of changing everything in the code and more importantly make it not wrong!!! So: 

Google Test vs ISTQB

TEST() = Test Case

Test Case = Test Suite

There are three assertion results: 

1. success, nothing to say
2. nonfatal failure (EXPECT_*), process keeps going 
3. fatal failure (ASSERT_*), stop running at the point of code produced fatal failure.

A Test case contains at least one test. It is encouraged to group tests into test with respect to the structure of the tested code.

A Test program can contain multiple test cases.

Assertions

ASSERT_EQ(x, y) << "extra message to be reported"

for (int i = 0; i < x.size(); ++i{

  EXPECT_EQ(x[i], y[i]) << "Vectors x and y differ at index" << i;

}

Anything that can be fed into ostream, can also be used as a custom failure message and fed by <<.

The following is some common assertions, they are just self explained by their names: 

[ASSERT | EXPECT]_TRUE  (condition)

[ASSERT | EXPECT]_FALSE (condition)

[ASSERT | EXPECT]_EQ    (val1, val2)

[ASSERT | EXPECT]_NE    (val1, val2)

[ASSERT | EXPECT]_LT    (val1, val2)

[ASSERT | EXPECT]_LE    (val1, val2)

[ASSERT | EXPECT]_GT    (val1, val2)

[ASSERT | EXPECT]_GE    (val1, val2)

<WARNING: the following string tests should only be used to compare C strings, for string object, the Primer recommend to use the above value tests>

[ASSERT | EXPECT]_STREQ (str1, str2)

[ASSERT | EXPECT]_STRNE (str1, str2)

[ASSERT | EXPECT]_STRCASEEQ (str1, str2)

[ASSERT | EXPECT]_STRCASENE (str1, str2)

Simple Tests

Creation of a test in a test case using the macro, TEST() :

TEST(testCaseName, testName) {

...  test body ... 

}

For example, for a given function, which is to be tested: 

int Factorial(int n);

The following tests can be created: 

// Test factorial of 0.

TEST(FactorialTest, HandlesZeroInput) {

  EXPECT_EQ(1, Factorial(0));

}

// Test factorial of positive numbers:

TEST(FactorialTest, HandlesPositiveInput) {

  EXPECT_EQ(1, Factorial(1));

  EXPECT_EQ(2, Factorial(2));

  EXPECT_EQ(6, Factorial(3));

  EXPECT_EQ(40320, Factorial(8));

}

So two related tests, HandlesZeroInput and HandlesPositiveInput have been created in the test case FactorialTest.

Test Fixtures

From my understand, the test fixture is just a test class with an extra context environment in which all tests are supposed to run. 

The following steps can create a test fixture: 

1. Derive a class from ::testing::Test and make sure its members can be accessed by its sub-classes.
2. Have a fun within the class body!!
3. To make any preparation, write a default constructor or SetUp() function.
4. To make any clean up, write a default destructor or TearDown() function.
5. Define subroutines if it is necessary.

Constructor/Destructor vs SetUp()/TearDown():

• It is a per-test set-up and tear-down logic for both. It means that, for each test, the fixture will be created and destroyed. 
• Normally, constructor/destructor is preferred: 
• constructor can initialise const for the test.
• sub-class will call constructor and destructor automatically. The test fixture will following the super-class. There is no risk of forgetting the calling of SetUp()/TearDown()
• For other rare cases, SetUp()/TearDown() is preferred: 
• It's not ideal to have code in the destructor that can throw exception. But TearDown() can handle it.
• No virtual function call in Constructor/Destructor, because they are statically bound. It is not possible to call overridden methods in a derived class.

For example for the testing of the following Queue class: 

template <typename E> 

class Queue {

  public:

    Queue();

    void Enqueue(const E& element);

    E* Dequeue();

    size_t size();

    ...

};

To create a test fixture for the testing: 

class QueueTest : public ::testing::Test{

  protected:

    virtual void SetUp() {

      q1_.Enqueue(1);

      q2_.Enqueue(2);

      q3_.Enqueue(3);

    }

    // virtual void TearDown()  // not need to clean up anything inside

    Queue<int> q0_;

    Queue<int> q1_;

    Queue<int> q2_;

};

To create tests for the above test fixture: 

TEST_F(QueueTest, IsEmptyInitially){

  EXPECT_EQ(0, q0.size());

}

TEST_F(QueueTest, DequeueWorks){

  int* n = q0_.Dequeue();

  EXPECT_EQ(NULL, n);

  n = q1_.Dequeue();

  ASSERT_TRUE(n != NULL);

  EXPECT_EQ(1, *n);

  EXPECT_EQ(0, q1_.size());

  delete n; // test has responsibility to clean n!!

  n = q2_.Dequeue();

  ASSERT_TRUE(n != NULL);

  EXPECT_EQ(2, *n);

  EXPECT_EQ(1, q2_.size());

  delete n; 

}

The things are behind the above tests: 

1. Construct QueueTest object (eg. t1)
2. t1.SetUp()
3. run IsEmptyInitially test on t1
4. t1.TearDown()
5. t1 is destructed
6. 1 - 5 repeats for another QueueTest (t2), this time running the DequeueWorks test.

Invoke tests

• TEST() and TEST_F() implicitly register their tests. 
• Use RUN_ALL_TESTS() to run all tests. It return 0 on success, 1 otherwise.
• Don't ignore the return value of the RUN_ALL_TESTS(), it should be passed to main and get it returned.
• Call RUN_ALL_TESTS() only once. Multiple calling is not supported by the Google Test.

main() function of the Google Test: 

int main (int argc, char **argv) {

  ::testing::InitialGoogleTest(&argc, argv);

  return RUN_ALL_TESTS();

}

::testing::InitialGoogleTest get arguments from main and remove the flags that it recognised. This provides a user with a flexibility to control the tests via flags.

By linking the test with test_main library, the user does not need to create a main for the tests.

Conclusion

Here is all primer of using the Google Test. There are still more features to be investigated. That would be included in my following notes.

Follow up

Having tested the sample tests in the release package of the Google Test, all is good at least for the sample1, which is the only one sample I have tried :(.

However, after embedding the Google Test into myself project. It is getting strange. The message is as follows:

Undefined symbols for architecture x86_64:
  "std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > space_tokeniser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)", referenced from:
      (anonymous namespace)::StrTokeniserTest_SpaceTokeniserTest_Test::TestBody() in string_tokeniser_unittest.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [string_tokeniser_unittest] Error 1

My initial thought is that the different linking libraries between Google Test and my code cause this discrepancy, like what the most stack overflow contributors' reply to this message in the MAC system.

However, in my Linux box, the problem persists and produces another error message. So I believe the replies is not fit for my situation.

In the end, a common linking error thread (undefined-reference-to-template-class-constructor) on the stack overflow is found very helpful for my solution, which moves template function definition into .h file to replace its original declaration there.

Thanks to Arron, who provided the details of the solution and description. It's always good to solve problems finally, even though it is exchanged by a large time spending.