Google Test

Embed Google Test into my work as a formal Unit Testing environment

It's time to include Google test into my code base and make my development work more efficiently, or more cumbersome, let's see:
(This is just my learning note, which includes a lot of original content from the primer of the google test. Thank you the development team of the Google Test.)

Installation

It seems that there is no installation required. the google test module seems not too expensive to compile. Maybe it's a good idea to just keep a portable version into my source tree.

I believe this is the design point, portability, that is emphasised by the prime at the beginning.

<But it is still need a small amount of work to link google test into Make, which is my current build tool. But I am planning to change to CMake.>

Concept

Google test has a different definition of the 'Test Case' or 'Test' from the ISTQB (International Software Testing Qualification Board) for historical reason. But the legacy is kept because of the difficulty of changing everything in the code and more importantly make it not wrong!!! So: 

Google Test vs ISTQB

TEST() = Test Case

Test Case = Test Suite

There are three assertion results: 

1. success, nothing to say
2. nonfatal failure (EXPECT_*), process keeps going 
3. fatal failure (ASSERT_*), stop running at the point of code produced fatal failure.

A Test case contains at least one test. It is encouraged to group tests into test with respect to the structure of the tested code.

A Test program can contain multiple test cases.

Assertions

ASSERT_EQ(x, y) << "extra message to be reported"

for (int i = 0; i < x.size(); ++i{

  EXPECT_EQ(x[i], y[i]) << "Vectors x and y differ at index" << i;

}

Anything that can be fed into ostream, can also be used as a custom failure message and fed by <<.

The following is some common assertions, they are just self explained by their names: 

[ASSERT | EXPECT]_TRUE  (condition)

[ASSERT | EXPECT]_FALSE (condition)

[ASSERT | EXPECT]_EQ    (val1, val2)

[ASSERT | EXPECT]_NE    (val1, val2)

[ASSERT | EXPECT]_LT    (val1, val2)

[ASSERT | EXPECT]_LE    (val1, val2)

[ASSERT | EXPECT]_GT    (val1, val2)

[ASSERT | EXPECT]_GE    (val1, val2)

<WARNING: the following string tests should only be used to compare C strings, for string object, the Primer recommend to use the above value tests>

[ASSERT | EXPECT]_STREQ (str1, str2)

[ASSERT | EXPECT]_STRNE (str1, str2)

[ASSERT | EXPECT]_STRCASEEQ (str1, str2)

[ASSERT | EXPECT]_STRCASENE (str1, str2)

Simple Tests

Creation of a test in a test case using the macro, TEST() :

TEST(testCaseName, testName) {

...  test body ... 

}

For example, for a given function, which is to be tested: 

int Factorial(int n);

The following tests can be created: 

// Test factorial of 0.

TEST(FactorialTest, HandlesZeroInput) {

  EXPECT_EQ(1, Factorial(0));

}

// Test factorial of positive numbers:

TEST(FactorialTest, HandlesPositiveInput) {

  EXPECT_EQ(1, Factorial(1));

  EXPECT_EQ(2, Factorial(2));

  EXPECT_EQ(6, Factorial(3));

  EXPECT_EQ(40320, Factorial(8));

}

So two related tests, HandlesZeroInput and HandlesPositiveInput have been created in the test case FactorialTest.

Test Fixtures

From my understand, the test fixture is just a test class with an extra context environment in which all tests are supposed to run. 

The following steps can create a test fixture: 

1. Derive a class from ::testing::Test and make sure its members can be accessed by its sub-classes.
2. Have a fun within the class body!!
3. To make any preparation, write a default constructor or SetUp() function.
4. To make any clean up, write a default destructor or TearDown() function.
5. Define subroutines if it is necessary.

Constructor/Destructor vs SetUp()/TearDown():

• It is a per-test set-up and tear-down logic for both. It means that, for each test, the fixture will be created and destroyed. 
• Normally, constructor/destructor is preferred: 
• constructor can initialise const for the test.
• sub-class will call constructor and destructor automatically. The test fixture will following the super-class. There is no risk of forgetting the calling of SetUp()/TearDown()
• For other rare cases, SetUp()/TearDown() is preferred: 
• It's not ideal to have code in the destructor that can throw exception. But TearDown() can handle it.
• No virtual function call in Constructor/Destructor, because they are statically bound. It is not possible to call overridden methods in a derived class.

For example for the testing of the following Queue class: 

template <typename E> 

class Queue {

  public:

    Queue();

    void Enqueue(const E& element);

    E* Dequeue();

    size_t size();

    ...

};

To create a test fixture for the testing: 

class QueueTest : public ::testing::Test{

  protected:

    virtual void SetUp() {

      q1_.Enqueue(1);

      q2_.Enqueue(2);

      q3_.Enqueue(3);

    }

    // virtual void TearDown()  // not need to clean up anything inside

    Queue<int> q0_;

    Queue<int> q1_;

    Queue<int> q2_;

};

To create tests for the above test fixture: 

TEST_F(QueueTest, IsEmptyInitially){

  EXPECT_EQ(0, q0.size());

}

TEST_F(QueueTest, DequeueWorks){

  int* n = q0_.Dequeue();

  EXPECT_EQ(NULL, n);

  n = q1_.Dequeue();

  ASSERT_TRUE(n != NULL);

  EXPECT_EQ(1, *n);

  EXPECT_EQ(0, q1_.size());

  delete n; // test has responsibility to clean n!!

  n = q2_.Dequeue();

  ASSERT_TRUE(n != NULL);

  EXPECT_EQ(2, *n);

  EXPECT_EQ(1, q2_.size());

  delete n; 

}

The things are behind the above tests: 

1. Construct QueueTest object (eg. t1)
2. t1.SetUp()
3. run IsEmptyInitially test on t1
4. t1.TearDown()
5. t1 is destructed
6. 1 - 5 repeats for another QueueTest (t2), this time running the DequeueWorks test.

Invoke tests

• TEST() and TEST_F() implicitly register their tests. 
• Use RUN_ALL_TESTS() to run all tests. It return 0 on success, 1 otherwise.
• Don't ignore the return value of the RUN_ALL_TESTS(), it should be passed to main and get it returned.
• Call RUN_ALL_TESTS() only once. Multiple calling is not supported by the Google Test.

main() function of the Google Test: 

int main (int argc, char **argv) {

  ::testing::InitialGoogleTest(&argc, argv);

  return RUN_ALL_TESTS();

}

::testing::InitialGoogleTest get arguments from main and remove the flags that it recognised. This provides a user with a flexibility to control the tests via flags.

By linking the test with test_main library, the user does not need to create a main for the tests.

Conclusion

Here is all primer of using the Google Test. There are still more features to be investigated. That would be included in my following notes.

Follow up

Having tested the sample tests in the release package of the Google Test, all is good at least for the sample1, which is the only one sample I have tried :(.

However, after embedding the Google Test into myself project. It is getting strange. The message is as follows:

Undefined symbols for architecture x86_64:
  "std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > space_tokeniser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)", referenced from:
      (anonymous namespace)::StrTokeniserTest_SpaceTokeniserTest_Test::TestBody() in string_tokeniser_unittest.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [string_tokeniser_unittest] Error 1

My initial thought is that the different linking libraries between Google Test and my code cause this discrepancy, like what the most stack overflow contributors' reply to this message in the MAC system.

However, in my Linux box, the problem persists and produces another error message. So I believe the replies is not fit for my situation.

In the end, a common linking error thread (undefined-reference-to-template-class-constructor) on the stack overflow is found very helpful for my solution, which moves template function definition into .h file to replace its original declaration there.

Thanks to Arron, who provided the details of the solution and description. It's always good to solve problems finally, even though it is exchanged by a large time spending.

Use LD_PRELOAD to load another version library before running

LD_PRELOAD is a fantastic method to debug the code or library.

Basically, this just record other's work on the LD_PRELOAD and description. That's is fair enough, so I won't bored to modify that:

You Are Here

LD_PRELOAD fun

with 3 comments

Here is a welcome digression from my previous Twitter oriented posts. I’m starting to play around with the LD_PRELOAD feature in the Linux dynamic linker. For those who might not know what this feature is, here is the description from ld.so (8).

LD_PRELOAD
              A whitespace-separated list of additional,  user-specified,  ELF
              shared  libraries  to  be loaded before all others.  This can be
              used  to  selectively  override  functions   in   other   shared
              libraries.   For  setuid/setgid  ELF binaries, only libraries in
              the standard search directories that are  also  setgid  will  be
              loaded.

So in pratical term, any libraries you specify in the LD_PRELOAD environment variable will loaded before any system libraries. This means that dynamic symbols in a loading program will be first searched in those libraries before being searched anywhere else. This means you can override any defined symbol you want in standard libraries.
Let’s start with a rather juvenile example. This will change the behavior of the read (2) function in order to make the user believe a file might have a different content.

ssize_t read(int fd, void *buf, size_t count) {     static int done = 0;     if (!done) {         char silly_str[] = "Haha you got overriden.\n";         size_t s = count > sizeof(silly_str) ? sizeof(silly_str) : count;         memcpy(buf, silly_str, s);         done = 1;         return s;     }     else return 0; }

If you compile this inside a library that is called, for example, libread.so, you can test this code by running:

> /bin/cat /etc/fstab
# /etc/fstab: static file system information.
#
...
> LD_LIBRARY_PATH=. LD_PRELOAD=libread.so /bin/cat /etc/fstab
Haha you got overriden.

That in itself is just a rather silly prank you can play on your friend’s computer if you happen to have access to it. Experienced programmer will start seeing potential uses for LD_PRELOAD. I am getting to that.
The subject of our next example will be the honorable ls (1). ls uses the opendir (3) function to open a directory and browse its files. It should react properly if it can’t open the directory. One way to test this is to make opendir() return NULL and observe how the caller reacts. You can do that using LD_PRELOAD.

DIR *opendir(const char *name) {     return NULL; }

> LD_LIBRARY_PATH=. LD_PRELOAD=libls1.so /bin/ls /tmp
/bin/ls: cannot open directory /tmp

What can you do now if you want to preserve part of the behavior of the function, or modify they result it returns? Your preloaded library will then need to use libdl to dynamically load the function it wants to modify the behavior.
The following example is a very simple override of the opendir (3) function which open a different directory than what the caller expects. I will explain more in detail the details of this function below.

DIR *opendir(const char *name) {     DIR *(*libc_opendir)(const char *name);     *(void **)(&libc_opendir) = dlsym(RTLD_NEXT, "opendir");     return libc_opendir("/tmp"); }

libdl is fortunately very simple to use. The naive approach would be to use dlopen (3) to open the C library, then get the pointer to the function you are calling using dlsym (3). In theory, this technique is valid and working, but doing that circumvents the LD_PRELOAD mechanisme because preloaded libraries can be chained and calling directly into the C library prevents other caller to override our own function.
In practice, calling dlopen() on libc on an Ubuntu Karmic system made some program crash and burn for reasons I will not attempt to explain. The next technique should be preferred on Linux system, especially when dealing with the system C library.
dlsym() has an option that makes the Linux dynamic linker search for the right symbol to be override. This is the RTLD_NEXT flag, which is to be used just for the purpose of wrapper dynamic library functions.
libdl the task of returning the pointer to the right symbol. The RTLD_NEXT option to dlsym() returns the right symbol.
The next and final example of the use of LD_PRELOAD will still use the valiant ls. In time for Christmas, this will modify the output of ls by randomizing the d_type field returned in the dirent structure by readdir (3). If you use colorized ls output, and I believe most of you probably do, you should see a pretty display of color whenever you list a directory by preloading this function.

struct dirent64 *readdir64(DIR *dir) {     static struct dirent64 *(* libc_readdir64)(DIR *dir) = NULL;     struct dirent64 *dent;     unsigned char rnd_dtype[7] = { DT_UNKNOWN, DT_REG,                                    DT_DIR, DT_FIFO,                                    DT_SOCK, DT_CHR,                                    DT_BLK };     if (libc_readdir64 == NULL) {         *(void **)(&libc_readdir64) = dlsym(RTLD_NEXT, "readdir64");         srand(time(NULL));     }     dent = libc_readdir64(dir);     if (dent != NULL)         dent->d_type = rnd_dtype[rand() % 7];     return dent; }

There is still a problem with this code on my new Ubuntu Hardy machine. The code from the preloaded library hangs before the program terminates. I do not understand why this happen and a search for this bug did not turn up anything. The problem doesn’t happen with Ubuntu Karmic.
There is nothing new about using LD_PRELOAD this way. Several very nice libraries have been built with the intention of modifying the behavior of typical libraries.

• fakeroot: “fakeroot provides a fake root environment by means of LD_PRELOAD and SYSV IPC (or TCP) trickery.”
• fakechroot: fakechroot provides a fake chroot environment to programs.
• libtrash:“[...] the shared library which, when preloaded, implements a trash can under GNU/Linux”
• cowdancer: cowdancer is an userland implementation of copy-on-write filesystem.

There are 29 projects matching LD_PRELOAD on freshmeat.net. You might have used some of them.
The code I have written for this demonstration is available on BitBucket.

Written by fdgonthier
January 11th, 2010 at 10:10 pm

Posted in Debian,Linux,Programming,Tips and Tricks
Tagged with bitbucket, libdl, Linux, opendir, preload

reference:  http://www.lostwebsite.net/2010/01/ld_preload-fun/