Making a Tool of Deception

"I wonder if I can...?” are dangerous words. They often lead to disappointment, occasionally to commitment, and almost always to spending time.

In this particular case I wondered if I could use lambdas and other modern C++ features to make powerful mocking constructs easy to use. It turned out that this time I hit all of the three consequences of my question, and the result is the Trompeloeil mocking framework [ trompeloeil ].

My unit-testing experience is heavily coloured by google-mock [ gmock ]. While I have tinkered with other mocking frameworks, [ gmock ] is the one I have a working experience with. This has undoubtedly influenced my view on how mocks are used.

The issues I wanted to address may sound like [ gmock ] bashing, but that would be unfair – the extra expressive power that C++11/14 gives over C++98 makes a huge difference. My list of desired features are:

1. Match parameter values as boolean expressions inline in expectations.
2. Write side effects for matched expectations inline as any statement.
3. Express return values for matched expectations inline as expressions.
4. Allow wild cards for “don’t care” values in expectations, even for overloaded functions
5. Easily understood lifetime of objects used in expectations.
6. Control of lifetime of a mock object that may be destroyed by the test subject.
7. Implementation a in single header file.
8. Compilation errors from simple mistakes (like forgetting return in a non-void function.)
9. Rely less on the preprocessor than [ gmock ] does.
10. Shorter compilation times compared to [ gmock ].

This article tries to explain how [ trompeloeil ] is made, although all solutions listed here are simplified to save space and keep focus on the important bits.

Mock implementations

The syntax chosen for defining and placing expectations on mocks is similar to that of [ gmock ]. Listing 1 shows the definition of a class with two mocked functions, and Listing 2 shows how expectations are placed on an instance.

class Mock
{
public:
 MAKE_MOCK1(foo, void(std::string));
 MAKE_MOCK1(foo, bool(int));
};
			

Mock obj;
REQUIRE_CALL(obj, foo("cat"));
REQUIRE_CALL(obj, foo(ANY(int))
  .RETURN(_1 > 0);
			

The first problem to solve is that the mock implementation of a member function must search for matching expectations, and this must also work when the signature types don’t match perfectly. In Listing 2 "cat" is not a std::string for the first expectation, but it is equal-comparable to one, and the wild card in the second expectation must only match the int overload.

The chosen implementation is that MAKE_MOCKn() adds a list of expectations as a member variable, and REQUIRE_CALL() creates an expectation object that is added to the list, which leaves the problem of knowing the type of the expectation object. A simplified, slightly pseudocoded, version of this logic implemented by the MAKE_MOCKn() macro, is shown in Listing 3, where PARAMS(num, sig) creates a parameter list of num parameters from the signature sig , and LINEID(name) appends the current line number to the name.

template <typename sig, typename ... U>
auto make_call_matcher(U&& ... u)
{
  using std::forward;
  using std::make_tuple;
  using param_t =
    decltype(make_tuple(forward<U>(u)...));
  using matcher = call_matcher<sig, param_t>;
  return new matcher(forward<U>(u)...);
}
#define MAKE_MOCK_NUM(num, name, sig)           \
  struct LINEID(tag_type)                       \
  {                                             \
    template <typename ... U>                   \
    static auto name(U&& ... u)                 \
    {                                           \
      return make_matcher<sig>                  \
      (std::forward<U>(u)...);                  \
    }                                           \
  };                                            \
  LINEID(tag_type) tag_                         \
    ## name(PARAMS(num, sig));
#define MAKE_MOCK1(name, sig)                   \
MAKE_MOCK_NUM(1, name, sig)
			

The idea is that REQUIRE_CALL(obj, func(params)) , can use decltype(obj.tag_func(params)) to get the tag_type , and from there call the static member function to create the matcher object. This works even if the type isn’t a perfect match, like using a c-string literal for a std::string parameter, and for the wild card, which can convert to the desired type when finding the tag, and compares equal to any value of the desired type. This takes care of item 4 in the list of desired features.

The logic for finding the list of expectations that the matcher object adds itself to is similar.

Matches and actions

The expectation object created by REQUIRE_CALL() is an instance of the template call_matcher<Sig, Value> , and is basically a simple struct containing additional conditions, side effects and a return handler. Listing 4 shows a simplified version.

template <typename Sig, typename Value>
struct call_matcher
{
  template <typename ... U>
  call_matcher(U&& u) :
    value(std::forward<U>(u)...) {}
  template <typename R>
  call_matcher& set_return(R&& r) {
    return_handler = std::forward<R>(r);
    return *this;
  }
  std::list<condition<Sig>>      conditions;
  std::list<side_effect<Sig>>    actions;
  std::function<return_handler_sig<Sig>>
                                 return_handler;
  Value                          value;
}
			

In call_matcher<Sig, Value> , Sig is the signature of the mocked member function, and Value is a tuple containing copies of all values given in the parameter list to the function in REQUIRE_CALL() .

In addition, condition<Sig> is std::function<bool(const param_tuple&)> , and side_effect<Sig> is std::function<void(param_tuple&)> , where param_tuple is a std::tuple<> with references to all parameters given in the call.

The mock implementation of a member function creates a param_tuple instance, and searches the list of call_matchers , checking first if value matches, and if it does, if all condition s match. If no match is found, a violation is reported. If a match is found, all action s are called and finally the result of calling the return_handler is returned.

If you look at the RETURN() in Listing 2, you see that the parameter is referred to as _1 . RETURN is a macro, with a shortened implementation in Listing 5.

#define RETURN(...)           \
 set_return([=](auto& x) {    \
 auto& _1 = mkarg<1>(x);      \
 auto& _2 = mkarg<2>(x);      \
 ignore(_1, _2);              \
 return __VA_ARGS__;          \
 })
			

The lambda parameter x becomes a reference to the param_tuple instance mentioned above, and mkarg<n>(x) returns the reference held by the tuple if n is a legal index, or an instance of illegal_parameter<n> otherwise. The latter ensures compilation errors if you accidentally refer to something that doesn’t exist. ignore() is a simple empty function that prevents compiler warnings for unused local variables. The use of auto in the parameter list for the lambda is the only construction in [ trompeloeil ] that requires C++14, in other places C++14 offers a convenience over C++11, but is not strictly needed. Extra conditions are handled similarly using a WITH() macro, and actions using a SIDE_EFFECT() macro. This construction takes care of items 1, 2, and 3 in the feature list. It also solves item 5, easily understood lifetimes of objects used. Any value given directly in the parameter list to REQUIRE_CALL() is copied and lives as long as the expectation object does. Any value in RETURN() , SIDE_EFFECT() and WITH() are copied/moved, and the lifetime ends when the expectation object is destroyed. There are also versions of the latter 3 macros, LR_RETURN() , LR_SIDE_EFFECT() and LR_WITH() , which use a reference capture for the lambda (LR for Local Reference, not an ideal name, but it works.)

If it will fail, fail immediately

The solution outlined above works, but it is not very friendly to an error prone developer using it. Forgetting a RETURN() in a non-void member function gives a run time error. A RETURN() with wrong type gives the all too familiar C++ template error vomit, and somehow squeezing in RETURN() several times uses only the last one added.

In order to provide better error pinpointing, REQUIRE_CALL() does not only instantiate a call_matcher template, it also instantiates a call_modifier template that operates on the call_matcher . A simplified call_modifier template is shown in Listing 6. The call_modifier template is instantiated with the type of the call_matcher , and matcher_info of the function signature. The helper return_of_t<> , is a simple template alias of the return type from a function signature.

template <typename Sig>
struct matcher_info
{
  using signature = Sig;
  using return_type = void;
};

template <typename RetType, typename Parent>
struct return_injector : Parent
{
  using return_type = RetType;
};

template <typename Matcher, typename Parent>
struct call_modifier : public Parent
{
  using typename Parent::signature;
  using typename Parent::return_type;

  template <typename H>
  auto set_return(H&& h)
  -> call_modifier<Matcher,
       return_injector<
       return_of_t<signature>,
       Parent
     >
  {
    using namespace std;
    using h_rt =
      decltype(h(declval<param_tuple>());
    using rt = return_of_t<signature>;
    static_assert(
      is_constructible<rt, h_rt>::value
      || !is_same<rt, void>::value,
      "RETURN for void function");
    static_assert(
      is_constructible<rt, h_rt>::value ||
      || is_same<rt, void>::value,
      "RETURN wrong type for function");
    static_assert(
      is_same<return_type, void>::value,
      "Multiple RETURN");
    matcher.set_return(forward<H>(h));
    return {matcher}
  }
			

This technique of using a template inhering stepwise modifications of known types as a trampoline for the actual work works very well for providing good error messages. static_assert is often messy because compilation doesn’t stop at failure, and the intended message is lost in loads of other messages. This technique, however, limits the mess substantially and typically provides good feedback that is not hidden in a long list of irrelevant problems. The conditions for each static_assert are stricter than necessary to avoid tripping several of them.

This takes care of item 8 in the list of desired features, good compilation errors for simple mistakes, but it does so at a compile time cost.

Now and then

I’m rather pleased with where this has come so far. Mocking with [ trompeloeil ] is easy, with very readable test code due to inline expressions in the expectations, and it is easy to understand the lifetimes of objects. Most error messages are good, but more work can be done there. In this article I have not shown how lifetime expectations are controlled, nor how you can decide which expectations must be met in sequence, and which are unrelated to each other, but those too are easily expressed.

Compilation time and binary size are disappointing. Better than [ gmock ], but only by a narrow margin, and the the frivolous reliance on the preprocessor feels like a failure.

Going forward, I really want to address the compilation times. Faster than [ gmock ] means it’s within what people accept, but I think it’s too slow for good edit-build-run TDD cycles.

I would also like to have a better MAKE_MOCK() macro, which doesn’t need the number of arguments explicitly, since that is a recurring source of unhelpful errors.

An amazingly cool feature would be if parameters could be referenced in expectations by their names, instead of positional identities.

If you have ideas for advancing [ trompeloeil ] further, please get in touch.

References

[gmock] https://code.google.com/p/googlemock/

[trompeloeil] https://github.com/rollbear/trompeloeil