↓Skip to main content

Implementing Basic RPC with C++20

study rpc c++ template
Table of Contents

This post explains how a basic RPC framework can be implemented by using modern C++ functionalities. The explanation in this post is heavily inspired from:

C++ Features used #

Parameter Pack 1 #

Parameter pack is similar to C variadic arguments that are used in printf() family:

int printf(char *format, ...);

which is implemented with va_ variadic function API [src]:

int printf(const char* format, ...) {
    va_list arg;
    int done;

    va_start(arg, format);
    done = vfprintf(stdout, format, arg);
    va_end(arg);

    return done;
}

Refer to this article to see more about va_list.

But parameter pack is used to specify zero or more template arguments:

template <typename... Types> struct Tuple {};
Tuple<> t0;
Tuple<int> t1;
Tuple<int, float> t2;

where all t0, t1, t2 are valid.

Parameter pack can be used to specify arbitrary number of function arguments. As types can be various, it is suitable to represent arguments altogether in one pack. Therefore, the following template function wrapperFunction can be instantiated with any function type:

template <typename ReturnType, typename... Args>
void wrapperFunction(std::function<R(Args...)>&& func) { ... }

int test1(int a, double b);    wrapperFunction(test1); // ReturnType: int, Args: <int, double>
std::string test2();           wrapperFunction(test2); // ReturnType: std::string, Args: None
std::vector<int> test3(int a); wrapperFunction(test3); // ReturnType: std::vector<int>, Args: <int>

Pack Expansion 1 #

This is what I could not undestand well. The reference says:

A pattern followed by an ellipsis, in which the name of at least one parameter pack appears at least once, is expanded into zero or more comma-separated instantiations of the pattern, where the name of the parameter pack is replaced by each of the elements from the pack, in order.

template <typename... Us> void f(Us... pargs) {}
template <typename... Ts> void g(Ts... args) {
    f(&args...);
}
g(1, 0.2, "a");

Here, &args... is a pack expansion (a pattern &args followed by an ellipsis). Therefore, the pattern is expanded to each of the elements from the pack. In other words, Ts... expand to int E1, double E2, const char* E3, and &args... expands to &E1, &E2, &E3 (reference marks are expanded to each of the elements), which makes Us... be deduced as int*, double*, const char**.

This pack expansion can be used to feed arguments with std::integer_sequence explained right below.

Fold Expression 2 #

This reduces or folds a parameter pack over a binary operator. This is also quite hard to understand, but it can be used to apply the same operator to all parameters in a pack.

template <typename... Args>
bool all(Args... args) { return (... && args); }
bool b = all(true, true, true, false);

This example from the reference uses unary left fold (... op pack) in return (... && args); statement. This expands the expression as: (((args1 && args2) && args3) && args4), which makes b false since args4 is given as false.

std::integer_sequence 3 #

The class template represents a compile-time sequence of integers. The reference also indicates that: when used as an argument to a function template, the parameter pack can be deduced and used in pack expanson.

template <typename T, T... ints>
void print_sequence(std::integer_sequence<T, ints...> int_seq) {
    std::cout << "The sequence of size " << int_seq.size() << ": ";
    ((std::cout << ints << ' ' ), ...);
    std::cout << "\n";
}

print_sequence(std::integer_sequence<unsigned, 9, 2, 5>{});
-> The sequence of size 3: 9 2 5

Here, fold expression is used in ((std::cout << ints << ' '), ...);. ints is a parameter pack, and , is an operator. This unary left fold expands the statements to std::cout << int1 << ' ' ints2 << ' ' ints3 << ' ' (check this question for more detail about expanding with , operator).

This integer sequence is used to specify each argument’s index in a tuple, with std::tuple_size constant expression:

template <typename Func, typename Tuple, std::size_t... I>
auto invoke(Func&& func, Tuple&& tuple, std::index_sequence<I...>) {
    return func(std::get<I>(std::forward<Tuple>(tuple))...);
}

using ArgsTuple = std::tuple<typename std::decay_t<std::remove_reference_t<Args>>...>;
auto tuple = std::make_tuple<ArgsTuple>(/* ... */);
invoke(function, tuple, std::make_index_sequence<std::tuple_size_v<ArgsTuple>>{});

where Tuple can be any types of tuple (note that tuple is a template class that accepts arbitrary types of data, e.g. std::tuple<int, double, std::string>). std::make_index_sequence<N> generates a integer sequence starting from 0, and it is used to get specific items from the tuple with std::get<I>(tuple). This pattern is expanded with a following ellipsis to all parameter pack in I, the size of which is the same to the size of tuple type ArgsTuple.

Implementing RPC #

Basic mechanism of RPC (Remote Procedure Call) is as follows.

  1. Map functions with unique function names.
  2. Serialize function name and arguments that are used to execute function in remote side
  3. Transfer serialized data to the remote node
  4. In the remote side, deserialize data.
  5. Execute requested function with deserialized function arguments.

Serialization #

We first serialize arbitrary number of arguments into byte stream. For this purpose, std::tuple is used as a temporary container of arguments, since STL containers are template classes so we could utilize template functionalities such as pack expansion, fold expression. The type of tuple can be defined with pack expansion:

template <typename... Args>
std::vector<uint8_t> serialize(const std::string& function_name, Args&&... arguments) {
    using ArgsTuple = std::tuple<typename std::decay_t<std::remove_reference_v<Args>>...>;
    ArgsTuple tuple = std::make_tuple(arguments...);
}

Assume that Args... is <int, int, double>, then ArgsTuple is expanded as std::tuple<int, int, double> thanks to pack expansion, which can contain every arguments in one container.

Now we serialize each item in the tuple container by using fold expression and std::integer_sequence:

template <typename... Args>
std::vector<uint8_t> serialize(/* omitted */) {
    ...
    msgpack::Packer packer{};
    [&]<typename Tuple, std::size_t... I>(Tuple&& tuple, std::index_sequence<I...>) {
        (packer(std::get<I>(tuple)), ...);
    } (std::forward<ArgsTuple>(tuple),
       std::make_index_sequence<std::tuple_size_v<std::remove_reference_t<ArgsTuple>>>{});
}

Here, I used a new feature in C++20: lambdas can have a list of template parameters. Before C++20, it was impossible so that another function (usually named as _impl) is required:

template<typename... Args>
std::vector<uint8_t> serialize(/* omitted */) {
  ...
  msgpack::Packer packer{};
  serialize_impl(packer,
                 std::forward<ArgsTuple>(tuple),
                 std::make_index_sequence<std::tuple_size_v<std::remove_reference_t<ArgsTuple>>>{});
}

template<typename Tuple, std::size_t... I>
void serialize_impl(msgpack::Packer& packer, Tuple&& tuple, std::index_sequence<I...>) {
  (packer(std::get<I>(tuple)), ...);
  ...    
}

For more information, refer to this article 4.

To feed each argument into the serializer (named packer), fold expression is used: (packer(std::get<I>(tuple)), ...);; for each index I (0 to # of arguments - 1), it calls operator() of packer to feed the corresponding argument. It may look different in terms of serializer; I used cppack, a msgpack implementation, and it uses operator() to de/serialize, like cereal.

Deserialization #

Deserialization is more complex, since we need a function signature and extract arguments from the given byte stream in the exact same order.

First, I registered functions to be called with the following register function:

template <typename R, typename... Args>
void callProxy(std::function<R(Args...)>&& func,
               msgpack::Unpacker& unpacker) {
    ...
}

std::map<std::string, std::function<void(msgpack::Unpacker&)>> functions;

template <typename R, typename... Args>
bool registerFunction(const std::string& function_name,
                      R (*function)(Args...)) {
}
    functions[function_name] = std::bind(&callProxy, function, std::placeholders::_1);

int testFunction(int a, int b) { ... }
registerFunction("testFunction", testFunction);

It is quite tricky, that it is callProxy function that is stored in functions map, not functions given by users. This is because of various function signatures; function map cannot store various signatures in one map.

For example, we might want to implement a template map:

template <typename ReturnType, typename... Args>
std::map<std::string, std::function<ReturnType(Args...)>> functions;

This implementation actually creates several maps when we register functions, so if we use registerFunction twice like:

int test1(int a, int b);
double test2(int a, double b);
registerFunction("test1", test1);
registerFunction("test2", test2);

Actually two maps are created during runtime: std::map<std::string, std::function<double(int, double)>> and std::map<std::string, std::function<int(int, int)>>. To prevent such problem, we use callProxy function.

We use std::bind to bind user’s function into callProxy. The second argument of std::bind is fed to the first argument of callProxy, so when a bound function is called, it does not have to be given again. We have to pass msgpack::Unpacker& since only a placeholder is bound for this place.

Later when we call the function, the code would be:

msgpack::Unpacker unpacker(data);
auto func = functions[function_name]; // func must be callProxy with bound user's function.
func(unpacker);

Now, we can call callProxy function with its name. We have to extract arguments from msgpack::Unpacker to call user function. Since callProxy already has typename... Args parameter pack, we could apply similar strategy that we used for serialization:

template <typename R, typename... Args>
void callProxy(std::function<R(Args...)>&& func,
               msgpack::Unpacker& unpacker) {
    using ArgsTuple = std::tuple<typename std::decay_t<std::remove_reference_t<Args>>...>;
    ArgsTuple tuple;
    R result = [&]<typename Tuple, std::size_t... I>
               (Tuple&& tuple, std::index_sequence<I...>) -> R {
        (unpacker(std::get<I>(tuple)), ...);
        return func(std::get<I>(std::forward<Tuple>(tuple))...);
    }(std::forward<ArgsTuple>(tuple), 
      std::make_index_sequence<std::tuple_size_v<ArgsTuple>>{});
    // Send result back
}

By implementing a type ArgsTuple with the parameter pack Args..., we can ask deserializer (unpacker) to extract each argument from the byte stream with fold expression: (unpacker(std::get<I>(tuple)), ...);. Note that std::get<I>(tuple) returns a reference, so it can be used to set the value into the tuple.

Then we can feed arguments in the tuple into the function by using pack expansion, and assuming we provide correct byte stream, it should work.

  1. CPPreference: Parameter Pack (Since C++11) ↩︎

  2. CPPreference: Fold Expression (Since C++17) ↩︎

  3. CPPreference: Integer Sequence (Since C++14) ↩︎

  4. std::index_sequence and its Improvement in C++20 ↩︎

↑