So I’ve just come across this very cool feature in C++17 that I plan on using called ‘fold expressions’. This is basically a way of expanding a bunch of operations. It is also very hard for me to wrap my head around, so we’re going to go through a (very simple) example here, since people don’t seem to explain what exactly happens.

This simple example prints out everything that is passed into the method to stdout:

template <typename... argn>
void fold_test_types( argn... params ){
    (std::cout << ... << params);
}

int main(){
    fold_test_types( 3, 4, "hi", 8);
}

The output is 34hi8, because we forgot to add any newlines or spaces, but that’s fine for this example.

Now what really tripped me up here is the fact that in the fold expression, we’re using the << operator to distinguish things, and it makes it very confusing as to what is actually happening. With fold expressions, the data between the parentheses is expanded, kinda like a template. Let’s look at the rules real quick:

( pack op ... ) (1)
( ... op pack ) (2)
( pack op ... op init ) (3)
( init op ... op pack ) (4)

1) unary right fold
2) unary left fold
3) binary right fold
4) binary left fold

op – any of the following 32 binary operators: + – * / % ^ & | = < > << >> += -= *= /= %= ^= &= |= <<= >>= == != <= >= && || , .* ->*. In a binary fold, both ops must be the same.
pack – an expression that contains an unexpanded parameter pack and does not contain an operator with precedence lower than cast at the top level (formally, a cast-expression)
init – an expression that does not contain an unexpanded parameter pack and does not contain an operator with precedence lower than cast at the top level (formally, a cast-expression)
Note that the open and closing parentheses are part of the fold expression.

https://en.cppreference.com/w/cpp/language/fold

This is a bit hard to understand, but the … is basically a keyword when it is in the parentheses like this. ‘pack’ is the name of the parameters, e.g. in our example it is ‘params’.

Looking at these rules, the rule that we are following in this example is #4, the binary left fold. Let’s look at the fold expression again, with some comments:

(std::cout << ... << params);
|    |      |  |   |    |  ^----ending paren
|    |      |  |   |    ^---- pack(params) - see definition above
|    |      |  |   ^---- operator(<<)
|    |      |  ^---- ...(keyword?)
|    |      ^---- operator(<<)
|    ^--- init(std::cout) - see definition above
^----Starting paren   

With these broken out, it should now be clear why this is #4, the binary left hold. The rules on how it folds are as follows:

1) Unary right fold (E op …) becomes (E1 op (… op (EN-1 op EN)))
2) Unary left fold (… op E) becomes (((E1 op E2) op …) op EN)
3) Binary right fold (E op … op I) becomes (E1 op (… op (EN−1 op (EN op I))))
4) Binary left fold (I op … op E) becomes ((((I op E1) op E2) op …) op EN)

https://en.cppreference.com/w/cpp/language/fold

Since we are using #4, apply the rule for #4. When the compiler runs, the code effectively becomes the following with the template substitution and the fold expression expansion:

void fold_test_types( int arg1, int arg2, std::string arg3, int arg4 ){
    ((((std::cout << arg1) << arg2) << arg3) << arg4);
}

int main(){
    fold_test_types( 3, 4, "hi", 8);
}

As you may be aware, I maintain dbus-cxx, and I’ve been working on it lately to get it ready for a new release. Most of that work is not adding new features, but updating the code generator to work correctly. However, this post today is not about that work, it is about the work that I am doing on the next major version of dbus-cxx(2.0). Part of this work involves using new C++ features, due to libsigc++ needing C++17 to compile now. With the new variadic templates that C++ has(actually since C++11), we can have more than 7 template parameters to a function. (This limit of 7 is arbitrarily chosen by libsigc++, it’s not a C++ limitation.)

Because of this however, some of the code in dbus-cxx needs to change in order to work correctly. The main portion that I’m working on right now has to do with getting the DBus signatures to work properly. Here’s a small piece of code that is currently in dbus_signal.h(after running dbus_signal.h.m4 through m4):

  /** Returns a DBus XML description of this interface */
  virtual std::string introspect(int space_depth=0) const
  {
    std::ostringstream sout;
    std::string spaces;
    for (int i=0; i < space_depth; i++ ) spaces += " ";
    sout << spaces << "<signal name=\"" << name() << "\">\n";

    T_arg1 arg1;
    sout << spaces << "  <arg name=\"" << m_arg_names[1-1] << "\" type=\"" << signature(arg1) << "\"/>\n";
    T_arg2 arg2;
    sout << spaces << "  <arg name=\"" << m_arg_names[2-1] << "\" type=\"" << signature(arg2) << "\"/>\n";
    sout << spaces << "</signal>\n";
    return sout.str();
  }

This method is created once for each overloaded type that we have. The important part is that T_arg is created once for each argument that we have. With variadic templates, this is impossible to do. The way to get the template arguments out of the variadic call is to do recursion.

Recursion + templates is not something that I’m very familiar with, so this took me a while to figure out. However, I present the following sample code for getting the signature of a DBus method:

  inline std::string signature( uint8_t )     { return "y"; }
  inline std::string signature( bool )        { return "b"; }
  inline std::string signature( int16_t )     { return "n"; }
  inline std::string signature( uint16_t )    { return "q"; }
  inline std::string signature( int32_t )     { return "i"; }
  inline std::string signature( uint32_t )    { return "u"; }

  template<typename... argn> class sig;

   template<> class sig<>{
   public:
   std::string sigg() const {
     return "";
   }
   };

   template<typename arg1, typename... argn>
   class sig<arg1, argn...> : public sig<argn...> {
   public:
   std::string sigg() const{
     arg1 arg;
     return signature(arg) + sig<argn...>::sigg();
   }
   };

int main(int argc, char** argv){
  std::cout << sig<uint32_t,uint32_t,bool,int64_t>().sigg() << std::endl;
}

Output: 
uubx

This took me a few hours to figure out exactly how to do it, so I’m at least a little proud of it! The other confusing part that I had to work out was how to use a recursive call(with signature()) also with the recursive call for the templates, which leads us to the following observation:

It’s recursion all the way down.

Earlier this week, I came across this question on softwareengineering.stackexchange.com.  This has some relevance for me, since at work two of our main projects follow the two sides of this design philosophy: one project is a more monolithic application, and the other follows more of a microservices model(e.g. many applications).  There are some reasons for this which I will now attempt to explain.

Option 1: Monolithic Application

The first project that I will explain here is our project that has a more monolithic application.  First of all, a brief overview of how this project works.  Custom hardware(running Linux) collects information from sensors(both built-in and often third-party over Modbus) and aggregates it and classifies it.  The classification is due to the nature of the project – sensor data falls into one of several classes(temperature, voltage, etc.).  This information is saved off periodically to a local database, and then synchronized with a custom website for viewing.  The project, for the most part, can be split into these two main parts:

• Data collector
• Web viewer
• Local viewer(separate system, talks over ethernet)

Due to the nature of the hardware, there is no web interface on the hardware directly, it is on a cloud server.

Now, the data collector application is a mostly monolithic application.  However, it is structured similarly to the Linux kernel in that we have specific ‘drivers’ that talk with different pieces of equipment, so the core parts of the application don’t know what hardware they are talking to, they are just talking with specific interfaces that we have defined.

In this case, why did we choose to go with a monolithic application?  Well, there are a few reasons and advantages.

Reason 1: As primarily a data collector device, there’s no real need to have different applications send data to each other.

Reason 2: The development of the system is much easier, since you don’t need to debug interactions between different programs.

Reason 3: Following from the first two, we often have a need to talk with multiple devices on the same serial link using Modbus.  This has to be siphoned in through a single point of entry to avoid contention on the bus, since you can only have one modbus message in-flight at a time.

Reason 4: All of the data comes in on one processor, there is no need to talk with another processor.  Note that this is not the same as talking with other devices.

Reason 5: It’s a lot simpler to pass data around and think about it conceptually when it is all in the same process.

Now that we have some reasons, what are some disadvantages to this scheme?

Disadvantage 1: Bugs.  Since our application is in C++(the ability to use C libraries is important), a single segfault can crash the entire application.

Disadvantage 2: The build can take a long time; the incremental build and linking isn’t bad, but a clean build can take a few minutes.  A test build on Jenkins will take >10 minutes, and it can still take several minutes to compile on a dev machine if you don’t do parallel make.

Overall, the disadvantages are not show-stoppers(except for number 1, there is some bad memory management happening somewhere but I haven’t figured out where yet).  The separation into three basic parts(data collection, local GUI, web GUI) gives us a good separation of concerns.  We do blend in a little bit of option 2 with multiple applications, but that is to allow certain core functionality to function even if the main application is down – what we use that for is to talk with our local cell modem.  Given that the data collection hardware may not be easily accessible, ensuring that the cellular communications are free from bugs in our main application is important.

Option 2: Multiple Applications

If you don’t want to make a monolithic application, you may decide to do a lot of small applications.  One of my other primary projects uses this approach, and the reason is due to the nature of the hardware and how things need to interact.

In our project with multiple applications, we have both multiple compute units and very disparate sensor readings that we are taking in.  Unlike the monolithic application where data is easily classified into categories, this project has even more disparate data.  Moreover, we take in a lot of different kinds of data.  This data can come in on any processor, so there is no ‘master’ application per se.  This data also needs to be replicated to all displays, which may(or may not) be smart displays.  We also want to insulate ourselves from failure in any one application.  A single bug should not take down the entire system.

To handle this, we essentially have a common data bus that connects all of the processors together.  We don’t use RabbitMQ, but the concept is similar to their federation plugin, in that you can publish a message on any processor and it will be replicated to all connected processors.  This makes adding new processors extremely easy.  All of the data is basically made on a producer/consumer model.

Advantage 1: Program resiliency.  With multiple applications running, a bug in one application will not cause the others to exit.

Advantage 2: You can easily add more processors.  This is not really a problem for us, but since data is automatically synchronized between processors, adding a new consumer of data becomes very simple.

Advantage 3: Data can come and go from any connected system, you need not know in advance which processor is giving out information.

This design is not without some caveats though.

Disadvantage 1: Debugging becomes much harder.  Since you can have more than one processor in the system, your producer and your consumer can be on different processors, or you could have multiple consumers.

Disadvantage 2: Because this is a producer/consumer system(it’s the only way that I can see to effectively scale), there’s no way to get data directly from an application(e.g. there’s no remote procedure call easily possible over the network).

Conclusion

There are two very different use cases for these two designs.  From my experience, here’s a quick rundown:

Monolithic Application

• Generally easier to develop, since you don’t have to figure out program<->program interactions
• Often needed if you need to control access to a resource(e.g. physical serial port)
• Works best if you only have to run on one computer at a time

Multiple Applications

• Harder to develop due to program<->program interactions
• Better at scaling across multiple computers
• Individual applications are generally simpler

Due to the nature of engineering, there’s no one way to do this that is best.  There are often multiple ways to solve a given problem, and very rarely is one of them unequivocally the best solution.

So a few years ago, me and a coworker had to figure out how many lines of code we had. This was either for metrics or for because we were curious, I can’t really remember why. I came across the script again today while going through some old code. Here it is in all its glory:

#!/bin/bash

let n=0; for x in "$@"; do temp1=`find $x | grep '\\.cpp$\|\\.c$\|\\.java$\|\\.cc$\|\\.h$\|\\.xml$\|\\.sh$\|\\.pl$\|\\.bash$\|\\.proto$'`; temp=`cat /dev/null $temp1 | grep -c [.]*`; let n=$n+$temp; if [ $temp -gt 0 ]; then printf "%s: " $x ; echo $temp; fi; done ; echo Total: $n

This took us at least an hour(or perhaps more), I’m not really sure.

Anyway, it was after we had done all of that work that we realized that wc exists.