Nicolás Brailovsky

Ohh design patterns. I love buzzwords. Who doesn’t? To increase my buzzword count I will be writing about a topic most people programming C++ should already know: RAII, resource acquisition is initialization.

Patterns everywhere

Knowing that Gamma et al didn’t list all the known patterns in the universe does come as a surprise to some, not sure why though. The twenty some patterns they write about in their now famous book are (arguably, perhaps) some of the most common design patterns, but the list hardly finishes there.

Some patterns only have meaning in a very specific context; a reactor is a nice pattern for handling asynchronous events yet most applications would never need it. Sometimes “the context” means a specific domain in which the application must work, like a web application, a real time application or a distributed application, sometimes the context is the language itself. RAII falls in this last domain, it only makes sense for C++ applications (actually there are others, but thinking what kind of languages would support this idiom is left as an exercise for the reader).

No, really. What is RAII?

If you made it past that long intro you are probably really interested in knowing what RAII is, and don’t know how to search it in Wikipedia. OK, I’ll explain it the best I can then: RAII means that a resource acquisition is the initialization.

Seriously. That is all the secret there is to RAII. Talking in code:

An example

Compare that to a visitor pattern. It’s just too simple to be of any use, isn’t it? Well, contrary to what Java fanboys tend to believe you can do lots of nice things without writing a bazillion lines of code.

Wow… a whole lot of code just for calling bar and baz. And as I wrote that without even compiling I’m sure there are too many hidden bugs, lots of code-paths my simple programmer’s mind can’t even begin to imagine which will cause my Expensive_Resource to be leaked!

Let’s rewrite that using RAII:

A lot nicer, isn’t it? But, what happened to all the try/catch if’s and closes there?

Where’s the magic?

The magic of RAII lies in how C++ handles exceptions. When we have a built object (can an object be in an unbuilt state?) it means it’s constructor has correctly ran. It also means it’s destructor will run when it goes out of scope, doesn’t mater HOW it goes out of scope.

See how brilant that is? Doesn’t matter if a function we’re calling throws, or if we need to return before reaching the end of the function: the destructor will be called and thus our Expensive_Resource will be free!

Why is this C++ specific?

This is an easy one: think how would you implement this in Java: right, you can’t. Not knowing when is your object going to be destroyed means you can’t do anything useful in its destructor, therefore RAII is deeply rooted within the memory management in C++ and it’s pretty much a language specific pattern (or is it?). But, is that so good?

Not everything is so great…

You are probably thinking this is the best discovery since ice cream was invented. Well, not so fast, RAII has it’s detractors too.

The first problem in RAII, it doesn’t have a graceful way of handling resource acquisition failure. If Expensive_Resource is a database, and it’s connection fails, we have a throwing constructor.

Even if throwing constructors are acceptable, throwing destructors may even be a worst idea: throwing while an exception is already active is a cause for concern (tip: it’ll crash your application, doesn’t matter how many try/catch blocks you use, due to stack unwinding issues).

And then, even if we don’t care about throwing destructors you have the issue of a release failure: how do you notify the user that a release failure has occurred? And what do you do, should it happen?

So, RAII is a great idea indeed, and it has it’s uses, but you should be careful when choosing this C++ specific pattern.

Wow, number eleven already. We’re getting more chapters here than Final Fantasy games. I didn’t even imagine there was so much to write about such an esoteric language features like templates. I do wonder if anyone will actually read it, but that’s a completly different problem.

Enough meta-meta talk: what can we do with all the things we have learned? We can calculate pi and e, we already showed that as an example on one of the first chapters. This chapter I’m going to write about what motivated me to explore the bizarre underworld of template metaprogramming. Some time ago I had to work with a Berkeley DB researching the feasibility of developing a magic cache for (real) DB table. Leaving aside the discussion of whether this is a good idea (the project did have a good reason to be researched) I hit a major roadblock when trying to provide a façde for every table; something like this: See the problem? To do something like that we’d need a virtual template method, and you can’t have that. After seeing that I thought to myself “Hey, I’ll use templates!”. Then I had two problems, but the damage was done, I couldn’t go back. What kind of contorted device could we implement to make such a devious requirement work? I’ll leave you to think it, the answers I came up with next week.

There are two well known macros from the preprocessor which every macro-sorcer must know. They are __FILE__ and __LINE__. You probably already know about them but anyway, __FILE__ will give you the current file and __LINE__ the current line. Easy, huh?

int main() {    printf("%s : %i", __FILE__, __LINE__);    return 0; }

The program above would give you “main.cpp : 3″ as a result. There is nothing going on at execution time, it’s all preprocesor wizardy. In fact with “g{++/cc} -E” you can even check what the “real” output is (-E means to return the preprocessor output. Keep in mind a lot of stuff will be included from the headers you use).

Well that’s nice and all, but g++ can top this easily:

There are a couple of notable things about this new “pretty function” thing:

• 1. It will demangle a function’s name
• 2. This time it isn’t a preprocessor secret thing but a real variable g++ will create.

You can easily use this for better logging functions now (with some macro wizardy, obviously).

So far we’ve learned the basic constructs of template metaprogramming (loops, branching, return values) and some basic list operations (getting the length of a list, appending and preppending elements, checking if an element is included in a list). Let’s put it all together by creating an operation to return the position of an element. It’ll be very useful later on too.

If we go back to the Includes operation we can get some help to define the Position operation: the position of an element in a list is one plus the position of the element we’re searching for in the tail, or zero if the head equals said element. The operation is not defined if the element is not in the list.

Translating to pseudo-code:

The translation to C++ is not so trivial this time. Try it, I’ll wait… ready? OK, let’s start

Looks easy… but doesn’t work. First problem, we can’t compare two types, remember? We need to use Eq<X, Y> again. Second problem, although we said the operation is undefined if the element is not included on the list, it would be nice if we could force the compiler to fail if (or when) that happens. Let’s rewrite the operation using a façade again, but adding an Assert:

Oh, we haven’t defined assert yet! There’s another problem, too: even if it won’t compile, the compiler will try to expand _Position< …, NIL > indefinately, causing an error after too many nested template calls. Not nice. We need to add a case to make the compiler stop:

All that code for such a simple operation, man. Also, see what we did with Assert<>? It seems making a compile fail is actually quite easy. That’s what I have most experience with.

We’ve been through quite a lot, and our toolboox should be quite big already. Next time we’ll start steering towards some sort of aplicability, trying to use some of all these stuff to implement a real, useful and working program… assuming that’s even posible.

By now we should have learned how to perform loops, branching and returns using templates. Let’s add a couple of useful operations to our library: append and prepend.

Prepending an element to a list is very easy: the result is a list (oh surprise) consisting of a head (the element we want to add) and a tail (the old list). In the pseudocode I’ve been using so far:

And in C++ (trivial, this time):

Appending is a little bit more difficult, as we need to first find the end of the list. Think for a second how would you define it… back? Ok, I’d define it this way: appending an element to the list yields a list, consisting of the same head and the result of appending said element to the tail. The null case, as usual, is appending an element to a NIL list; in this case the result is a list with the element itself. So:

Looks complicated what it follows the same structure as the rest of the basic-ops:

Easy. Now, what happens if we want to add a default value for Lst, so we can use Append to create lists? Easy too, but we need a façade this time; just rename Append to _Append, then

I promised to add one more operation to our toolbox, returning the position of an element, but this post is getting quite long and I’m afraid it may be too much for the average attention span of a programmer… we’ll leave it for next time.

Remember last time? We learned how to get the lenght of a list. This time I’ll introduce some more of these basic ops. Let’s begin with “Nth”: getting the Nth element of a list; which, remember, in this case is a type, not a concrete element. This means the Nth element will be something like int, char, const char*, not 1, 2 or 3. We introduced a trick to get around this limitation before using a template <int>, go there to refresh your memory if needed.

So, what would the coloquial definition of “Nth” be? I’d put it like “The operation Nth for a list equals the head of the list for N = 0 and Nth (minus one) of tha tail otherwise”. A little bit more formally:

Translating this to C++ should be a breeze to you now. Try it, I’ll wait. Read? OK, this is MY answer:

Though the structure is very similar to the previous “basic operation”, getting the length of a list, the concept is quite different. This time we’re defining a return type recursivly. Anyway, it was too easy indeed, let’s try a more complex operation now.

How can we check if an element exists on a list? Seems easy enough, an element is included in a list if the head equals the element itself or if the element is included in the tail. In the pseudo language I just invented:

Looks easy, right? Well, there’s a bug there, can you spot it? Yeah, we’re missing the false condition. We should add a third spetialization:

Again, let’s translate the pseudocode to C++. Try it, I’ll wait. Read? OK, this is MY answer:

Looks nice, doesn’t it? Too bad it won’t work, you can’t compare two types. What would (int == char) mean in C++? We need a helper there, some kind of trick to compare to types. We can use partial template spetialization again:

With this little struct now we can write our include operation this way:

Very esoteric looking, the right mix of Haskel, C++ and booze to ensure job security for life. Next time we’ll find a way to search for the position of an element, a somewhat more complicated task.

* Post from the wayback machine. I wrote this a long time ago but it got way down the posts queue, don’t know why *

I liked my vacations very much, thank you. Some people enjoyed vacations from me too. At work they even decided to keep until my return. Upon my arrival a nice coredump was waiting at my desk, so to speak. Check it out, isn’t it beautiful?

Yeah, that’s getting killed inside zlib. Nice way to start the year, a bug in zlib. What led me to that conclusion? Easy, the same compressed file worked in Ubuntu. Must be a bug in zlib then!

The next step was getting zlib’s code and adding enough printf’s to know the problem was in the middle of the file, not at the beginning nor the end; indeed, most of the file could be correctly decoded, but then it just died. This looked more and more like a bug in zlib.

I began to scramble things around, trying to isolate the problem. Things just got weirder, the same code worked fine if instead of being inside a thread I was on the main thread. If you have psychic powers you now have enough information to know what the problem was. Although I should have known too (these wasn’t even the first time I saw a problem like this one!) I was mind set on finding a bug in zlib, which now, it seems, only appears while interacting with ACE (in my defense, I did see these kind of bugs too).

Fiddling around with the code some more, even stranger backtraces began to appear. First this one:

And then this other one, which led me into the right direction:

That last stack trace got me to think how could it be possible for an otherwise working program to coredump while creating an stdlib object. I mean, stdlib is quite well tested, isn’t it? Then it struck me: the keyword isn’t stdlib but creating. It was allocating memory from the stack, upon entering the function.

Some more research later I found out that Solaris default thread size is about 1 mb, while in Ubuntu this thread is of about 8 mb. And I also noticed the buffer I was allocating for zlib was taking up space in… the stack.

If there’s something to learn from this story is that you should always know what goes in the stack: only small objects should live there, and you should always know the max stack depth a function could reach. Otherwise it may come back and bite you in the ass when you’re back from your vacations.

Remember where were we last time? We had this code to define a list:

Now, to increase our template-foo, let’s practice some basic operations. The same operations you would implement to practice your skill any other functional language. If I remember correctly these where useful when learning Haskel: getting a list’s lenght, getting the Nth element, appending and preppending elements… that sort of stuff.

Let’s start with the most basic: getting the length of a list. We don’t really have a for loop so using recursion is the only way. It gets easier if we think again on our definition of list: “think of a list as tuple, two elements, the first (called head) will be the first element of the list and the second element as another list or a NIL object”. Whit this definition of a list, then it’s length turns to be 1 (the head) + the length of the remaining list (the tail), with a special case for the length of a NIL object which should always be 0. In template-speak:

I know. You are thinking “wait, what?”. Well, even for this basic case we need to use some esoteric language features:

• typename is needed to tell the compiler LST::Tail is a type and not a static variable (like Length::result is). Did you remember that from chapter IV?
• We have to use recursive templates, but you probably already figured that out. You should remember this from chapter II.
• We can provide a spetialization of a template. You should also remember this from chapter II.

Obviously, you can write it this way too:

The rest of the “basic” list-operations are quite similar, but I’ll leave that for another post.

We have been building our template meta-foo for five chapters now, and I think we are ready to move on to more advanced topics. We will be borrowing a lot more from functional languages from now on, so you may want to actually start practicing some template metaprogramming to keep advancing. In our previous entries we worked with basic building blocks, making it quite easy to keep in mind the whole “program flow”. Now it won’t be so easy anymore, as we’ll be using real metaprogramming (i.e. templates operating on templates) so a lot more thought will be needed for each program. Another point to keep in mind, you don’t have a debugger here. All the magic occurs at compile time so there is no gdb to step through your program to find a logic flaw. There’s a little trick to check if you are too far off from the target but, mainly, you’ll have to think for yourself.

Let’s start with any functional programming course basics: lists. We have to think, first, how can a list make any sense when you only have types and no values. It means you can have a list like “int, char, void**, Foo”, and not something like “1, 2, 3″. Or, can you? There’s a way to trick the compiler into creating a type from a integral value:

Voila! Now you can create a list of numbers. For our next trick, let’s implement the list itself. No pointer magic, think of a functional definition of a list. Come on, I’ll wait… ready? OK, a list is a tuple T of two values, in which the first element, called head, is the first element of the list and the second element, called tail, is either a list or the NULL element.

Quite a mouthful… let’s go over that definition again:

So, as an example, a list of numbers could be expressed as:

Closing up… how would you define this list in C++? Easy:

We need here a NIL type to use as a list ending element. We could also use a default template type, so we won’t have to write the last NIL to end a list definition. Thus we have now:

Nice. You should remember the following rules:
1. We can use template to define a template class, defining a new type based on a number instead of another type
2. We can’t “store” a value in a type… unless we store it as a static value, that is.
3. Using a convention for defining result holding variable names is very useful, as there are no interfaces and more than once we’ll be using a result from an unknown class

With that said, let’s translate the list (1, 2, 3) to Tmpl C++

Not so bad to start with. Next time we’ll be doing something a little bit more useful with this list.
One last note, initializing a static const int in the definition of the class may be non portable (some compilers seem to have trouble with it). An enum may be used instead.

By now we have learned the basics for a nice template metaprogramming toolkit: Loops with recursive template definitions Conditionals with partial template specializations Returns using typedefs

Unfortunately that’s all you need for a Turing complete language, meaning now we have the power, bwahahaha! Mph, I’m sorry, back on topic, it means we can now create a fully functional and useful template metaprogramming device… for approximating e, nonetheless. Oh, you think that’s not useful? Well though luck, that’s all you get for now:

Looking nice, isn’t it? You should have all what’s needed to understand what’s going on there. Even more, almost everything has been explained in previous articles, with the exception of EqBase. But that’s left as an exersice for the reader because the writer is too lazy.

If you think any part of the code requires clarification ask in the comments. Next, a long overdue topic: lists using template metaprogramming. Guaranteed to blow your mind into little pieces!