Nicolás Brailovsky

Ohh design patterns. I love buzzwords. Who doesn’t? To increase my buzzword count I will be writing about a topic most people programming C++ should already know: RAII, resource acquisition is initialization.

Patterns everywhere

Knowing that Gamma et al didn’t list all the known patterns in the universe does come as a surprise to some, not sure why though. The twenty some patterns they write about in their now famous book are (arguably, perhaps) some of the most common design patterns, but the list hardly finishes there.

Some patterns only have meaning in a very specific context; a reactor is a nice pattern for handling asynchronous events yet most applications would never need it. Sometimes “the context” means a specific domain in which the application must work, like a web application, a real time application or a distributed application, sometimes the context is the language itself. RAII falls in this last domain, it only makes sense for C++ applications (actually there are others, but thinking what kind of languages would support this idiom is left as an exercise for the reader).

No, really. What is RAII?

If you made it past that long intro you are probably really interested in knowing what RAII is, and don’t know how to search it in Wikipedia. OK, I’ll explain it the best I can then: RAII means that a resource acquisition is the initialization.

Seriously. That is all the secret there is to RAII. Talking in code:

An example

Compare that to a visitor pattern. It’s just too simple to be of any use, isn’t it? Well, contrary to what Java fanboys tend to believe you can do lots of nice things without writing a bazillion lines of code.

Wow… a whole lot of code just for calling bar and baz. And as I wrote that without even compiling I’m sure there are too many hidden bugs, lots of code-paths my simple programmer’s mind can’t even begin to imagine which will cause my Expensive_Resource to be leaked!

Let’s rewrite that using RAII:

A lot nicer, isn’t it? But, what happened to all the try/catch if’s and closes there?

Where’s the magic?

The magic of RAII lies in how C++ handles exceptions. When we have a built object (can an object be in an unbuilt state?) it means it’s constructor has correctly ran. It also means it’s destructor will run when it goes out of scope, doesn’t mater HOW it goes out of scope.

See how brilant that is? Doesn’t matter if a function we’re calling throws, or if we need to return before reaching the end of the function: the destructor will be called and thus our Expensive_Resource will be free!

Why is this C++ specific?

This is an easy one: think how would you implement this in Java: right, you can’t. Not knowing when is your object going to be destroyed means you can’t do anything useful in its destructor, therefore RAII is deeply rooted within the memory management in C++ and it’s pretty much a language specific pattern (or is it?). But, is that so good?

Not everything is so great…

You are probably thinking this is the best discovery since ice cream was invented. Well, not so fast, RAII has it’s detractors too.

The first problem in RAII, it doesn’t have a graceful way of handling resource acquisition failure. If Expensive_Resource is a database, and it’s connection fails, we have a throwing constructor.

Even if throwing constructors are acceptable, throwing destructors may even be a worst idea: throwing while an exception is already active is a cause for concern (tip: it’ll crash your application, doesn’t matter how many try/catch blocks you use, due to stack unwinding issues).

And then, even if we don’t care about throwing destructors you have the issue of a release failure: how do you notify the user that a release failure has occurred? And what do you do, should it happen?

So, RAII is a great idea indeed, and it has it’s uses, but you should be careful when choosing this C++ specific pattern.

Wow, number eleven already. We’re getting more chapters here than Final Fantasy games. I didn’t even imagine there was so much to write about such an esoteric language features like templates. I do wonder if anyone will actually read it, but that’s a completly different problem.

Enough meta-meta talk: what can we do with all the things we have learned? We can calculate pi and e, we already showed that as an example on one of the first chapters. This chapter I’m going to write about what motivated me to explore the bizarre underworld of template metaprogramming. Some time ago I had to work with a Berkeley DB researching the feasibility of developing a magic cache for (real) DB table. Leaving aside the discussion of whether this is a good idea (the project did have a good reason to be researched) I hit a major roadblock when trying to provide a façde for every table; something like this: See the problem? To do something like that we’d need a virtual template method, and you can’t have that. After seeing that I thought to myself “Hey, I’ll use templates!”. Then I had two problems, but the damage was done, I couldn’t go back. What kind of contorted device could we implement to make such a devious requirement work? I’ll leave you to think it, the answers I came up with next week.

There are two well known macros from the preprocessor which every macro-sorcer must know. They are __FILE__ and __LINE__. You probably already know about them but anyway, __FILE__ will give you the current file and __LINE__ the current line. Easy, huh?

int main() {    printf("%s : %i", __FILE__, __LINE__);    return 0; }

The program above would give you “main.cpp : 3″ as a result. There is nothing going on at execution time, it’s all preprocesor wizardy. In fact with “g{++/cc} -E” you can even check what the “real” output is (-E means to return the preprocessor output. Keep in mind a lot of stuff will be included from the headers you use).

Well that’s nice and all, but g++ can top this easily:

There are a couple of notable things about this new “pretty function” thing:

• 1. It will demangle a function’s name
• 2. This time it isn’t a preprocessor secret thing but a real variable g++ will create.

You can easily use this for better logging functions now (with some macro wizardy, obviously).

So far we’ve learned the basic constructs of template metaprogramming (loops, branching, return values) and some basic list operations (getting the length of a list, appending and preppending elements, checking if an element is included in a list). Let’s put it all together by creating an operation to return the position of an element. It’ll be very useful later on too.

If we go back to the Includes operation we can get some help to define the Position operation: the position of an element in a list is one plus the position of the element we’re searching for in the tail, or zero if the head equals said element. The operation is not defined if the element is not in the list.

Translating to pseudo-code:

The translation to C++ is not so trivial this time. Try it, I’ll wait… ready? OK, let’s start

Looks easy… but doesn’t work. First problem, we can’t compare two types, remember? We need to use Eq<X, Y> again. Second problem, although we said the operation is undefined if the element is not included on the list, it would be nice if we could force the compiler to fail if (or when) that happens. Let’s rewrite the operation using a façade again, but adding an Assert:

Oh, we haven’t defined assert yet! There’s another problem, too: even if it won’t compile, the compiler will try to expand _Position< …, NIL > indefinately, causing an error after too many nested template calls. Not nice. We need to add a case to make the compiler stop:

All that code for such a simple operation, man. Also, see what we did with Assert<>? It seems making a compile fail is actually quite easy. That’s what I have most experience with.

We’ve been through quite a lot, and our toolboox should be quite big already. Next time we’ll start steering towards some sort of aplicability, trying to use some of all these stuff to implement a real, useful and working program… assuming that’s even posible.

An interesting topic for a change: is Linux binary portable? That is, can we take a binary file and be sure it’ll run in any other Linux system? What happens if we broaden that to any POSIX system, will it blend? Eh, I mean, will it run?

Doing some research on the subject I wrote down a list of the thought process which led my to an (inconclusive) answer:

1. First we should define what a binary is for us: When we talk about a binary we are usually thinking about a compiled binary file, not an interpreted script file like Ruby or Python. Those are for people who like things to actually work, so let’s focus on a compiled executable file, like a C/C++ application.

2. Defining compiled file: What could it be other than a sequence of bytes the microprocessor can understand? Yes, that’s right, it’s sort of interpreted code, only there’s electronics behind, not more code. This brings us to the first interesting conclusion: the executable must be (leaving emulators aside) compatible with the architecture you’re on. Running Sparc? Well then, the binary better be compiled for Sparc because otherwise to the uP will not make any sense.

3. Format: as any other thing, a binary file must have a format. That is a standard which defines the structure the file will follow. ELF is the binary format for Linux and it’s quite standard. Of course, if the binary format is a standard then we should get perfect portability between different platforms running on equal architecture. Unfortunately that’s not the case.

4. (Cont’d) Why don’t we? The binary depends not only on compile time “stuff” but a loading time linking occurs: the executable binary will get linked with the system files like glibc, or any other dependency on a shared library it may have.

So, what are the keypoints for Linux binary portability? Architecture, binary format and system libraries.

Of course, making the executable run is only part of the equation, as running and segfaulting on the spot is not so nice either. For this last part you’ll have to closely follow the standards defined by POSIX for paths and stuff like that.

Epilogue

As an epilogue, we could add that Windows binary compatibility tends to be great. Running binaries from 12 years back is no small feat, yet this leads to a whole lot of other problems: an incredible complex loader, security bugs, backwards compatibility headaches, et al. The old new thing is a great source of information for this topics, I’m quite illiterate about Windows binaries nowdays

Followup links

• http://evanjones.ca/portable-linux-binaries.html
• http://autopackage.org/docs/devguide/ch07.html (I swear this one was working when I wrote this text… a year ago)

Oh, talking about binaries:

By now we should have learned how to perform loops, branching and returns using templates. Let’s add a couple of useful operations to our library: append and prepend.

Prepending an element to a list is very easy: the result is a list (oh surprise) consisting of a head (the element we want to add) and a tail (the old list). In the pseudocode I’ve been using so far:

And in C++ (trivial, this time):

Appending is a little bit more difficult, as we need to first find the end of the list. Think for a second how would you define it… back? Ok, I’d define it this way: appending an element to the list yields a list, consisting of the same head and the result of appending said element to the tail. The null case, as usual, is appending an element to a NIL list; in this case the result is a list with the element itself. So:

Looks complicated what it follows the same structure as the rest of the basic-ops:

Easy. Now, what happens if we want to add a default value for Lst, so we can use Append to create lists? Easy too, but we need a façade this time; just rename Append to _Append, then

I promised to add one more operation to our toolbox, returning the position of an element, but this post is getting quite long and I’m afraid it may be too much for the average attention span of a programmer… we’ll leave it for next time.

Remember last time? We learned how to get the lenght of a list. This time I’ll introduce some more of these basic ops. Let’s begin with “Nth”: getting the Nth element of a list; which, remember, in this case is a type, not a concrete element. This means the Nth element will be something like int, char, const char*, not 1, 2 or 3. We introduced a trick to get around this limitation before using a template <int>, go there to refresh your memory if needed.

So, what would the coloquial definition of “Nth” be? I’d put it like “The operation Nth for a list equals the head of the list for N = 0 and Nth (minus one) of tha tail otherwise”. A little bit more formally:

Translating this to C++ should be a breeze to you now. Try it, I’ll wait. Read? OK, this is MY answer:

Though the structure is very similar to the previous “basic operation”, getting the length of a list, the concept is quite different. This time we’re defining a return type recursivly. Anyway, it was too easy indeed, let’s try a more complex operation now.

How can we check if an element exists on a list? Seems easy enough, an element is included in a list if the head equals the element itself or if the element is included in the tail. In the pseudo language I just invented:

Looks easy, right? Well, there’s a bug there, can you spot it? Yeah, we’re missing the false condition. We should add a third spetialization:

Again, let’s translate the pseudocode to C++. Try it, I’ll wait. Read? OK, this is MY answer:

Looks nice, doesn’t it? Too bad it won’t work, you can’t compare two types. What would (int == char) mean in C++? We need a helper there, some kind of trick to compare to types. We can use partial template spetialization again:

With this little struct now we can write our include operation this way:

Very esoteric looking, the right mix of Haskel, C++ and booze to ensure job security for life. Next time we’ll find a way to search for the position of an element, a somewhat more complicated task.

* Post from the wayback machine. I wrote this a long time ago but it got way down the posts queue, don’t know why *

I liked my vacations very much, thank you. Some people enjoyed vacations from me too. At work they even decided to keep until my return. Upon my arrival a nice coredump was waiting at my desk, so to speak. Check it out, isn’t it beautiful?

Yeah, that’s getting killed inside zlib. Nice way to start the year, a bug in zlib. What led me to that conclusion? Easy, the same compressed file worked in Ubuntu. Must be a bug in zlib then!

The next step was getting zlib’s code and adding enough printf’s to know the problem was in the middle of the file, not at the beginning nor the end; indeed, most of the file could be correctly decoded, but then it just died. This looked more and more like a bug in zlib.

I began to scramble things around, trying to isolate the problem. Things just got weirder, the same code worked fine if instead of being inside a thread I was on the main thread. If you have psychic powers you now have enough information to know what the problem was. Although I should have known too (these wasn’t even the first time I saw a problem like this one!) I was mind set on finding a bug in zlib, which now, it seems, only appears while interacting with ACE (in my defense, I did see these kind of bugs too).

Fiddling around with the code some more, even stranger backtraces began to appear. First this one:

And then this other one, which led me into the right direction:

That last stack trace got me to think how could it be possible for an otherwise working program to coredump while creating an stdlib object. I mean, stdlib is quite well tested, isn’t it? Then it struck me: the keyword isn’t stdlib but creating. It was allocating memory from the stack, upon entering the function.

Some more research later I found out that Solaris default thread size is about 1 mb, while in Ubuntu this thread is of about 8 mb. And I also noticed the buffer I was allocating for zlib was taking up space in… the stack.

If there’s something to learn from this story is that you should always know what goes in the stack: only small objects should live there, and you should always know the max stack depth a function could reach. Otherwise it may come back and bite you in the ass when you’re back from your vacations.

Remember where were we last time? We had this code to define a list:

Now, to increase our template-foo, let’s practice some basic operations. The same operations you would implement to practice your skill any other functional language. If I remember correctly these where useful when learning Haskel: getting a list’s lenght, getting the Nth element, appending and preppending elements… that sort of stuff.

Let’s start with the most basic: getting the length of a list. We don’t really have a for loop so using recursion is the only way. It gets easier if we think again on our definition of list: “think of a list as tuple, two elements, the first (called head) will be the first element of the list and the second element as another list or a NIL object”. Whit this definition of a list, then it’s length turns to be 1 (the head) + the length of the remaining list (the tail), with a special case for the length of a NIL object which should always be 0. In template-speak:

I know. You are thinking “wait, what?”. Well, even for this basic case we need to use some esoteric language features:

• typename is needed to tell the compiler LST::Tail is a type and not a static variable (like Length::result is). Did you remember that from chapter IV?
• We have to use recursive templates, but you probably already figured that out. You should remember this from chapter II.
• We can provide a spetialization of a template. You should also remember this from chapter II.

Obviously, you can write it this way too:

The rest of the “basic” list-operations are quite similar, but I’ll leave that for another post.

Perfection is finally attained not when there is no longer anything to add but when there is no longer anything to take away, when a body has been stripped down to its nakedness.

Antoine de Saint-Exupery