Nicolás Brailovsky

Last time we had a virtual template dispatch problem… we got to the point of knowing which was the index of the cache we were searching for, now we need to actually retrieve an instance of that cache. That’s a problem. Why? To begin with, there are no instances, only types!

The next logical step would be to create a Map device, to map a list of types to a list of instances… let’s see how can we do that, in pseudocode

Looks easy. How can we map that to c++?

All those next.next.next.instance look ugly. Let’s use some more meta-magic to get the Nth instance (why not a [] operator? several reasons, you can’t mix non-const ints with templates nicely, there would be problems to define the return type… all those options are workable but it’s easier if we do this in another device.

Remember that one from the toolbox? Now we know how to get a specific index position, yet getting the instance is a different problem (the Nth device returns a type, not an instance). We should do something different, the problem is knowing the return type. What’s the return type for the Nth instance of the Instances list?

Not so easy, right? This is the translated C++:

And the code from fetching the instance itself is even more difficult, so I’ll leave that for next time.

Remember our virtual template method problem, from the other time? (I know, I said the answer was scheduled for a week after that post, but then I just forgot about it). May be we could avoid the virtual part by keeping a list of all our caches… how would we know which one should we dispatch the message to? Easy, using templates.

Instead of a list let’s keep two, for twice the fun. One for the rows cache, another for the PKs. We can use PK to know which ROW Cache should we choose. Let’s try to write a pseudo code for it:

Doesn’t look too hard. Building on our previous toolbox, let’s use Eq, Position and the definition of a list:

Great, now we can find an element on a list of types. The real virtual dispatch for the next entry

Wow, number eleven already. We’re getting more chapters here than Final Fantasy games. I didn’t even imagine there was so much to write about such an esoteric language features like templates. I do wonder if anyone will actually read it, but that’s a completly different problem.

Enough meta-meta talk: what can we do with all the things we have learned? We can calculate pi and e, we already showed that as an example on one of the first chapters. This chapter I’m going to write about what motivated me to explore the bizarre underworld of template metaprogramming. Some time ago I had to work with a Berkeley DB researching the feasibility of developing a magic cache for (real) DB table. Leaving aside the discussion of whether this is a good idea (the project did have a good reason to be researched) I hit a major roadblock when trying to provide a façde for every table; something like this: See the problem? To do something like that we’d need a virtual template method, and you can’t have that. After seeing that I thought to myself “Hey, I’ll use templates!”. Then I had two problems, but the damage was done, I couldn’t go back. What kind of contorted device could we implement to make such a devious requirement work? I’ll leave you to think it, the answers I came up with next week.

So far we’ve learned the basic constructs of template metaprogramming (loops, branching, return values) and some basic list operations (getting the length of a list, appending and preppending elements, checking if an element is included in a list). Let’s put it all together by creating an operation to return the position of an element. It’ll be very useful later on too.

If we go back to the Includes operation we can get some help to define the Position operation: the position of an element in a list is one plus the position of the element we’re searching for in the tail, or zero if the head equals said element. The operation is not defined if the element is not in the list.

Translating to pseudo-code:

The translation to C++ is not so trivial this time. Try it, I’ll wait… ready? OK, let’s start

Looks easy… but doesn’t work. First problem, we can’t compare two types, remember? We need to use Eq<X, Y> again. Second problem, although we said the operation is undefined if the element is not included on the list, it would be nice if we could force the compiler to fail if (or when) that happens. Let’s rewrite the operation using a façade again, but adding an Assert:

Oh, we haven’t defined assert yet! There’s another problem, too: even if it won’t compile, the compiler will try to expand _Position< …, NIL > indefinately, causing an error after too many nested template calls. Not nice. We need to add a case to make the compiler stop:

All that code for such a simple operation, man. Also, see what we did with Assert<>? It seems making a compile fail is actually quite easy. That’s what I have most experience with.

We’ve been through quite a lot, and our toolboox should be quite big already. Next time we’ll start steering towards some sort of aplicability, trying to use some of all these stuff to implement a real, useful and working program… assuming that’s even posible.

By now we should have learned how to perform loops, branching and returns using templates. Let’s add a couple of useful operations to our library: append and prepend.

Prepending an element to a list is very easy: the result is a list (oh surprise) consisting of a head (the element we want to add) and a tail (the old list). In the pseudocode I’ve been using so far:

And in C++ (trivial, this time):

Appending is a little bit more difficult, as we need to first find the end of the list. Think for a second how would you define it… back? Ok, I’d define it this way: appending an element to the list yields a list, consisting of the same head and the result of appending said element to the tail. The null case, as usual, is appending an element to a NIL list; in this case the result is a list with the element itself. So:

Looks complicated what it follows the same structure as the rest of the basic-ops:

Easy. Now, what happens if we want to add a default value for Lst, so we can use Append to create lists? Easy too, but we need a façade this time; just rename Append to _Append, then

I promised to add one more operation to our toolbox, returning the position of an element, but this post is getting quite long and I’m afraid it may be too much for the average attention span of a programmer… we’ll leave it for next time.

Remember last time? We learned how to get the lenght of a list. This time I’ll introduce some more of these basic ops. Let’s begin with “Nth”: getting the Nth element of a list; which, remember, in this case is a type, not a concrete element. This means the Nth element will be something like int, char, const char*, not 1, 2 or 3. We introduced a trick to get around this limitation before using a template <int>, go there to refresh your memory if needed.

So, what would the coloquial definition of “Nth” be? I’d put it like “The operation Nth for a list equals the head of the list for N = 0 and Nth (minus one) of tha tail otherwise”. A little bit more formally:

Translating this to C++ should be a breeze to you now. Try it, I’ll wait. Read? OK, this is MY answer:

Though the structure is very similar to the previous “basic operation”, getting the length of a list, the concept is quite different. This time we’re defining a return type recursivly. Anyway, it was too easy indeed, let’s try a more complex operation now.

How can we check if an element exists on a list? Seems easy enough, an element is included in a list if the head equals the element itself or if the element is included in the tail. In the pseudo language I just invented:

Looks easy, right? Well, there’s a bug there, can you spot it? Yeah, we’re missing the false condition. We should add a third spetialization:

Again, let’s translate the pseudocode to C++. Try it, I’ll wait. Read? OK, this is MY answer:

Looks nice, doesn’t it? Too bad it won’t work, you can’t compare two types. What would (int == char) mean in C++? We need a helper there, some kind of trick to compare to types. We can use partial template spetialization again:

With this little struct now we can write our include operation this way:

Very esoteric looking, the right mix of Haskel, C++ and booze to ensure job security for life. Next time we’ll find a way to search for the position of an element, a somewhat more complicated task.

Remember where were we last time? We had this code to define a list:

Now, to increase our template-foo, let’s practice some basic operations. The same operations you would implement to practice your skill any other functional language. If I remember correctly these where useful when learning Haskel: getting a list’s lenght, getting the Nth element, appending and preppending elements… that sort of stuff.

Let’s start with the most basic: getting the length of a list. We don’t really have a for loop so using recursion is the only way. It gets easier if we think again on our definition of list: “think of a list as tuple, two elements, the first (called head) will be the first element of the list and the second element as another list or a NIL object”. Whit this definition of a list, then it’s length turns to be 1 (the head) + the length of the remaining list (the tail), with a special case for the length of a NIL object which should always be 0. In template-speak:

I know. You are thinking “wait, what?”. Well, even for this basic case we need to use some esoteric language features:

• typename is needed to tell the compiler LST::Tail is a type and not a static variable (like Length::result is). Did you remember that from chapter IV?
• We have to use recursive templates, but you probably already figured that out. You should remember this from chapter II.
• We can provide a spetialization of a template. You should also remember this from chapter II.

Obviously, you can write it this way too:

The rest of the “basic” list-operations are quite similar, but I’ll leave that for another post.

We have been building our template meta-foo for five chapters now, and I think we are ready to move on to more advanced topics. We will be borrowing a lot more from functional languages from now on, so you may want to actually start practicing some template metaprogramming to keep advancing. In our previous entries we worked with basic building blocks, making it quite easy to keep in mind the whole “program flow”. Now it won’t be so easy anymore, as we’ll be using real metaprogramming (i.e. templates operating on templates) so a lot more thought will be needed for each program. Another point to keep in mind, you don’t have a debugger here. All the magic occurs at compile time so there is no gdb to step through your program to find a logic flaw. There’s a little trick to check if you are too far off from the target but, mainly, you’ll have to think for yourself.

Let’s start with any functional programming course basics: lists. We have to think, first, how can a list make any sense when you only have types and no values. It means you can have a list like “int, char, void**, Foo”, and not something like “1, 2, 3″. Or, can you? There’s a way to trick the compiler into creating a type from a integral value:

Voila! Now you can create a list of numbers. For our next trick, let’s implement the list itself. No pointer magic, think of a functional definition of a list. Come on, I’ll wait… ready? OK, a list is a tuple T of two values, in which the first element, called head, is the first element of the list and the second element, called tail, is either a list or the NULL element.

Quite a mouthful… let’s go over that definition again:

So, as an example, a list of numbers could be expressed as:

Closing up… how would you define this list in C++? Easy:

We need here a NIL type to use as a list ending element. We could also use a default template type, so we won’t have to write the last NIL to end a list definition. Thus we have now:

Nice. You should remember the following rules:
1. We can use template to define a template class, defining a new type based on a number instead of another type
2. We can’t “store” a value in a type… unless we store it as a static value, that is.
3. Using a convention for defining result holding variable names is very useful, as there are no interfaces and more than once we’ll be using a result from an unknown class

With that said, let’s translate the list (1, 2, 3) to Tmpl C++

Not so bad to start with. Next time we’ll be doing something a little bit more useful with this list.
One last note, initializing a static const int in the definition of the class may be non portable (some compilers seem to have trouble with it). An enum may be used instead.

By now we have learned the basics for a nice template metaprogramming toolkit: Loops with recursive template definitions Conditionals with partial template specializations Returns using typedefs

Unfortunately that’s all you need for a Turing complete language, meaning now we have the power, bwahahaha! Mph, I’m sorry, back on topic, it means we can now create a fully functional and useful template metaprogramming device… for approximating e, nonetheless. Oh, you think that’s not useful? Well though luck, that’s all you get for now:

Looking nice, isn’t it? You should have all what’s needed to understand what’s going on there. Even more, almost everything has been explained in previous articles, with the exception of EqBase. But that’s left as an exersice for the reader because the writer is too lazy.

If you think any part of the code requires clarification ask in the comments. Next, a long overdue topic: lists using template metaprogramming. Guaranteed to blow your mind into little pieces!

By now you should have noticed the warnings were not in vain: we are exploring a bizarre side of C++ here, a side many people prefer to, wisely, ignore. Luckily it probably is too late for you, there is no way back. Only a long spiraling way down into the arms of despair and cryptic compiler error messages… mwahahahaha. But now, let’s see where we are. In previous entries we learned how to return values, how to define recursive devices and how to provide a partial specialization. Let’s see know how can we use partial specialization and complex return type definitions for some more fun template metaprogramming tricks. We had a fraction and a ScalarMultiplication operation for Frak:

In previous entries we learned how to return values, how to define recursive devices and how to provide a partial specialization. Let’s see know how can we use partial specialization and complex return type definitions for some more fun template metaprogramming tricks. We had a fraction and a ScalarMultiplication operation for Frak:

Let’s try to add an operation to simplify a Fraction. Simplify< Frak<2, 4> > should return 1/2. Mph… simplifying a fraction means dividing it by the MCD. A quick trip to Wikipedia reveals a nice recursive way to implement an MCD device:

I won’t get into much detail as the link explains it a lot better than whatever I could try, but do take a look at the definition of MCD<X, 0>: that’s a partial specialization. No magic there. Back to our simplifying device, we now have all the parts for it. Going back to it’s definition we can see that simple(fraction) = fraction / mcd(fraction). Then:

Quite a mouthful, but a lot simpler than what you think as there is a lot of unnecessary code there. Until new_num and new_den, no surprises. Typedeffing a Frak is not new, either. typedef typename is something new: typename tells the compiler you’re referring to a name inside a template class, otherwise it’d try to refer to a static variable inside said class (*). Knowing what each thing does we can simplify it:

It is a matter of style really. In this case I’d rather use the second one because it matches better its colloquial definition, but if you think the first one is more readable go with it… it doesn’t really matter though, no one will ever even try to read this kind of code if you intend to use it in a real application.

Next time: a “useful” (**) and complete template metaprogramming device, using the complete toolset we’ve been learning in this crazy templating series.

(*) Think of it this way:

In a template you don’t know if Bar is a typename or varname because there’s no access to the specific template definition.
As a rule of thumb, if the compiler complains then add typenames.

(**) Results may vary according to your definition of useful.