Keeping the noise out of a large fraction of iterator-based loops is great, but other common for loops are missing out! They have not received a convenient shorthand. Luckily, you can make some yourself by feeding the right containers to the range-based for statement.

For example, the regular counting loops

for (auto i = begin, e = end; i < e; ++i)
for (size_t i = 0, e = container.size(); i < e; ++i)


can be written as

for (const auto i : range(begin, end))
for (const auto i : indices(container))


if you define indices() and range() the right way.

Using the range-based version has no drawbacks: it is easier to read, quicker to write and compiles to equivalent assembly (calls to operator!= and operator++ get optimized away even at g++ -O1).

You can do a lot with the range-based for statement. What I have found useful so far is:

• range(begin, end): for (auto i = begin, e = end; i != e; ++i)
• range(end): same as range(0, end)
• indices(container): same as range(container.size())
• keys(container_of_pairs): shorthand for getting only value.first
• values(container_of_pairs): shorthand for getting only value.second

The code is available on github under the Boost Software License. It has no external dependencies and using it should be as simple as putting the header file into your include path. It is also fairly minimal, implementing only what is needed to make it work. A lot could be added, like rbegin(), rend(), generic reverse(), correct iterators, …

The approach has its limits, and one of them is exposed when you implement something like enumerate():

for (auto e : enumerate(c)) {
    // e.index is the current index
    // e.value is the current value
}


For standard containers (as well as the keys() and values() ranges above) the type in the for statement controls whether you get a copy or a reference of what you are iterating over as well as whether it should be const or not:

for (const auto e : c) // e is a const copy
for (auto & e : c) // e is a mutable reference


That can only work as long as what you want to put into e is an lvalue (i.e. c.begin() is an output iterator). It breaks down for enumerate() which cannot return a reference without extra overhead.

That is why I decided to split enumerate() into two functions. enumerate_byref() always grabs ‘value’ by reference and enumerate_byval() makes a copy. Usage then becomes:

for (const auto e : enumerate_byref(container))
    e.value = true; // ok, modifies container!
for (auto e : enumerate_byval(container))
    e.value = true; // ok, changes copy


Which is far from the ideal syntax, but at least it is explicit about what is going on.

Thanks to my employer, CeleraOne GmbH, for permitting me to publish this.

• RPATH and RUNPATH
• Qt Creator editor news
• New QML editor features
• Qt Simulator is going public

the first level of Star Guard

the same scene in the editor

A few things came together that caused me to make this level editor:

• I was looking for an excuse to do something interesting with QML.
• Star Guard is a great game and stores its levels in friendly, easily accessible XML files.
• When I emailed Loren about the project, he allowed me to use some of the game’s artwork.

The whole thing was surprisingly easy to do, took a couple of days and amounted to less than 1200 lines of code – one half of it QML and the other C++.

If you don’t know about QML: It’s a fairly new part of the Qt framework, essentially an extension of JavaScript to build user interfaces declaratively – and great fun to work with. All the details can be found on its documentation page and if you want to get started, the Qt 4.7 beta1 packages have just been released.

The editor uses QML to describe every aspect of the user interface. It’s turned out to be very useful in the beginning of the project when the editor was still QML-only and the interface layout and behavior changed a lot. It also means that it’d be very easy to redo the user interface now – even for someone without any C++ knowledge. Also, not having to recompile after every step simply makes fiddling with the UI a lot more pleasant.

Of course there’s the disadvantage of not following the native look and feel. But no one ever complains that most games don’t use it – and I don’t think it’s necessary for a level editor to look just like your spreadsheet application either.

The C++ part deals with the backend work, like providing a model that holds the map, a QML item that paints it, and loading from and saving to files. I used the QML plugin mechanism to expose these to the UI code.

The editor is fully functional; I consider it done and will not continue working on it. If someone wants to improve it or base his own tile based level editor on it, feel free! I release the code under the Boost Software License 1.0.

Linux binary (64 bit) and source code (12 MB)
Windows binary and source code (10 MB)
source code only (.tar.bz2, 20 kB)
source code only (.zip, 30 kB)

In addition to up-to-date dependencies, this release incorporates a wealth of fixes and improvements by Benjamin Kramer, Frits van Bommel, Kelly Wilson, Leandro Lucarella, Matti Niemenmaa, Moritz Warning, Robert Clipsham, Tomas Lindquist Olsen and me.

Linux x86-64 download

• lots of bug fixes
• x86-64 support is mature
• inline asm improved (we now define D_Inline_Asm)
• cross-compilation support
• uses boehm-gc during compilation (x86-32 only)
• D specific optimizations:
  • turn GC allocations to allocas if possible
  • simplify or remove certain calls to D runtime functions

The command line interface of LDC now has added options in line with other LLVM based tools. Please use the ldmd wrapper if you want a drop-in replacement for DMD.

Linux x86-32 download
Linux x86-64 download

Tomas Lindquist Olsen
Christian Kamm
Frits van Bommel
Kelly Wilson

The original summary was:

The new D1 compiler based on the strong fundament of LLVM, the DMD frontend and the Tango runtime is maturing rapidly. We’ll take a brief look at LLVM, report on LLVMDC’s status and discuss the future of the project.

Have fun watching the video and thanks again to the organizers for an excellent conference.

Here’s a short list of major changes since then:

• Almost immediately after the lecture we renamed the compiler to LDC, which has worked out very well and should simplify the next presentation by avoiding tounge twisting sentences that contain LLVM, LLVMDC, DMD and GDC in close proximity.
• We released LDC 0.9 for x86-32 Linux in early January.
• Support for x86-64 Linux has improved significantly due to contributions by Kelly Wilson and Frits van Bommel. Our platform support entry for it now reads: “no big open issues,
  dstress results pretty much the same as x86-32 Linux” – The next release will very likely include x86-64 Linux binaries!
• LDC switched to LLVM 2.5.
• ABI conformance work continued: The D calling convention is fully implemented and naked functions are supported.
• A lot of tickets have been closed; this includes a large amount of bugfixes, but also higher level issues like allowing template functions to be inlined and basic cross compilation.

LDC ships with a precompiled Tango rev 4237 and passes all except two of Tango’s unittests (io.digest.Md2 and text.locale.Posix fail). DStress results also look favorable and the GtkD demos work. The chances are good that your code will work with it too!

There are several known issues, the most severe being:

• LDC does not compile DWT successfully for unknown reasons
• an LLVM 2.4 bug sometimes leads to linker errors when unreachable code is optimized away
• LDC doesn’t fully follow the D calling convention (but it’s close!)
• we inherit every DMD-frontend related bug in the D bug tracker
• … there are more in our tracker

If you encounter a bug, please check our bug tracker and create a new ticket if the issue isn’t listed yet. Maybe you are feeling adventurous and want to try fixing it yourself; in that case take a look at our getting started guide.

LDC could support other platforms. Furthest along so far are

• x86-64 Linux: needs people to start fixing smaller bugs, exception bug (LLVM issue)
• x86-32 Mac: small runtime issues, needs tests
• x86-32 Windows: exceptions not supported (LLVM issue)

but support for these platforms won’t improve on its own! Several friendly people have offered their help – we need more of those!

For those with big CTFE memory needs, we have an experimental version of LDC available which has the compile-time garbage collector enabled. In the future, we’re going to experiment with a forward reference hack, but right now it still introduces too many regressions.

Feedback and questions are appreciated and should go to the mailing list. Alternatively, we’re often seen in #ldc on FreeNode.

Tomas Lindquist Olsen, Christian Kamm

First, the throw statement. Its basic job is simple: invoke the exception handling runtime by calling _Unwind_RaiseException with the address of an _Unwind_Exception struct. This struct contains, among some private data, an eight-byte exception class to identify the language and vendor it originates from (for LDC we set it to “D1\0\0″ and “LLDC”) and a cleanup callback. Since it is necessary to communicate the exception object that is being thrown to the handler code, this struct is embedded in a larger one. Later, the address of this surrounding struct can be computed from the address of the unwind_info member.

Consequently, the outer struct looks like this

struct _d_exception {
  Object exception_object;
  _Unwind_Exception unwind_info;
}

and the code to invoke the unwinding runtime is straightforward:

void _d_throw_exception(Object e) {
    if (e !is null) {
        _d_exception* exc_struct = new _d_exception;
        exc_struct.unwind_info.exception_class[0..4] = "LLDC";
        exc_struct.unwind_info.exception_class[4..8] = "D1\0\0";
        exc_struct.exception_object = e;
        _Unwind_RaiseException(&exc_struct.unwind_info);
    }
    abort();
}

What happens on a throw is essentially the following:

• _Unwind_RaiseException traces the stack by looking at the unwind tables and, for each frame that has a landing pad set up, calls a ‘personality function’, asking it whether it can handle the exception object.
• Once one is found, it traces the stack again, this time telling the personality functions to execute the code in any intervening finally blocks.
• In the end, it calls the function for the final landing pad with arguments indicating that control is to be transferred to the catch handler.

Luckily, exception handling in D can be implemented using only a single personality function for all landing pads. This personality function decides what to do for each individual landing pad by parsing the language specific area of the unwind data. This area contains three tables: the callsite table, the action table and the classinfo table.

• The callsite table maps instruction address ranges to indices into the action table. These address ranges mark the beginning and end of the code in a try block.
• The action table contains chains of indices into the classinfo table and values that will be used to identify the action to the handler code later. An action corresponds to a catch or finally block.
• The classinfo table holds the addresses to the classinfos of each class used in a catch parameter.

When the personality function is called and given the context of a certain landing pad, it looks up the instruction pointer, finds the right entry in the callsite table and then walks the corresponding action chain. For each possible action, it checks whether the thrown exception object is derived from the class specified by the respective classinfo. Once a match is found, it knows that this landing pad is responsible for the exception. When it is called again with instructions to transfer control to the handler, the personality function passes the exception object and the index from the action table to the hander code.

If you’re interested in the code that accomplishes this, take a look here.

The last step required to make EH work is to provide the handler code and to write out the correct unwind tables. Let’s look at some user code and what it is essentially turned into by LDC (of course the actual output is LLVM IR). The situation grows considerably more complex when there are nested try-catch-finallys in the same stack frame, but I hope this snippet illustrates the basic ideas.

The llvm.eh.* intrinsics get the exception object and the action table index that are passed in by the personality function as mentioned above. But there’s more going on here: the selector intrinsic also tells LLVM what the data in the unwind tables should be. In particular, the personality function and the exception classinfos are set here. The zero indicates the finally block. The call to code_try() has been turned into an invoke, which makes LLVM emit an entry in the callsite table for it.

As you can see, the unwinding runtime and LLVM code generator are tied closely via the two intrinsics and thus supporting other runtimes such as Windows structured exception handling will be nigh-impossible without changes to LLVM. Hopefully, getting llvm-gcc to support exception handling on Windows will be enough of an incentive for the LLVM team to provide that feature eventually.

Another thing to bear in mind is that LLVM’s exception support is, at the moment, very C++ specific. The code generator can fill the language specific data area only with the three C++ style tables mentioned above. Fortunately, D’s exceptions are similar enough that we can get the right behavior by inserting suitable values into these tables.

For now, the implementation in LDC has only been tested on x86 Linux, though the PowerPC target should work as well. EH on x86-64 Linux will supposedly be enabled in the next LLVM release. The remaining issues should be solved as LLVM matures, enabling LDC to provide correct exception handling support on more platforms.

This is not a failing in itself. Indeed most users will never have the need or desire to compile their compiler from source. Yet it also prohibits testing patches for the front-end, makes debugging nigh impossible and generally complicates helping with DMD’s development.

The alternative – and often the only alternative if your target is not x86 Linux or Windows – is GDC. David Friedman took the open front-end and tied it to the GCC back-end, he even supports D 2.0 to some extend. While it works well and is fully open source, it has, unfortunately, remained a one-man effort. For me it was a combination of GCC’s daunting code base and the fact that there had been no activity for several months that stopped me from helping out.

Instead I eventually joined a different project: LDC. It is similar to GDC in that it also takes the open DMD front-end and aims to combine it with an equally free code generator. The back-end, though, is different: as the name suggests LDC emits LLVM bitcode, which can be compiled to native code, but could also be used for Just-in-Time compilation among other things. I’m also happy to say that even though LLVM is a large and ambitious project, it remains surprisingly easy to learn and work with.

Almost the first thing I did when I started contributing to LDC a few months back, was to integrate Thomas Kühne’s exhaustive DStress test suite with LDC: it is very useful for finding bugs and regressions and can even serve as a sort of crude progress indicator (crude because there is no relationship between number of bugs and number of tests; there are about 1000 tests for inline assembly and only 30 or so for exception handling). As you can see, there are still a handful (517 at the last count) of regressions with respect to DMD, but their number has been decreasing steadily.

In the last months, Tomas Lindquist Olsen, who started the project about a year ago and is responsible for almost all of its existing functionality, and I have added the last major missing parts to the compiler: inline assembly (thanks to David Friedman for the asm parsing and rewriting code!), exception handling and the synchronized statement are now supported. With these out of the way, goals for the future are squashing bugs, fixing some linking issues and getting LDC to work properly on Windows.

Speaking of other platforms: As Tomas and I both develop on x86 Linux, other configurations didn’t get much testing yet. Some people reported partial successes on FreeBSD and Sparc but more testers would be welcome. You can generally contact us by email or in #ldc on freenode. All in all, we’ve made great progress and if we can keep it up, LDC will be in the same league as DMD and GDC soon.

This refreshing change from usual development practice along with it being a challenge as well as a good time with friends have made Game-in-a-Day events appealing enough for whole communities (edit: link removed, unfortunately dead by now) to have formed around them. Last October I decided to mail all my friends and host one myself.

After the usual scheduling woes Roman Pohrt, Torsten Gellert, Felix Stürmer and Anne Pohrt (part-time) brought their computers and participated. Julian Bischof couldn’t bring his equipment but created some music for us remotely.

As our group contained no real artists, it was clear from the start that our result would not look particularly pretty. On the other hand, everyone had some programming experience, albeit evenly distributed between C++, D, Python and Java. Consequently, we figured we would try to make up for it in gameplay and features. That also meant we could skip the obligatory asteroids clone and tackle something more complicated.

The collective search for a game idea finished quickly. We were going to improve on C-Dogs by supporting multiple players on one computer. We also planned to replace its line-of-sight effect with my (then recent) soft shadows implementation, hoping to overshadow our lack of art by composing our levels from dark hallways and by using lots of special effects.

Next was the choice of programming language and libraries. Since there was no common denominator, we selected Python and pyglet. It turned out to be a very good decision. Even though half of us had not used the language before, the essentials were easy to pick up and soon everyone was able to contribute: At the end of the first day I made our previously hardcoded weapons system more modular, then went to bed. When I pulled the changes next morning, there were four different guns (and an electro zapper) and someone had added code so they could be put down and picked up.

Our source management tool, mercurial, was employed less successfully. We had one or two cases where we accidentally automerged into the wrong direction and propagated those incorrect changes through our repositories. However, I maintain that these issues could have been avoided if we had been more familiar with mercurial and that a centralized version control system would have had even more trouble with the highly concurrent nature of our development.

This is when we got started. After about 16 hours of work (spread out over the weekend) we ended up with the current feature set:

• up to three players can play simultaneously
• they can move, shoot and bump into walls
• the levels are randomly generated
• there are three kinds of monsters, all with different behaviors</li>
• players have hitpoints and can die
• five different types of weapons lie scattered around the levels and can be picked up
• background music plays
• finding the blue goal portal teleports to the next level

From the start we structured our efforts pretty well. At least one person was always working on the next logical step or an absolutely necessary subsystem. We also implemented quick placeholders for things that were to come later. For instance, it took a lot more effort to get solid random level generation working than we expected, but due to a square placeholder level it did not delay the development of other components at all.

Nevertheless, there are some things that did not get implemented or fixed in time:

• the game is too slow
• collision detection breaks often
• level generation sometimes produces incorrect geometry
• line of sight and shadows are missing
• there is no menu

The slowness is definitely our major issue. We started profiling and sped up some parts of the code considerably, but it wouldn’t help. In the end, we even concluded that while Python had been an ideal choice from the perspective of getting everyone involved and productive quickly, it had also failed us due to its lack of speed.
In hindsight, I’m pretty sure we were wrong in our assumption that some Python code was to blame for the speed deficiency. If I comment out our level drawing, which is basically a set of OpenGL calls, my framerate increases dramatically. Thus this is probably some OpenGL usage or hardware acceleration issue. (edit: installing the new pyglet 1.1 makes the game run almost as fast as it did with the level drawing commented out; it was a library problem after all)

While the unreliable collision detection is merely an effect of the low framerate, the missing menu is excusable and the incorrect level generation is rare and only a graphical glitch, the lack of shadows and line of sight is painfully obvious. We just didn’t get around to implementing it. It seemed to not be essential for basic gameplay and hence was postponed time and again. Without it, however, our game is missing a key aspect; something that was supposed to make it unique and different.

Even with these defects, I’m amazed by what we have achieved in such a short time. Especially considering that we started from scratch with a programming language half the team was not familiar with. It was a pity we had to stop when the result was not yet what we had envisioned, but that’s the premise of these events: try to get as far as you can before the time is up, then stop and take a look at what you’ve done.

We certainly had a good time and I plan on doing this again next autumn.

Screenshots: