OSMesa gripes

Offscreen rendering is a desirable instrument for many graphics developers. It's an invaluable debugging tool. Gnash's testsuite has had an infrastructure added for graphics testing quite a long time ago. When I recently rewrote the Cairo renderer, I also added support for off-screen rendering.

Recently, I finished rewriting the OpenGL renderer. In the course of this rewrite, a certain other developer noted at least four times that he would like to see an offscreen rendering ability for the OpenGL renderer. :) Naturally, I agreed.

OpenGL does not provide an offscreen rendering system, so many OpenGL implementations have devised a system of their own. For Mesa, which is the standard free software OpenGL implementation, OSMesa was developed. It follows that my next step was to utilize OSMesa in Gnash. [1]

After writing an initial implementation, I discovered that although every OSMesa call returned successfully, the offscreen buffer was not being written to. After a long debugging session I discovered that the linking was at fault. Apparently, -lGL -lOSMesa yields no results, whereas with -lOSMesa -lGL the offscreen renderer works brilliantly.

That is, until you decide to switch back to normal rendering mode. Because not only does OSMesa care about the linking order, normal rendering will not work if you have OSMesa linked into your program, even if you never make a single call to OSMesa's routines!

One is left to wonder if OSMesa is so rarely used that few people came across these issues, or that simply the developers do not care.

[1] Of course, OSMesa will not work for OpenGL implementations other than Mesa, so enabling automated graphical testing on other platforms requires some more effort. AGL, OS X's toolkit also provides an offscreen rendering system. The same can probably be said for WGL, but non-free DRI-based OpenGL implementations for free operating systems may omit the offscreen buffer.