As mentioned in the past few articles, I intended to use this blog as a means of explaining “useful things” we’ve done in ISPC – in particular, as part of the OSPRay project – that are hard to describe in any other medium. Now while some of the previous topics were more useful “across the board” (ie, even for little kernels, no matter what program), this article will cover what is arguably – at least for projects on the scale of OSPRay – even more useful …. namely, our experience with mapping at least “some” C++/object oriented programming concepts into non-trivial ISPC programs.
Background: ISPC and C++ …
Looking at the design and features of ISPC, it’s pretty good at expressing low-level performance related issues like data layouts, vectorized (and in particular mixed-vectorized-and-scalar) control flow, etc … but somewhat less strong in the area of building large, complex code bases. Sure, in theory you could, but you rather quickly run into limitations like no templates, no STL, no virtual functions (or, in fact, any sort of methods), etcpp. As such, the way it looks like it is intended to be used is in a mix with C++ – C++ for the “Big system framework”, and ISPC for a few, performance-critical kernels. And of course, in that sense it’s not all that different from other frameworks such as OpenCL, either.
There are, however, cases – like building an entire, non-trivial renderer – where that design isn’t all that easy to stick with. In OSPRay, for example, the very base concept is that as a ray tracer it can do all these abstractions of “I don’t care which actual type of geometry it is, as long as it can provide a bounding box” etc, which just cries out for a C++ object hierarchy … and which you want to use so fine-grained that you really don’t want to always go back-and-forth between “host code” and “device code” (in particular because unlike for most OpenCL use cases, there is no difference between host code and device code in ISPC – it usually does all run on the same CPU!).
As such, pretty early on we decided to go and try to “emulate” as much of C++ in ISPC as we could, using function pointers etc in exactly the same way the first C++ compilers were built, too (for those old enough to remember: the first C++ compilers were actually “pre-“compilers that compiled from C++ to C, then used a C compiler to do the rest!). In this article, I’ll explain a bit on how we did this.
Templates
Starting with the bad news first – templates don’t exist. As in “not at all”. And no, there’s no good or easy way to do it. If I had one wish for ISPC, it’d be exactly that – give me some templates. Not even the full-features ones with which you can do template-metaprogramming, and write programs that you can neither understand the program nor the compiler’s error messages any more … but some basic templates like templating over base types and integers would go a long way.
The way we solved it in OSPRay therefore is, to large degree, a plain “not at all”. In some cases we’ve “solved” it by always going with a good-enough intermediary representation (eg, intermediary frame buffer tiles always store floats, even when bytes would have been enough). In a few cases, we’ve gone with using preprocessor macros, that emulate template parameters through textural processing. E.g., you first define a “template” using something like that:
#define define_vec3_operator(T) \
inline vec3##T operator+(vec3##T a, vec3##T b) { .... }\
inline vec3##T operator-(....) { ... } \
...
… then “instantiate” this with something like that:
define_vec3_operators(f); // defines vec3f+vec3f, vec3f*vec3f etc define_vec3_operators(i); // ... #undef define_vec3_operators
That certainly ain’t a nice solution, but it’s a solution, which is better than nothing. I suggest you use it sparingly, but it can be very useful. For example, in our volume sampling code using this eventually led to some 2x performance improvement (I won’t go into details here, but if interested can write more about it).
Inheritance and Virtual Functions
Though templates would be “useful” to have, a proper class hierarchy – and virtual functions – is even more important. It’s just very hard to imagine a non-trivial ray tracer without object oriented programming and virtual functions. For example, in OSPRay we did want to be able to say that there are “Geometries” that can “intersect a ray” and “query surface information re a given intersection”, no matter whether that geometry is a triangle mesh, a set of spheres, a implicit iso-surface, or anthing else … and the best way to do that is, of course, a “Geometry” base class, with virtual methods for “Geometry::intersect(Ray)” and “Geometry::postIntersect(Hit)”, etc.
Unfortunately, ISPC doesn’t have these language features – but quite fortunately, that is only a result of the parser not offering it, because everything you do need to implement this paradigm is there. As such, we can emulate it:
Inheritance
To start with, let’s look at inheritance. Say we want each Geometry to have a bounding box, and both a TriangleMesh and a Sphere to be geometries. In C++, we’d write this as follows:
/* inheritance example, c++ */
class Geometry {
box3f bounds;
};
class TriangleMesh : public Geometry {
std::vector<vec3f> vertex;
...
};
class Sphere : public Geometry {
vec3f center;
float radius;
};
In ISPC, we can’t directly do that – but if you look at how inheritance actually works in C++, all it means is that basically each “inherited” class first has all the members of its parent, and then adds some of its own. As such, what we can do in ISPC, is roughly the following:
/* "pseudo"-inheritance example, ISPC */
struct Geometry {
box3f bounds;
};
struct TriangleMesh {
Geometry super; // "inherit" all parent fields
vec3f *vertex;
...
};
struct Sphere {
Geometry super;
vec3f center;
float radius;
};
Now ignore for a moment that the “std::vector” in TriangleMesh::vertex is no longer a vector; and ignore that the syntax is a bit “cumbersome” – but the base concept of inheritance is there, at least for class members, and at least for single, straight-line inheritance: Every pointer to a derived class can always be cast to a pointer to the parent class, and then act just like the parent, which is exactly what inheritance means.
Before I go on to how to do the same thing for methods – and virtual functions – two quick notes on this:
Inheritance, polymorphism, and pointer-demotion
First, as part of how its polymorphism works C++ will automatically “demote” any pointers when calling a function expecting the parent class:
/* this works in C++ : */
void foo(Geometry *) { ... }
void bar(TriangleMesh *mesh) { .... foo(mesh); ...}
In ISPC, that same code won’t work, because even though a Mesh “acts” the same way as a Geometry it still isn’t a geometry from the type system. As such, you actually have to do this manually by either type-casting, or by passing a reference to the actual inhertied base class:
/* this is how it works in ISPC */
void foo(Geometry *uniform) { ... }
/* this C++ equivalent will *NOT* work: */
void bar0(TriangleMesh *uniform mesh) { ... foo(mesh); .... }
/* this *will* work */
void bar1(TriangleMesh *uniform mesh)
{ ... foo((Geometry *uniform)mesh); ... }
/* this will *also* work */
void bar2(TriangleMesh *uniform mesh)
{ ... foo(&mesh->super); ... }
Inheritance – and the need for discipline
A second note on this concept of inheritance is a personal “recommendation”: Be very, very careful, because with all the manual typecasting you take away any sort of error checking from the compiler… so this may easily lead to “funny” errors. There are, however, ways of making this easier – for example, in the above example the second solution (bar2) is somewhat less prone to errors than the bar1 – not much, but somewhat.
Second, we also learned (the hard way, of course) that a good – and above all, consistent – naming scheme is super important, in particular if the code grows in complexity. Initially, in OSPRay every hierarchy used a different name – one used “parent” for the inherited members, some used “geometry” or “volume”, some used “super”, some “inhertied”, etc – and keeping that straight turned into a nightmare. Today, we consistently use “super”, and that saves a lot of headache. The same topic will come up below.
Methods and “this”
In C++, you can add a method right into a class, which is very useful. Again, you can’t do that in ISPC, but if you look at it closely enough, a “method” – at least a non-virtual one – is nothing other than a static function with an implicit “this” that points to the class that this method lives in. As such, to emulate:
/* method in C++ */
class Sphere : public Geometry {
box3f getBounds();
vec3f center; float radius;
};
box3f Sphere::getBounds()
{ return box3f(center-radius,center+radius); }
becomes
struct Sphere { vec3f center; float radius; };
uniform box3f Sphere_getBounds(Sphere *uniform self)
{ return make_box3f(self->center - make_vec3f(self->radius),
self->center + make_vec3f(self->radius); }
Before we go on, again a few notes: First, there’s no method definition, because ISPC simply can’t do that.
Second, the biggest difference you’ll see is that the implicit “this” of the C++ variant is gone, and is replaced with an explicit variant of this (which we called “self”) – and we have to explicitly prefix each member access with this “self->” thing. But though somewhat more ugly, it does the trick.
Third, one thing that does require some attention is the use of “uniform” vs “varying”: It is actually pretty important whether “self” is a uniform or a varying pointer, or a pointer to a uniform or a varying base class. Though my own research compiler equivalent could handle this fact (and automatically generate code for all variants), in ISPC you can’t, so you may eventually have to end up writing at least two variants of this method – one with a uniform ‘self’ pointer, and one with a varying. That may be cumbersome, but it works.
One final note on method: Just like for inheritance, I can’t over-emphasize the importance of good and consistent naming. Initially we used all kind of schemes and names (“self”, “this”, “sphere”, “geometry”, “_THIS”, are all examples we used at various times) all over the place, and again, that turned into a mess. In the end, we always use “self”, and always use it as the first parameter … and switching to this has been very useful indeed.
Virtual Methods
Now that we have inheritance and methods, let’s go on to the really intersting stuff, which is “inheriting functions” – also known as “virtual functions”. Once again looking at how virtual functions actually work in practice, they’re nothing other than a function pointer to a method (which we now know how to do) that is inherited from the base class (which we now know how do to, too). So, let’s slightly extend our C++ example:
/* virtual example in C++ */
class Geometry {
virtual box3f getBounds() = 0;
};
class Sphere : public Geometry {
virtual box3f getBounds() override;
vec3f center;
float radius;
};
box3f Sphere::getBounds() { return box3f(center-radius ....); }
In ISPC, we first have to define a function pointer for the Geometry::getBounds() function – which is a method, and as such needs an implicit ‘self’, which in this case has to be pointer to a geometry:
struct Geometry {
uniform box3f (*getBounds)(Geometry *uniform self);
};
Note this doesn’t actually declare a function itself, and thus can’t take a function body – it merely states that this geometry will contain a pointer to a function/method with the given signature: a “self” (which happens to be a geometry) going in, and a box3f going out. We still have to write this function (like we’d have to do in C++), and – unlike C++ – have to actually set this function pointer to the right function – so before we talk about inheritance and “overriding” this function, let’s briefly insert the topic of “constructors”.
Constructors
In C++, a lot of what constructors do is fully automatic. Sure, you can overwrite them, but in the above example, the constructor would always and automatically assign the virtual method table (with all the function pointers to the virtual methods). The ISPC variant of course doesn’t have implicit constructors, so no matter which “getBounds” functions you might be writing, the pointer in that class will remain undefined unless we explicitly set them. So, let’s write a (explicit) constructor first:
void Geometry_Construct(Geometry *uniform self)
{ self->getBounds = NULL; }
In this example, we’ve set the ‘setBounds’ to NULL, which of course is only marginally better than leaving it undefined (though you could argue that as a virtual function it should be “0”, too). So let’s make this a bit more useful by “forcing” the caller to pass a useful function:
void Geometry_Constructor(Geometry *uniform self,
uniform box3f (*getBounds)(.....))
{ self->getBounds = getBounds;
This way, whoever wants to construct a geometry has to specify that function, which is useful.
Overriding Virtual Functions
Now, back to actually creating a userful subclass of geometry, and overriding its virtual function. For our Sphere, we can do the following:
struct Sphere {
Geometry super; //< note this "inherits" the bounds function!
vec3f center;
float radius;
};
Note now the “super” also inherits the Geometry’s getBounds() method – though to be more exact, it inherits the function pointer , not the method itself.
Let’s create there Sphere’s method we want to override with:
uniform box3f Sphere_getBounds(Sphere *uniform self)
{ return make_box3f(self->center + ........ ); }
and write a constructor for the sphere object:
void Sphere_Constructor(Sphere *uniform self)
{
// first, make sure the 'parent' is constructed, too
Geometry_Constructor(&self->super);
// ... then 'override' the getBounds method
self->super.getBounds = Sphere_getBounds;
}
… or, using the constructor that expects afunction pointer, it’ll look like this:
void Sphere_Constructor(Sphere *uniform self)
{
Geometry_Constructor(&self->super,
Sphere_getBounds);
}
Aaaaand – as soon as you try this out, you’ll immediately see that this won’t actually work, because the signatures of Sphere_getBounds doesn’t actually match the signature of the Geometry::getBounds pointer: the former expects a sphere, the latter a geometry, and as stated above, ISPC does not have C++’es automatic pointer demotion.
As such, you have two choices:
- you can give the derived methods use the signature of the parent, and (manually) upcast the “self” pointer to the derived class’es type; or
- you insert the right typecasts for the function signatures.
As an example of method 1:
/* Option 1: using the signature of the parent class */
void Sphere_getBounds(Geometry *uniform _self)
/* note how 'self' is a Geometry (not a sphere), just as
the parent's getBounds method type would expect */
{
// first, typecast 'us' to our real type
Sphere *uniform self = (Sphere *uniform)_self;
return .... self->center ....;
}
And, for completness, here an example for method 2, using some typecasts to make the code less ugly:
typedef uniform box3f (*Geometry_getBounds_t)(Geometry *uniform);
/* the Sphere::getBounds, with its own type */
void Sphere_getBounds(Sphere *uniform self) { ... }
/* and use of that in the constructor */
void Sphere_Construct(Sphere *uniform self)
{
Geometry_Construct(&self->super,
(Geometry_getBounds_t)Sphere_getBounds);
}
Once again, which of the two options you go with doesn’t matter too much, as long as you’ll remain consistent (else it does turn into a veritable nightmare). I personally prefer method 1, because the typecasts “feel” a bit more localized, but that’s probably only a matter of taste.
Calling Virtual Functions
With all that, the actual process of calling a virtual function is pretty mundane:
void buildBVH(Geometry **uniform geoms, uniform int numGeoms)
{
...
for (uniform int i=0;i<numGeoms;i++) {
...
bounds[i] = geom[i]->getBounds();
...
will do the trick just fine. Where it does get a bit more tricky is if we’re not dealing with a varying “warp” of object instances: ISPC can actually call a vector of function pointers just fine (it’ll implicitly and automatically serialize over all unique instances of it); but at least the way we’ve done it above the ‘self’ parameter of each such instance expects a uniform, so just calling a varying’s worth of getBounds()’s won’t work.
Option 1 for this is to actually implement each virtual method twice – once with a uniform self, and once with a varying …. but that’s a lot of work, and ugly. Instead, what we did is go with – as always in such cases – the “foreach_unique to the rescue” option, and serialize explicitly:
void Scene_postIntersect(Scene *uniform self, varying int geomID)
{
Geometry *varying geom = self->geom[geomID];
/* geom is a *vector*, so we can't just call geom->getBounds(),
at least not with geom as a parameter to it */
// instead: serialize over unique geometries
foreach_unique(uni_geom in geom) {
... uni_geom->getBounds(uni_geom);
}
}
The end … ?
Oh-kay – guess that was a much longer post than expected, but still hope it’s useful for those setting out to write non-trivial ISPC programs themselves. (Or, of course, those trying to get a better idea of the concepts behind OSPRay!). If you are interested in having a closer look at those concepts used in practice, by all means go and have a look at the OSPRay sources, at https://github.com/ospray/OSPRay (and in particular, the class hierarchies of Geometry, Renderer, etc) – in fact, it’s all over the place in OSPRay.
Most of what I described above will sound trivial to those that have done similar coding before; though I fear those that haven’t will still struggle, because truth be told, it’s not as easy as doing C++ with “real” C++. That said, the concepts described above have been immeasurably helpful in OSPRay – in fact, I couldn’t imagine OSPRay in any other way, nor could I imagine how to do it in any other way other than with “real” C++.
As such, I hope this has been useful…. and as always: Comments/Criticism/Corrections … all welcome!
Ingo's Blog 