(sorry for the delay, this is a bit outdated – this was apparently stuck in my Draft folder, and never sent out 
)
Finally …. I can share this paper.
I’m actually super-happy that this paper is finally out – I’ve now been working on this particular problem/idea – on and off – for at least seven years (the first rudimentary steps on that happened when I was still at Intel!); it just took forever to actually get it to where it really worked. Part of the reason this took so long was that the first ideas were too complicated to work well, and the project needed several different breakthroughs/heureka-moments to actually work. Another is that the project has undergone at least four complete “rewrite-from-scratch”es because I kept on finding new abstractions and cleaner ways to do things as it went (and the partitioners have been rewritten more often than that, too!). Yet another is that someway half-way through I switched from doing everything on CPUs to doing everything on GPUs, and that requires some additional re-design and re-writes, and also changed some of the parameters of what the system had to cope with – one one hand that was great because one needs way fewer nodes because one is no longer limited by tracing, shading, and texturing speed … but using GPUs also means that the memory limit is both lower and harder than on the host, and in particular, that communicating between different GPUs is a bit harder than sending some MPI messages between CPUs. In theory, MPI can send data directly between different GPUs, even with RMDA etcpp (in fact, that’s awesome tech!); and yes, I’ve seen that work in practice. The problem, though, is that setting up a system to allow that isn’t exactly trvial (to say it mildly); some Supercomputing centers etc have that (mostly) down for the kind of HPC codes that their users run, but if you think about just renting a few nodes in the cloud and compiling your own MPI …well, think again.
Anyway – the “final” breakthrough came when I finally sat down and built my own stone-soup cluster in my own basement, using lots of left-ofter hardware, some RTX 8000 cards donated for the purpose by NVIDIA, and a ton of additional parts I ended up buying …. and once I was actually able to interactively run stuff that project finally managed to get together some time late fall, and the paper finally got written by the end of the year, just in time to be submitted to Siggraph.
Now as it turns out Siggraph decided to reject it – not complaining here, just saying that I got a bit of a bad foreboding when after I tagged everybody from NVIDIA, everybody from Intel, and everybody I had recently collaborated with there weren’t that many people left that would be the kind of experts in ray tracing or data-parallel rendering you’d want to review that paper. Turns out two reviewers really liked the paper (the external ones, if I had to guesss), but two others didn’t, which apparently was enough to kill it. I’m also not going to complain about the reviews from said two reviewers (just saying that no, the paper is not about “BVH building”, and yes, even if it had been I am aware of some of the more recent papers on BVHes…. anyway, I digress). Anyway; I can also feel for these reviewers – there’s a paper you’re clearly not an expert in, you have no idea how to judge it, and in case of doubt you’d rather reject it than being afterwards asked how you could possibly have accepted that. So anyway, paper got rejected there – that threw a giant wrench into a total of three different follow-up papers that were/are already in the works, but hey, that’s the nature of the peer review process – sometimes you’re lucky, sometimes you aren’t.
After the Siggraph reject I uploaded the paper to ArXiv (https://arxiv.org/pdf/2204.10170.pdf); then resubmitted to HPG, where the reviewers were a bit more favorable to it, and where I just uploaded the final version this weekend. I could of course have already shared that paper after it was uploaded to ArXiv, but didn’t want to cause any confusion among the HPG reviewers if they read any Tweets or Blogs while reviewing the paper, so decided to wait ’til the final version.
Ingo's Blog 