| ▲ | Gaussian Point Splatting(momentsingraphics.de) |
| 134 points by ibobev 6 hours ago | 46 comments |
| |
|
| ▲ | keyle 5 hours ago | parent | next [-] |
| It will be interesting to see the first AAA game that uses these methods instead of rendering a 3D world. Even if made from CGI worlds, it would be a very interesting approach and with somewhat predictable performances. Reminds me of Ecstatica [1], a 1994 game that had intense visuals with a very odd/different rendering engine made of 3D ellipsoids; in a way really crude splats in gouraud shading. [1] https://ecstatica.fandom.com/wiki/Ecstatica |
| |
| ▲ | jayd16 5 minutes ago | parent | next [-] | | Bladerunner: Revelations used a similar technique to bake down large CGI worlds with expensive lighting into something that ran on a Pixel 1 at VR specs. Its honestly really very hard to work with this stuff because you ultimately need to be able to meshes inside these scenes triangle seas and you need to do it in a way that plausibly fits in the world. You can't have unlit characters walking around a baked lit scene and have them fit in. That's just from a visual design perspective. You also always want to have bounce light from your dynamic things onto the baked scene and depending on the tech, you might not even be able to spatially place a dynamic thing and have it properly occlude what splats it needs to occlude. As is, its a niche technology for games. That might change one day. https://github.com/googlevr/seurat
https://www.youtube.com/watch?v=Pf5Q3bvXj8E | |
| ▲ | cyber_kinetist 4 hours ago | parent | prev | next [-] | | Note that the first published work of rendering Gaussian Volumes was in this 1991 paper (https://articles.tomasparks.name/publications/Westover1991.p...) - so 3DGS is really a rehash of an old method from the 90s! The contributions of 3DGS lie in how fast you can make them in modern GPU hardware (tiling + sorting with threads), and how to make the pipeline differentiable so that you can fit the Gaussian splats with photogrammetry data. Similar to the history of deep learning, it became technically feasible once the GPU hardware was powerful enough. | |
| ▲ | sqrt_1 4 hours ago | parent | prev | next [-] | | There was this FPS demo recently
https://playcanv.as/p/qxGSuzYq/ People have also converted some small sections of Unreal 5 demos into splats
https://superspl.at/scene/692c4f91 Or perhaps use a real world scan - it was suggested this one would make an ideal setting for zombies
https://superspl.at/scene/6359774f | |
| ▲ | grumbel 4 hours ago | parent | prev | next [-] | | Many years ago there was a game called Casebook[1], a small little detective game where you investigated rooms for clues. But unlike similar FMV games where you jumped from point to point, it had photorealistic environments that could be smoothly walk around in, much like later lightfield or gaussian splatting experiments. [1] https://www.youtube.com/watch?v=o-VAaC5BgVE | | |
| ▲ | cubefox 3 hours ago | parent [-] | | Any idea on how they achieved this? | | |
| ▲ | Cieric 2 hours ago | parent [-] | | I can't say I know how they actually did it, but taking a look at the trailer I can point out that it looks like the spaces are confined and your character is on rails. I'm mainly going off of the instant direction changes that don't appear to be 45 degrees off from the camera direction. Once it's constrained down to a single line/path you could do some wild things like cube mapping a video, where the position in the video is tied to the characters position. I can't say I know how they would take that video though, my best guess there is the scenes are constructed in 3d software, just it was to expensive for real time rendering. | | |
| ▲ | cubefox an hour ago | parent [-] | | Cube mapping a video sounds plausible, this is commonly known as 360° video. Putting the camera on rails (though I don't really notice rails in this case) and tying the video playback speed to the speed of the rail movement has also been done in the past in some pre-rendered PlayStation games, though without cube mapping. But I think it's not pre-rendered in this case. It looks far too realistic for a game that is at least 17 years old. My best guess: they captured the 360 degree videos with a real camera (stabilized in some way) and edited the equipment out frame by frame. |
|
|
| |
| ▲ | avaer 5 hours ago | parent | prev | next [-] | | This is "rendering a 3D world". It's basically the exact same techniques that traditional rendering uses, just with a different primitive that's not triangles. Everything else pretty much carries over. If you mean the technique of splatting specifically, Dreams for PS4 [1] is prior art. If you mean pre-rendering, there's Myst and games like the original FF7 for PS1. [1] https://en.wikipedia.org/wiki/Dreams_(video_game) | |
| ▲ | modeless 3 hours ago | parent | prev [-] | | Dreams for PS4 used point splatting and has a very unique look as a result. The splats were created from distance fields instead of being scanned, so they don't look like modern gaussian splats. They have a painterly look instead. https://youtu.be/2ltgkcoQzow |
|
|
| ▲ | boppo1 an hour ago | parent | prev | next [-] |
| I really wantt to get into splatting and I have the tools: good camera, v comfy in blender, comfy with graphics programming ideas, 4080. But I haven't found a good 'all in one intro' to it yet. Possibly because I'm foss-biased and have dismissed proprietary options. But does anyone know of a good 'vertical tutorial' on this stuff? |
| |
|
| ▲ | samch an hour ago | parent | prev | next [-] |
| It seems like there are fairly regular posts on HN about splatting, and most appear to be fairly technical or proof-of-concept level. While the outputs look nice, I’m not sure that I could distinguish them from a nice ray-traced scene. What I think I’m missing is the “why?” of splatting. What are the material benefits of this area of research? |
| |
| ▲ | jerf an hour ago | parent [-] | | At the moment, combining your statement "I’m not sure that I could distinguish them from a nice ray-traced scene" and adding "your graphics card can move through them in real time so cheaply that it can easily be used as a component in other tech even at high frame rates" covers it pretty nicely. There's some research into how to make them move or do other things they don't do very well, but the fact that you can swoop through them in real time on cell-phone level of power means they fit a lot of niches. Plus the fact you can "record" them from a real-world physical environment without ever having to "model" it opens up a lot of utility too. Personally I suspect they are getting a bit more attention then they "deserve"; people aren't talking about their weaknesses very much and I think that's resulting in some overexcitement. Some of the "we can replace everything with splats!" reminds me of the people who still don't understand why "if GPUs are thousands of times faster than CPUs why don't we run everything on GPUs?" is basically not even a sensible question. I don't see them as ever being the foundation of a graphics stack, but they definitely have a place as part of a well-rounded menu of techniques that can be brought to bear on a wide range of problems. |
|
|
| ▲ | HexDecOctBin 5 hours ago | parent | prev | next [-] |
| Can someone point to a resource/tutorial for learning point splatting (the 90s rendering technique)? Gaussian Splatting has completely over taken the search results, and the original technique is now near impossible to find. |
| |
|
| ▲ | sorenjan an hour ago | parent | prev | next [-] |
| When looking at their linked interactive viewer it looks like they need 128 spp for the image quality to equal 3dgs. Maybe you can reduce that with some temporal tricks and noise reduction filtering, but that's still a lot of samples. |
|
| ▲ | phrotoma 5 hours ago | parent | prev | next [-] |
| I love this site design. It uses the entire width of the monitor rather than a slender column of pixels down the middle with large blocks of unused space on either side, with a font for my old man eyes. <3 |
| |
| ▲ | zokier 5 hours ago | parent | next [-] | | > It uses the entire width of the monitor rather than a slender column of pixels down the middle with large blocks of unused space on either side Umm on my machine it has 560px margin on both sides with the content being only 474px sliver in the middle? | |
| ▲ | simonklitj 5 hours ago | parent | prev | next [-] | | Imo they need to pad it just a bit. My scrollbar overlaps. | |
| ▲ | docheinestages 5 hours ago | parent | prev [-] | | Maybe use Tampermonkey? |
|
|
| ▲ | djmips 4 hours ago | parent | prev | next [-] |
| Could this be a new direction for Google Streetview perhaps? |
|
| ▲ | Epitaque 3 hours ago | parent | prev | next [-] |
| Did not read the paper (sorry) but I wonder how this compares to mesh splatting (https://meshsplatting.github.io/). I feel like mesh splatting can produce higher quality results because triangles are very good at representing sharp features, and gaussians aren't. |
| |
| ▲ | dpark 2 hours ago | parent [-] | | But only in the same sense that triangles are bad at representing curves, right? It seems that’s a wash. |
|
|
| ▲ | MattCruikshank 3 hours ago | parent | prev | next [-] |
| My dumb idea... do outdoor scans, and then convert the contents into 1m^2 blocks... And then, just dumbly stitch them together. Kind of like Minecraft... but with user-generated gaussian-splat blocks. |
|
| ▲ | cyber_kinetist 5 hours ago | parent | prev | next [-] |
| Really nice idea for 3DGS rendering - though the main problem is the noise (an unfortunate issue for all Monte-Carlo based methods). I think future papers would probably continue improving on this method and focus on how to sample the points more efficiently while being unbiased (similar to how ray-tracing solved their performance issues). Or maybe... we can just add a deep-learning based denoiser and call it a day! |
|
| ▲ | lucamark 5 hours ago | parent | prev | next [-] |
| This feels like Monte Carlo rendering applied to rasterization. I'm wondering if it's a brand-new or a well established methodology |
| |
| ▲ | pixelesque 5 hours ago | parent | next [-] | | It's not new - that was sort of my point with my other comment. At least if it's progressive (so refines and resolves over time), this has been done with pointclouds in the VFX industry in GPU shaders for years in terms of stochastically drawing different points so eventually the whole point set gets rasterised to a fidelity threshold. | | |
| ▲ | lucamark 5 hours ago | parent [-] | | ookay, thanks for the clarification! So, the interesting part here seems to be the 3DGS-specific opacity correction and GPU workload mapping. Am I wrong? | | |
| |
| ▲ | avaer 5 hours ago | parent | prev | next [-] | | Monte Carlo in 3dgs is established enough that Spark [1] has been doing it for a while in the browser. https://github.com/sparkjsdev/spark | | |
| ▲ | cyber_kinetist 4 hours ago | parent [-] | | Cannot find anything related to Monte Carlo methods in the source code. I thought Spark implemented a conventional 3DGS pipeline with LoD optimizations (And it seems they do the sorting on the CPU using Rust/WebAssembly because of WebGL limitations) |
| |
| ▲ | convolvatron an hour ago | parent | prev [-] | | that goes all the way back to the Kajiya rendering equation https://en.wikipedia.org/wiki/Rendering_equation |
|
|
| ▲ | DamnInteresting 3 hours ago | parent | prev | next [-] |
| Video overview of the technology: https://www.youtube.com/watch?v=X8yRlA7jqEQ Ordinarily I don't prefer video, but the visuals are helpful here. Also, an online interactive, but it seems to only work in Chrome: https://superspl.at/scene/ff1d0393 |
|
| ▲ | praveen9920 6 hours ago | parent | prev | next [-] |
| Sorting the gaussians is the compute heavy part in gaussian splatting. So, Im guessing this will give only marginal improvement in terms rendering speed. |
| |
|
| ▲ | cubefox 5 hours ago | parent | prev | next [-] |
| Their point splatting method is orthogonal to level-of-detail rendering (they reference a few papers which try to do this), so both point splatting and LoD could be combined in the future for an even greater performance gain during rendering. They already implement occlusion and frustum culling. Point splatting does introduce a lot of noise though, and their denoiser introduces ghosting, but they say a more sophisticated denoiser would give considerably better quality. |
|
| ▲ | pixelesque 6 hours ago | parent | prev [-] |
| > millions of threads Really?! What OSs can handle that many native threads? Also, this seems quite similar to stochastic progressive drawing of pointclouds for realtime that has been done for > 15 years in the VFX industry with GPU shaders in a tiled/bucketed fashion, unless this isn't progressive maybe? (The fact it's been accepted for Siggraph likely indicates it's slightly different). |
| |
| ▲ | Calavar 6 hours ago | parent [-] | | I believe they mean GPU threads. Plenty of cuda files in their repository. | | |
| ▲ | pixelesque 5 hours ago | parent [-] | | Fair enough, but that's then only absolutely max 1024 threads per SM, which wouldn't get anywhere near 1 million, given 5090 only has 192 SMs... Future proofing I guess... | | |
| ▲ | cyber_kinetist 5 hours ago | parent | next [-] | | You can launch much more logical threads than the available physical threads. The GPU scheduler will automatically dispatch the work to the SMs. | |
| ▲ | ks6g10 2 hours ago | parent | prev | next [-] | | Just like 2 threads can execute on the same core at the "same" time, i.e. no synchronization, the same is true for GPU threads/ thread groups. | |
| ▲ | zipy124 5 hours ago | parent | prev [-] | | I guess they never say that they execute at the same time technically haha |
|
|
|