// KOTAKU — GAMING
AI Hallucinations Are A Feature, Not A Bug
Since 2020, a rapidly evolving neural network technology called NeRF (Neural Radiance Fields) has been used to create 3D scenes from 2D photographs and videos. The most famous example is probably Luma AI, available to use in a browser, which can turn your mobile videos into geometric spaces. In the last couple of years, a big leap forward arrived with (and this is real) Gaussian Splatting, which dramatically speeds up the process through the magic of tiny balls. Now Nvidia is showing off a new prototype tech called ArtFixer which allows its AI to fill in the information gaps with what it calls an “open auto-regressive model,” generating what it imagines should appear in areas missing from the initial footage. Which all sounds neat, until you realize this is why we can’t have nice things.
3D Gaussian Splatting is a technique for creating an explorable 3D render from photographs or video with the ultimate result of making RAM more expensive. It has also been used for special effects shots in movies during the last year, and is arguably a way to more quickly render geometric structures for gaming, although right now it absolutely cannot do that. But the key takeaway points here are:
You’ve likely seen it in the wild: it was used in the recent Superman movie for rendering the holograms of Supe’s Kryptonian parents, and indeed in Sinners to allow Michael B. Jordan to interact with himself. It’s also less effectively used in music videos such as A$AP Rocky’s “Helicopter”:
And yeah, it looks utterly awful. But Nvidia reckons it has a solution.
3D scene reconstruction works great until the camera never sees part of the scene.
ArtiFixer from NVIDIA Research is an open autoregressive model that fills in the missing geometry that other methods leave blank.#SIGGRAPH2026 paper, code + demo: https://t.co/D9PX2OzbZf pic.twitter.com/AGQicvVKkW
Nvidia’s paper on its ArtFixer tool begins with a line that is absolutely incomprehensible if you’re not already AI-pilled:
Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas.
First, we train a powerful bidirectional generative model with a novel opacity mixing strategy that encourages consistency with existing observations while retaining the model’s ability to extrapolate novel content in unseen areas.
Sadly for me, I have now sort of fathomed what this all means, and it seems to break down to saying that current technologies are rubbish at hallucinating the missing spaces when trying to create a 3D space from flat images. For an imperfect example, think of the weird gaps you get in Google Earth and Street View, which is compiled from overlapping photographs taken by a 360 degree camera. While that’s starting with far more information than ArtFixer is designed to use, you still get weird anomalies where stitching goes wrong and the photographs don’t overlap properly. This would be an AI that could fill in those gaps with what it claims are photorealistic renders, although entirely based on what the software makes up based on its hallucinatory nature.