// HACKER NEWS — CYBERSECURITY
Anatomy of a Texture
Last year I had to write code which converted texture data between gaming platforms. Going into it, I seriously underestimated the complexity of texture memory layouts.
I'm currently working on a team at 505 Games porting a custom PC game engine to consoles. One of the many challenges faced has been around texture conversion. I knew there were a lot of complexities and subtleties to it. I recognized most of them in isolation. But, it wasn't until I had to write working code which handled all these details in tandem that the full complexity really sank in. And so, I thought this would make for an interesting blog post!
This blog post will explain the complexity of texture memory layout, along with why it is necessary. This will be done through the lens of someone trying to debug the calculation of memory addresses for each part of a console texture. Due to the subject matter, this post will have to get a little bit more technical.
I will assume an understanding of programming fundamentals, memory layouts, as well as familiarity with basic video game technology and digital imagery.
Throughout this article we'll use a 1024x1024 RGBA texture using BC7 as an example. For illustration purposes, let's use a simple wood texture.
Converting this texture between platforms requires us to keep track of a lot of details. For the purpose of this article I will explain the following: block compression, texel ordering, mips, and texture tiles. There are other details such as pitch, depth, and texture array index. These mainly require simple offsets which are slightly annoying but do not add any interesting theory, so I will ignore them.
Here is what that image would look like if we ignore all of these complications and interpret the result as a simple stream of color values:
We will primarily discuss most of these components in a platform-agnostic way. Where a distinction is necessary, we will rely on the view taken by DirectX 12.
The reason is performance. There's a number of orthogonal techniques applied, each of which complicate texture representation but improve rendering performance. Most (e.g. texel ordering & texture tiles) exist to improve cache locality. That is, doing our best to store data as close as possible to all other data we expect to need at the same time. Good locality drastically decreases the amount of time wasted waiting for memory transfers - which is one of the most expensive operations in modern computer hardware. While block compression also helps with locality, it primarily improves memory transfer speed by decreasing texture size in memory. Finally, we have mips which drastically reduce rendering cost by downsampling the entire texture in multiple steps ahead of time - allowing us to avoid the many expensive texture samples we'd otherwise need any time we want to average all texels in a surrounding area.
From the perspective of someone debugging texture loading across platforms, every one of these techniques is another wrinkle to keep track of.