CSE 381 – Advanced Game Programming Loading & Caching Data

Real Games Have tens of thousands of “data files” Loading them one by one isn’t practical might not fit into memory together too much overhead loading individually What’s the alternative? resource files

Game Resources I.e. Game Assets Game data Art, animations, sounds 3D meshes, map levels, etc Must be loaded small Must keep quality Formats platform dependent

Using Resources Source Files Art: BMP, JPG, TGA, PNG, etc. Sound: WAV, OGG, MP3, etc. Meshes: X, COLLADA, etc. Video: AVI, BIK, etc. Game Resource Files ZIP, WAD, etc. Pack & Compress Game Application Layer Game Subsystems Renderer, Sound, Logic, etc. Resource Cache Get Cached Resource Read From Disk

Typical Game Estimates Total ~ Gigabytes of data Map/Level Data small and easy to compress 3D Object Meshes & Environments tens of megabytes for geometry 3D Mesh/Object Animation Data tens of megabytes Sprite & Texture Data hundreds of megabytes Sound, Music, Recorded Dialog typically largest component Video & Pre-rendered Cinematics most expensive

Resource Loading Objective Get in and out as quickly as possible Minimize search time on disk How? load large blocks together should be things you’ll likely use together order resource blocks carefully on disk according to use Ex: Group by Level Read a Block of Data for Level 1 Level Blocks and Common (shared resource) blocks

Packaging Resources into a Single File Avoids wasting space Makes loading faster Good for proprietary issues Store the data in the format you plan to use it In what format? WAD ZIP Etc.

ZLib Free, open source DEFLATE compression algorithm Typical compression ratios are 2:1 to 5:1 Can be used to compress many files into one file Table of contents stored at the end of the file

Zip File Internal Structure File 0 TZipLocalHeader File Data File 1 File 2 TZipDirFileHeader[0] TZipDirFileHeader[1] … TZipDirHeader …

One structure for each file in zip Name of file Type of compression Size of file before & after compression

The ZipFile Class How does it work? 1.Opens a ZIP file 2.Reads the directory into memory 3.Uses the directory to index to the rest of the file class ZipFile {bool Init(wstring &resFileName) bool ReadFile(int i, void *pBuf) FILE *m_pFile; char *m_pDirData; int m_nEntries const TZipDirFileHeader **m_papaDir;

#pragma pack(1)? What’s that? Continuous ZIP file memory Put around structs TZipLocalHeader, TZipDirHeader, and TZipFileHeader Why? disables some memory C++ optimization that might spread them out So? We want to stream them in large blocks Rule of thumb: Pack your structs anytime you define a struct that will be stored onto a disk or in a stream

The Resource Cache Sits on top of your resource files Manages memory Manages loading Predict resource requirements before you use them minimize cache misses

Types of Cache Misses Compulsory Miss happens when the desired data is first requested and loads for first time Capacity Miss happens when the cache is out of space and must throws something out to load in the data Conflict Miss system incorrectly determined data was no longer needed and threw it out Poor Resource Cache management causes Thrashing

GCC Resource Types Resource & ResHandle IResourceFile & ResourceZipFile IResourceLoader & DefaultResourceLoader ResCache

A Simple Resource Cache ResCache Maintains 2 data structures Linked list: Nodes appear in the order in which the resource was last used Every time a resource is used, move it to the front of the list Map find resource

World Design & Cache Prediction Options: Load Everything at Once Load Only at Pinch Points Load Constantly