The Xbox had strong requirements for loading times. This is probably a linear (lin) record of how the data was loaded unoptimized from the disk. And just written to a file.

So in this file seek doesn’t do anything because seek kills the requirement of 45 sec per loading screen.

Instead the logic is as follows: check if a .lin file exists. Yes: open a handle to it and only read from it with fread, what ever currently is at the current file position. No: while reading any file write the read bytes to a .lin file in the order they are read.

This gives a highlyy optimized .lin file which can be read from the disk into memory, without creating a better dedicated loading mechanism.

So if your really would like to unpack this. The first file being read is most likely the key, as it dictates what comes next. If it is a level model, then the position of the player in it might affect which other files to load etc.

In short it’s not a file format in the classical sense, it’s a linear stream of game data.

▲

landr0id 14 hours ago | parent [-]

>This is probably a linear (lin) record of how the data was loaded unoptimized from the disk.

Yes, it's buried deep in the details but it's basically just every byte read being written in a linear stream to an output file.

I don't know which stage of grief this is, but since I wrote this blog post I've now ported my IDA debugger scripts to a dedicated QEMU plugin which logs all I/O operations and some other metadata. I tried using this technique to statically rewrite files by basically following DataLoad (with unique identifier) -> Seek -> Read patterns.

There's some annoying nuance to deal with (like seeking backwards implying that data was read, tested, then discarded) but I got this working. Unfortunately some object types encode absolute offsets in them that need to be touched up, so a couple of object types fail to load correctly in external tooling and the PC build of the game.

Now I'm just using this data to completely reimplement the game engine's loading logic from scratch using a custom IO stream which checks the incoming IO operation (seek/read) against what I logged from the game engine to ensure a 1:1 match with how the game loads data.

▲

WatchDog 13 hours ago | parent [-]

Have you done any analysis of what proportion of the lin file is being read in total?

You stated in the blog post, that your goal is to try and find unused content, however if as described, the file is just a record of how the game loads the data, then it won't contain any hidden unused assets, since unused assets would never have been read from the original unoptimised file, and thus never written to this optimized file.

	▲	landr0id 12 hours ago \| parent [-]
		I agree and don't think there's any unused data. For `common.lin` for instance my parser reads it basically to the end and there's some small amount of data that's unused. I never actually quantified the amount but I'm fairly certain it's <100 bytes. Probably a bug in there. The goal post has shifted so far beyond my original intentional at this point. The devs working on the EnhancedSC mod have a strong desire to port some Xbox assets/maps to PC, so I'm mostly doing it at this point as an attempt to help them out. *On second thought, there's definitely some unused scripting functionality. Script functions which are unused are still included in their parent classes and are loaded if the parent object is loaded, even if never directly called. Whether or not any of this is interesting is another story. Textures and models though will definitely not be present unless they're used in some non-visible way.