MeshiSoftworks
patreon
Dev Journal: Mod Loading Finished!
🕑 Added 2024-09-30 20:49:36 +0000 UTCAs soon as I started to add modding to the game, I ran into a problem. VT uses a lot of mid-sized textures to render its characters, and in order to ensure no visual glitches (Missing faces, etc.) these textures ideally all need to be loaded in within a single frame. The reason this is an issue is because Unity cannot display image files as-is, it needs to read the image and convert its data to a Texture first. While this process is very fast, it can still cause an ever so slight amount of lag, which is, in my opinion, unacceptable.
So, how do you get around this? My initial solution, and the solution that's present up to v0.11b, is probably the simplest and worst one. Just load all the images into memory and leave them there the entire time the game is running. This adds a long load time to the start of the game, makes it take way more resources than it needs to, and straight up crashes the game if you try to use too many mods at once, but it does let you load in images quickly.
This obviously needed to get solved, so I started experimenting with different methods of speeding it up. My first idea was to simply try and convert the images into Textures as needed, using the least compression and the fastest conversion process I knew of, but this still lead to that aforementioned lag. A 2048x2048 image just contains too much information to be processed in the needed amount of time.
My next thought was that, if the raw image was too big, I just needed to find a way to write the Texture to a new file and load that, instead. While this wouldn't let me load the Texture directly in a usable state, it would be considerably less data that I would need to load in order to create a new Texture (16MB -> 4MB). So, after spending some time trying to figure out FileStreams and BinaryWriters, I finally got it working and it actually functioned perfectly.
But now I had a new problem. For every single .png, a 4MB Texture was being created, ballooning the size of the mod to ~20 time its size. Moreover, you now had dozens of extra files to deal with, which is just a pain in the ass. I needed a way to both compress the textures, and store all of them within a single file.
In order to do this, I needed to create an archive file; calculate the size in bytes of a number representing the Texture's name's hashcode, its byte offset, and its size in bytes; write this number so I know the offset of my data; create my Texture with all relevant data; compress it using LZ4; write that data to a Stream; write the data I calculated the size of before to a header in my archive file; repeat for every Texture; and finally write all my compressed Textures. Then, I just needed to use the name, byte offset, and byte length I stored before to find the exact position and size of the Texture I needed to load and decompress it into a new texture.
And this took probably... 90% of the time. If anything was off by even a single byte, all of the data would become useless, so I had a hell of a time trying to get it all figured out. In the end, it worked perfectly, though! After being packed, there are no longer any load times for mods, and you can have any number going at the same time without any issues (Mods can be cut down to 1/4 of their size, too!)
Thanks for reading! Perhaps not the most exciting topic for a journal, but as someone who sucks at this kind of programming I'm really proud to have this done. As for the next update, I really don't have any good estimate about when it will be out, but let's say a month or so for now. In the meantime, if you want to follow along with development I will be continuing to release betas in the discord whenever I have new features to test. Thank you so much for your support, and I'll see you next week!
Comments
hentai-san030
Well thank you
Rowan Dunning
Yep, you can find them in the discord channel
hentai-san030
Wait there's mods for this?
Lucas You
Ah, it's the memory VS efficiency problem, as always. The later you solve it, the nastier it becomes. So it's great to hear that you found a good balance point in the early stage.
V-Ate
Always interesting to learn about the technical challenges and solutions like this.
kirin uchiha
I couldn't even begin to understand how to do this stuff haha, so honesty all this is pretty impressive work in my eyes Congratulations!
PAVAGA
Dark Magic neither Darth Vader nor Voldemort would touch with a 10-feet pole. I understood perfectly though, I'm a programmer as well.
Halivakari
That is extreeeeemely commendable that you figured that out. Good job! As a programmer myself, I actually found this log entertaining to see your thought process and solutions. Haha
