AnduoGames
patreon
Anduo Report July 21st, 2023
🕑 Added 2023-07-22 07:38:38 +0000 UTCHello little heroes! We are reporting on our lewduty, this week with personal experience in development, sprites and Lovense partnership!
THIRD CRISIS
Sempiturnon:
Hey there!
I am the person compiling, sometimes writing extra things for these Patreon reports both for Third Crisis, Rosas Are Red and Erokin!
The little banners you see on Discord and here are also made by me:
We recently teamed up with Lovense to bring you much more interactivity in Third Crisis than ever with the Lovense toy support.
If you have not read about it yet, you can here.
You can get Lovense toys for a whopping 55% off until August 1st. Here is our affiliate link.
We get 10% from the sales, so in a way you enjoying yourself will also bring back a bit to us. ♡
JuggerDown:
JuggerDown made this amazing Jenna's dialogue sprite!
Aine:
Aine has been as busy as ever making in-game sprites and making sure the sex scenes look extra crisp and ready to party!
Rated:
Rated has been revamping some previous cutscenes to make sure they look their best for the games health updates!
Vils:
Hey all! Vils here. Not much to show this week, because I’ve been taking a little break since the release fiasco we had this month. What I can do is go into more detail on the chain of events that lead up to the latest version of the game being unplayable for a few hours.
The Titty Bug
Chapter 1: RemoteSettings
Alright, let’s go back to the beginning. 0.55.0 just got released to Patrons, and I was getting ready to build the Steam and Public releases. While setting up our RemoteSettings variables for the patron release, I encountered this fun little curveball from the lovely people over at Unity.
So it became abundantly clear that we would need to do a 0.55.1 patch release to go in and replace our current RemoteSettings implementation with Unity’s new RemoteConfig so we can fix the ingame announcements to actually work again (thanks Unity).
Obviously if we were going to do a 0.55.1 anyways, we could take a couple extra days to go in and add additional fixes, especially to the new Lovense implementation from the bits of feedback we had already received. This resulted in the additional changes we did to the Lovense settings menu as seen in the screenshot below:
After a bit of doing, I finally got RemoteConfig working with this lovely set of test variables to make sure ingame announcements worked.
While we were finishing up 0.55.1, Rated had just finished a quick little image of Jenna with Lovense toys for our promotional material outside the game, but once it was done, we had the cool idea to actually put that image into the game as a pinup, and have it unlock if you connect a Lovense toy to the game. This is where the trouble begins.
Chapter 2: Cause and Event
It was a quick and easy implementation to get the pinup unlocking properly, thanks to the systemic nature of how TC is programmed, things are really easy to hook into with just a single line and then our systems take care of everything.
We rely very heavily on an event-based system, meaning we attach code to events that run that code only at certain times like let’s say when you press a button, when a level is loaded, or more case-in-point, when a Lovense toy is successfully connected to the game. We can add to these event-triggers from anywhere in the code, meaning we can make this really modular system that is really easy to add to.
The drawback to having systems triggered by systems like this, is sometimes we can accidentally trigger things that have unforeseen consequences (foreshadowing).
The Lovense implementation I made will set itself up to run before the first level is loaded, this is where a lot of our systems set themselves up to prepare for the game. Because it gets setup during this time, it’ll automatically connect to any Lovense toys when you launch the game if the options are correct and the Lovense app is running on your phone with toys connected to it. So if I want to catch the first time the game will connect to a Lovense toy, I have to hook into that ‘Toy Connection’ event before it gets run by the Lovense implementation’s startup code, meaning I have to run my pinup code before the ‘Before Level Load’ event is run.
Unity has just the event for me though, instead of BeforeSceneLoad, I can hook into AfterAssembliesLoaded, which is basically after all the script DLL files are loaded up and the code is ready to be executed before the first level starts loading (before BeforeSceneLoad is called, and therefore before my Lovense implementation sets itself up).
So perfect, I’ll just hook my code up to this AfterAssembliesLoaded event, and in that event: get the pinup I want to be unlocked, and then hook into our Lovense ToyConnected event to unlock the pinup when a toy connects. Easy! This is where the unforeseen consequences began (more foreshadowing).
Chapter 3: Why the fuck is this random shit erroring out?
After 0.55.1 was building and we began our checks on it, I began noticing this strange error popping up in the Windows build of 0.55.1. Jenna was completely invisible in all areas of the game, as if her character did not exist. At the time I thought it was just a build issue, because sometimes we get fun little random errors that occur which can just be fixed by building again.
So I did some digging and found this error:
One of her items (which had a logic object attached to it) suddenly decided to stop initializing properly due to… the logic object not existing, causing her whole character initialization to crash, leaving her character uninitialized and therefore invisible ingame. So I fixed up that error with a sanity check.
The (third) crisis was not averted.
What I mean by sanity check is basically checking that an object you’re about to do things with even exists. Trying to do something to an object that doesn’t exist results in a ‘null reference exception’ error because you can’t do anything to something that doesn’t exist, so the code will stop executing any further (and crash in most cases). In this case I added some code that checks if the logic object on the item actually does exist before the item tries to do anything with it, and if it doesn’t exist, don’t do anything. Now Jenna was visible again and the game seemed to be working as intended with a quick glance at the main menu and gallery and some playtesting of a save in the editor. Problem solved, right? Wrong!
Chapter 4: Unforeseen Consequences
With 0.55.1 ready to go, I began pushing all the releases out, including Steam/GOG (0.54.0) and Public (0.53.0) which were all running off the same codebase as 0.55.1.
After a long day of getting everything good to go and pushed out, I was ready to grab a nap (it was 10am and I hadn’t slept yet lmao), and that’s when the reports started rolling in.
At this point, bear and I went into full troubleshooting mode trying to find what was causing this titty bug to happen. Due to the nature of it and my (correct) intuition that this issue was somehow connected to the null reference exception from earlier, we started our search with Jenna’s Vibrosuit item:
During our search, I say something that is even more foreshadowing of the true nature of this issue.
While it wasn’t exactly the ScriptableManager that was broken, it did play a part, and the entire game was indeed broken.
Chapter 5: The Search
Our search was bringing us to conclude that the issue was from somewhere within FsmTemplateHolder.
This system basically handles almost all scriptable logic that gets run by Finite State Machines (FSMs) using Playmaker, a Unity addon that allows us to do really easy-to-use visual scripting to handle the logic of things like mission states, items, achievement checks, things like that which require design work, iteration, and ease-of-use. So instead of making bespoke code to handle the flow of the Throb Patrol mission or for the interactions with Jenna’s Vibrosuit, we can just make FSMs. Usually Playmaker intends for these FSMs to be part of the level and saved in it as objects within the level, but due to our needs for ‘floating’ FSMs connected to things like items and achievements that aren’t tied to any level, we would need a system that creates a ‘holder’ object that basically exists apart from the level structure.
The issue was that something was causing this ‘holder’ object to disappear, taking all the logic FSMs with it, and therefore causing almost the entire game to completely stop functioning.
There was another strange quirk to the titty bug that we were trying to investigate to help us find the core issue. In new games, the titties were out, but in loaded games, the titties weren’t out.
The reason why this was happening was because clothing items will save what visual state they are in, separate from any FSM logic (which did not exist anymore because of this bug we were searching for). Since loaded games had FSM logic in previous versions of the game that had already set these visual layers up properly and saved their states, it would display nicely, but in new games, the FSM logic had never existed and therefore never set the visuals properly. This confirmed to us that the FSM logic of the Vibrosuit wasn’t being created and wasn’t running, helping to narrow our search down.
And here was when we began to realize the origin of the real problem. The FsmTemplateHolder singleton did not exist, and was not creating a replacement for some reason.
Singletons are essentially an instanced object where there can only be one of that instanced object. In Unity, this pattern is useful when you need something to be instanced in the level as a GameObject, but accessible from almost anywhere in code. This makes it a perfect use for our FsmTemplateHolder, which needs to be a GameObject and exist as an instance in the game so it can hold all of our instanced FSM logics from our items, achievements, and whatnot.
So now we were looking at our Singleton and PermaSingleton classes, these are two sides of the same coin where a Singleton is something that can be deleted when a level changes, and a PermaSingleton exists outside of the level structure and NEVER gets deleted EVER. It always exists.
Here bear found that the ‘Instance’ variable was null during gameplay, i.e. didn’t exist. Instance is the variable which holds the instance of the singleton GameObject that holds all the FSMs for our FsmTemplateHolder class. This meant that somehow the PermaSingleton instance did not exist, which as I said before, should NEVER be the case.
At this point the sleep deprivation was setting in and I was getting absolutely tilted out of my mind at how strange and seemingly nonsensical this critical gamebreaking bug was being. We were combing through code that we hadn’t touched since basically the very first 0.1.0 release of Third Crisis.
So now that we had narrowed in on the main cause of all these strange issues, we began thinking of fixes. When a PermaSingleton was destroyed, it would stop another PermaSingleton from being created, because it would assume that the game is shutting down, and wouldn’t allow another one to be created to stop the game from leaving a ghost object in the editor (which freely swaps the game between Play and Edit mode). However, if we just made this behaviour only happen in the editor and call it a day, we could possibly still be seeing even more hard to nail down issues with logic just disappearing out of nowhere whenever this PermaSingleton decided to let itself be destroyed.
We needed to keep searching to find WHAT was somehow destroying our PermaSingleton.
Unfortunately due to how Unity handles GameObject destroying, we couldn’t get a callstack (list of method calls) to find what originated the call to destroy the object, meaning we couldn’t just stick a log into OnDestroy and then find what was destroying our singleton.
We found a post from someone that said that OnDisable would give us the callstack we wanted, so we tried that, but unfortunately…
At this point we went with the bruteforce route and stuck logs before every single Destroy call to see if that would help us find it.
Now we were narrowing in even more, it wasn’t any scripts that were destroying the singleton, it was something that the Unity Engine itself was doing that would destroy the singleton.
What bear found was that when our singleton’s OnDestroy method was called, the scene name was “”. Nothing. There was no scene loaded. Remember my explanation of which events I plugged into for the pinup? Our issue was happening before BeforeSceneLoaded was called.
We had finally found our culprit.
Bear explains the root issue nicely here:
Chapter 6: The Fix
The issue was that I made a ScriptableManager call in AfterAssembliesLoaded, so ScriptableManager sets itself up in this weird void before a scene is even attempted to be loaded, and it then sets up every ScriptableObject contained within it (Items, Characters, etc). Since items have logic attached to them, they then call FsmTemplateHolder to instantiate their FSM logic, which then causes FsmTemplateHolder to set itself up and create a singleton to hold its FSMs, and since this was done in the weird no scene void, once the scene starts loading, Unity destroys any GameObjects from the void, which caused our FsmTemplateHolder singleton to be destroyed, and since our singleton thought that the game was closing because it got destroyed, it didn’t allow another instance of itself to be created when it was needed, and since there was now no FsmTemplateHolder singleton instance, none of our FSMs could instantiate properly, meaning none of them worked, causing Jenna’s titties to be out when you clicked new game (and also completely breaking most aspects of the game in general like combat).
Here is the fix helpfully illustrated with an arrow:
All I had to do to fix it was move that line where I get the pinup to instead be inside the OnToyConnected event, which would be called along with everything else in BeforeSceneLoaded, which happens after Unity deletes every random floating GameObject in the void before BeforeSceneLoaded is called.
That’s the nature of bugs sometimes when you’ve got a lot of systems interacting with each other. The blessing is it’s easy to add things, the curse is that it’s harder to troubleshoot things when they go wrong.
Once we realized the problem and fixed it, we put out a status update in the Discord so you would all know what was going on while we waited for builds to finish:
I got all the builds setup and building (0.55.2, 0.54.2, 0.53.2)
And then once they were done, I started pushing out all the builds and finishing up the last bits of the monthly release checklist.
Then did the final announcement to let everyone know that the game was patched up and working again.
Chapter 7: The Post Mortem
Timeline
We put Third Crisis (0.55.1) up with a critical gamebreaking bug at 10am EST, took 3 hours to find and fix the bug at 1pm EST, then another 5 hours to get the 0.55.2 builds built and fully pushed out. All in all, the timeline of the issue recovery was about 8 hours once discovered.
The Good
We were able to find the incredibly elusive bug within a reasonable time due to our troubleshooting process and had it patched within the same day once we realized the gravity of the issue.
The Bad
This issue eluded our build pre-release checks and made it into 0.55.1 in the first place, thinking a simple sanity check fixed it instead of looking for the root problem.
Lessons Learned
We’ll do a few more intensive checks of gameplay itself instead of just quick checks to ensure the game launches and things like the gallery work. These checks need to be in the build itself before uploading. We do our more intensive playtests in editor and that doesn’t always catch issues like these that only happen in a build.
Our current pre-release checks are:
- Check that the game launches.
- Check that the gallery works.
- Check that saves work.
Our additional pre-release checks will now be:
- Check that inventory functions.
- Both in a New Game and Loaded Game.
- Check that combat functions.
- Check that dialogue functions.
This has been updated in the release checklist template and will catch bugs like the titty bug before release in the future.
The other lesson learned is to not stop at a quick little sanity check to just remove an exception when something strange and unexplainable happens (like logic somehow not existing when it is clearly attached to an item), especially not in a file that hasn’t been changed in years. When something unexplainable happens, there is usually a deeper issue at play.
Desva:
Hey, Desva here. We still are in the process of bug fixes and quality-of-life changes. With the help of some people in our Discord, we fixed so many typos. On this note, a special thanks to prisoner416. They went ham on the typos. There are likely still some more, so if you find them send them our way in our bug-report channel. For quality-of-life updates, I added a few additional interaction points for specific interactions, mainly when you are naked or wear Guard armor.
For TC:R, I went on a journey and looked into isometric/3d. We will likely stick to the same view as in TC. It’s all still in the early stage and can change tho.
ROSAS ARE RED
Zanzo:
Between fixing and polishing various things here and there, the cat gets a little more personality and now has different reactions for when it doesn’t want to eat, go out or be pet. Functionality-wise, the Cat Game is well underway and going forward, it’s mainly about presentation now.
As always thank you for reading and supporting us through development! ♡
Comments
Anduo Games
Sometimes firefighter scenarios as programmers sound like epic battle stories when recounted haha
DoctorDiscord
Holy shit. What a BATTLE.
Anduo Games
Heya Brad, TC:R in terms of the timeline in the story is a prequel! („• ᴗ •„)
Brad Meyer
TC:R is a remake or the sequel you were talking about in the last report?
Hope Bills and Bear coding had a extra day off after that bug incident. It was intense! All those errors because of that sneaky line. Congrats for finding that one!
