Touhou-Project.com
patreon
Tables are for squares
🕑 Added 2019-12-24 16:46:23 +0000 UTCHey all, hope you’re enjoying the holidays!
I’ve finally gotten around to finalizing some of the things I’ve been working on. I had put off the work until both NanoWriMo was over and things on my end quieted down.
So here’s a bit of preamble: a couple of months ago I tried to do triage on what was most urgent on the site. While taking stock of what would need to be done first to have a solid foundation for sexy new features, I also came to the conclusion that any meaningful change needed to have the code base way more polished and refactored. Combining that with a few things I wanted to try out, I created cupcakes (you may have seen it already if you hang out on Discord). It should be seen as a test area where I may try new things or just mess around for the heck of it.
That brings me to the big changes that have happened this time around. The actual HTML structures of threads and boards have been altered significantly. Partially rewritten, mostly simplified and overall modernized. The Kusaba X code had a lot of antiquated ideas regarding how data is displayed, including the overuse of tables. I assume that the decision was made so that the proportion of elements were kept more or less constant but it’s a pain to deal with things like images and text within a table as if you want to add or remove something, the whole thing might just end up looking awful. Tables are great for actual data tables but not so much for fanfiction with girls wearing silly hats.
Replacing tables with more general things like <div> and <span> allow for greater flexibility on how the data shows up across a range of devices and screen sizes. Applying style sheet rules to get posts to look exactly as wanted is way easier, too. Finally, this also simplifies some things on the script-size when we’re doing dynamic manipulation on data in your web browser.
In short, what has happened is that the site should now look and behave more consistently. There are fewer special rules needed for mobile devices and a much more simplified code base which means that adding new things on top ought to take much less effort. It’s put me back on track for some mobile-centric features that have been on my wishlist for a while. But more details on that some other time!
So, if this sort of stuff is important for the future of the site, why did it take me this long to get around it?
Since the changes were fundamental, in the most direct sense, there were many components that were interconnected and would be affected by any change. So it made sense to go somewhat big since I’d have to change everything else anyhow. This is why I needed to focus exclusively on it as mapping out what needed to get done, devising the best approach, actually coding, then testing every other affected part to check for breakage or unexpected behavior (easily at least third of the code base) took a very long time and not much of it could be automated.
I’ve talked about it before but after pulling data from the database, pages are constructed by passing this information through a template which outputs a finished result. There’s about two dozen templates that are used for different things on the site. The main ones that needed changing were the one that did part board pages (e.g. for thp.moe/th), the one for threads and the one for archived threads. Those three are relatively similar in structure for obvious reasons with special cases and information that’s needed for their own quirks.
Now, aside from updating these templates to the new template engine and adding a few features over the years (like spoilers and handling webms/thumbnails better), they more or less work the same way that they have for the last 10+ years. That is to say that it takes a large amount of data, is put into a loop where a post is built at a time and then it’s output. Because the original developers were so very talented, the templates are basically duplicated with half of it being for the first post that starts the thread and the rest for any replies. Starting the see the problem? The mere fact that there’s a segregation implies that these posts have different rules and elements to them.
This brings us back to tables! All replies used to be constrained in a table, while the original post was something else and had its own rules for appearances. What I did was to vastly simplify the code by making nearly all the structure common. There’s only a few checks for very particular things (like adding an extra class or having data on when the thread was last bumped) which were unavoidable but it’s clearer to understand and modify the code now. There’s fewer than 200 lines of code in each template now, as opposed to nearly double before. I won’t have to waste time duplicating efforts when adding things nor thinking about special cases for the bloody replies that were table elements with columns and rows etc.
Of course, you have to be careful that the site looks and behaves in more or less the same way after you do all that. That requires a lot of testing to see if there are mistakes or edge cases. Not only that, this is when all the other things that interact with something so basic come under scrutiny.
The way the site looks (colors, font sizes, padding etc) usually depend on style rules, mostly found in CSS that’s loaded up. These style sheets have a structure and syntax of their own, applying changes to elements that match their information. It follows then that any changes to the foundations would require the things on top to also be changes. Getting everything to look nice, or at least the same as before, then takes effort.
Like I said earlier, this project made me realize that I might as well go big. In terms of CSS this meant basically rewriting most of the styles (think chernobyl, photon etc). Why? For starters, they all duplicated a lot of information. Each style sheet had entries for every element modified, even if they had them in common with one another. As all style sheets are loaded when a page is loaded, this meant increased file sizes and resource usage which still matters on the ever-popular mobile segment of our userbase. Even something silly like changing an existing rule to make the margin of a post a pixel larger meant applying that same change to 6-7 files.
I ended up offloading most of those kinds of things to a single common file and leaving only a few things like colors up to each individual style sheets. This also meant that I could determine a standardized font that would be consistent across the whole site and not be a hodgepodge depending on the style sheet that was loaded (and system fonts). While I was at it I made quite a few tweaks to the site’s appearance to provide a more uniform experience even if you’re on a phone or tablet. Keep in mind that all this work involved maybe a thousand lines of code deleted, modified or added coupled with necessary testing to make sure things were working as intended.
The front page also had to undergo alterations of the templates and CSS but, as it’s less complex, it took less time to get that sorted out. Still, it was important to get done as having a firmer foundation to work off makes future changes there easier to do.
Now, the last big part of this fundamental change involved the scripts run on the site. Every option in the ‘settings’ menu qualifies as do a few other things like the timers and clicking on a post number and adding it to the post box. As these also depend on the HTML structure base to identify which parts to play around with they needed checking up on. Fortunately, since I pretty much created the main script file from scratch in another big overhaul years ago, there weren’t very many surprises. Only a few things needed some rewriting to deal with the new structures, mostly to deal with edge cases.
I spent more time fixing a few outstanding bugs, like post previews firing too quickly on mouseover, and adding in minor quality-of-life things such as a justified text option than with actually making the rest of the code compatible. I managed to simplify a lot of the mobile-specific script stuff, since many of the special rules weren’t needed anymore thanks to both the unified HTML and the overhauled CSS. It’s a good launching point for future mobile features and I expect to be able to remove even more special code in the future.
Oh, the asynchronous requests did need an afternoon or two of work to make sure they worked correctly. These are things like pressing the expand thread button or the automatic adding of posts to an open but also the previewing quoted posts and actual post previewing. The main issue with those weren’t the script-y bits but the response that was parsed back into a format that’s added to the HTML dynamically. As the underlying structure changed, it was important that previewed posts or expanded threads all followed the new formula. Not difficult work to do after all the initial step of the HTML templates but it still is time consuming as it needs the go rigorous testing as well.
There’s still work to be done with those functions since I think I can consolidate a lot of repeated code into a single file but that’s a relatively low-priority task and I preferred to have all these changes live before the end of the year.
A lot of this definitely is behind-the-scenes sort of stuff that won’t really be noticed but there are changes that are subtle but likely noticed by eagle-eyed user. I made a compromise when consolidating op/reply structure and now the thread information (subject, name, time) is always over any thumbnail and the file information. This previous varied on whether or not it was OP or the length of the various bits of text. Likewise, I updated the time formatting to have an extra space between the day of the week and the rest of the time. This is so that it breaks the line cleanly on smaller screens without loss of information.
Fun fact: I applied that time change retroactively to the archives and so I wrote a quick script to scan old threads and replace things. As I’ve already got experience doing just that for spoiler images, webms and a other things I’ve written up about in the past it took me very little time to get it done. Yes, even MiG now has the time display in the new style.
This is likely the last one of these technical posts that I’ll have for the year. I hope that it’s been an interesting read for you and thanks for taking the time to read my technical ramblings. There’s a lot of plans for the future and I look forward to sharing them as they mature.
Until next time, take it easy!
