furarchiver
patreon
About the latest changes
🕑 Added 2023-05-15 09:41:56 +0000 UTCFurarchiver processes about 30 requests every second. That's quite a lot for something that I just operate in my spare time and to for free. It's at the point where it gets increasingly important to optimize things. As a service operator you want to complete requests as fast as possible, and this wasn't always the case in the past. The latest version addressed some of those problems.
The biggest problem wasn't even the number of people, but the things they do. First of all, a handful of people were automatically scraping the site, and some of them would do this as fast as they could. When I pushed the last update I made it harder to do so for them. This should leave more capacity for normal visitors. There's no reason to build bots for furarchiver anyways because I literally provide a zip file with all artist content.
The zip file was the next problem. A not insignificant number of users would start a zip download, and then eventually cancel it, possibly because they would eventually realize that the zip for some artists can be quite large. In the latest update I added a feature that causes the server to abort zip file creation if the user stops the download. In the past the output was just sent into the void, which is not a big problem because zip files are not that CPU intensive, but it still eats CPU time for nothing.
Finally, the bandwidth. As you can see from the screenshot, the server transferred over 600 GB in 3 days, and this number was much higher before the latest update. Visitor analytics have shown that most people don't download all artist content, but rather pick individual files from the gallery. In the past, the gallery would pull the image from the server and simply display it as a thumbnail. This used up a lot of bandwidth. Now, the images are resized on the server before being sent to the gallery page. Only when you click on a thumbnail will the full size image load. You can see this because it briefly shows the thumbnail until the full resolution has been transmitted. This change has cut the bandwith to about 1/5 of what it used to be. The resized copies are cached on the web server for reuse with other visitors.
I managed to apply the new resizing system to gif animations in a way that they're still animated.
