gamersnexus
patreon
Massive Bench Overhauls & what we've been up to
🕑 Added 2024-07-15 00:30:16 +0000 UTCHey everyone,
We've been grinding away at the benches lately! I'm not sure how obvious it's been externally, but production volume has been way up and we've really been getting to work on the deep-dive content.
Other than the various investigations, we recently published an ITX case review, two ATX case reviews, the NH-D15 G2 review, and the Thermalright AIO review. Here's what we've been working on to facilitate all of those, as a behind-the-scenes update for our Patreon backers:
Test Bench Overhauls
We recently totally overhauled the following test benches with new methodologies and components:
CPU Cooler review bench
CPU review bench
ATX case review bench
CPU Cooler Review Bench
Changes we made here mostly included debuting the 14900KF with a 250W and 200W heat load for CPU cooler reviews. We did this following our 3D scanning content and "golden sample" coldplate video for coolers. This allows us to accommodate both Intel and AMD heat spreader styles (and die locations), which means we can now better observe if a cooler has a strength or fault on a particular CPU.
With the old AM4 bench, Mike would typically spend about one entire work week, or about 40 hours, putting together the tests for the cooler. The way we were able to add Intel testing without inflating the time requirement was mostly through clever automation. We've always had automated cooler testing, but my recent changes moved us from singular pass automation (which was occupied for 30 minutes at a time) to multiple-pass automation, including better synchronization of ambient logging data against those passes, which means that a technician can now fire off 3 tests back-to-back. With cool-down periods, this means the Intel cooler bench can fully manage itself for about 2-3 hours without human intervention.
The point at which Mike intervenes is to re-mount the cooler. We typically do 2-3 mounts and at least 2-3 passes per mount (occasionally a mixture of 2 passes + 1 pass for heavier load periods, then if there's variance, we do more).
The improvements I implemented to the cooler bench largely revolved around the ambient logging methodology and synchronization with the cooler thermal data, which means that our accuracy has actually improved to be closer to +/- 0.5 C, down from about 1.0 C previously. We still use error bars at +/- 1.0 C or 0.75 C, depending, to allow conservative room for error. It was a huge improvement to process flow and means we can now accomplish more in the same time, so instead of just cutting the workload, we filled the same time we were already using to accommodate both platforms. Mike has been thrilled working with the bench as it's made life a lot easier for all of us.
CPU Review Bench
I'll start with the data side, since that's more tangible for viewers. We've made these changes for our CPU tests:
Premiere and Photoshop have been added back in. These were temporarily removed pending some changes in Puget's suite previously
Spec Workstation has been added permanently, following popular demand from our audience during the Threadripper reviews
We have fully overhauled the games suite, including the addition of Dragon's Dogma 2 -- I'm excited about this one since it was such a CPU nightmare at launch
We have updated FFXIV to the new benchmark, F1 to the 2024 version, we have retired several games (such as Shadow of the Tomb Raider and, finally, GTA V -- for now, we may bring it back for the GTA VI launch)
Our process here has improved significantly as well. In just one work week and without any real OT spent on it, Mike and Patrick were able to get about 28 CPUs through our full test suite. This used to be something that'd take over a month to do. The big improvement here was actually to the management side. I started building-out a test matrix for the team with buttons and drop-downs in a spreadsheet, which allows clearer communication of who's working on what and when progress is blocked.
On the team side, Mike and Patrick have been running the tests. By their own choices, everyone on the team has a slightly different schedule (whenever someone joins, I ask what hours they'd prefer or if they want me to assign some). Patrick and Mike have anywhere from a 3-6 hour gap between their start times, depending on day, which works out great for everyone. For the team, that's the schedule they wanted. For GN, that means we can have around-the-clock coverage of benches (with me covering the remaining hours on the clock) without anyone being overworked. We've been excited about the new CPU testing because our capabilities have expanded significantly with the process and workflow updates.
A lot of this is sort of 'blah blah blah' company improvements stuff, so for what it actually means for the content, I'm thinking this:
We're going to experiment with generational charts in the upcoming CPU content, as we have too many CPUs to fit in the normal comparative charts anyway. Rather than just cutting them, I'll make some new/special one-off charts
We're going to have more workstation/production coverage
The new games suite has most our games being 2022, 2023, and 2024 launches
ATX Case Review Bench
We've been talking about the improvements here for a while now. The big change was adding a flow-through GPU, as we observed about a 5-degree Celsius increase in CPU temperature (at our heat load) with flow-through versus without. This benefits the GPU, but impacts the CPU. We wanted to reflect this in testing, as most modern cards are moving this direction (and it's a good design -- they should).
We're also using TG Kryosheets, as we found the performance to be comparable to paste but without the aging concerns. This resolves a significant long-term concern of our old bench.
And this sounds small, but we also standardized GPU support brackets with this new test bench. In our old case bench, we had to keep an eye on sag as it could feasibly impact contact between pads and FETs.
We've also started building a special test rig... but I can't get into that yet. We'll show that on the main channel when it's ready.
That's enough of an update for now! If you've been wondering about the behind the scenes, that's been a lot of it. We've been really loading the website up with content as well, which is where most the prior Patreon update effort was allocated to get it up and running. If you haven't been in a while, it's worth checking out again. We have almost every cooler review we've done converted to text and have been working on converting all the new content to text as it goes up. Jimmy has been doing a great job at keeping up with all those video-to-article conversion (and without ads, which is largely thanks to the Patreon support!).
Thanks again,
-Steve
Comments
Mark H. Smith
Thanks Steve! You and the entire crew should be very proud.
Max Eliaser
I'm super stoked to see the return of case and cooler reviews. Much respect for taking the time to ensure you can keep delivering all this great consumer journalism to the usual GN standard. Here's to the return of 50-row bar graphs in the future!
FrenziedManbeast
I see absolutely no reason to upgrade from 12th to 13th or 14th gen Intel. Especially with the variable unknowns of 13th/14th gen instability. Not even worth the headache or the volatility IMHO. I'm also running a 12700K atm, and I'm not touching a thing!
Leviathanprym
supper cool to see the changes. welcome back GN team, been a while
Chairman Meow
Great job as always guys. Cant wait until your next video on Intels shenanigans, and which mystery company also was leaking customer info like a sieve.
Joe Cahn
Hey GN Team. Your coninued use of the 12700 to test GPUs is the main thing keeping me from upgrading mine. If they have a crazy black Friday deal on a 14700k I might be tempted to just swap it in though. Thanks for all that you guys do.
Alastair Ozmond
Hi Steve
