GameMakersToolkit
patreon
GMTK Game Jam 2023 Post Mortem
🕑 Added 2023-07-21 05:23:16 +0000 UTCHey all! As is customary, I like to wrap up the GMTK Game Jam with some thoughts, feelings, and lessons for future jams.
If you somehow don't know by now... GMTK Game Jam is a community event where thousands of game designers try to make a game in just 48 hours. And the game has to fit a theme - this year, that theme was "Roles Reversed".
Once again, this jam was bigger than the year before. 23,193 joined up the jam, and they collectively produced 6,836 games.
It was almost 7,000 at the very end of the jam, but people are free to pull their game out of the jam, so the number ticks down over time.
(And yes, we broke Itch's servers again).
It also had the most ever ratings: 160,000. Wow!
What went right: Clearer rules and directions
I've gotten much better at making the rules as clear as possible. The jam page on Itch has all the relevant details, and I now change the page as the jam goes along. When people are submitting games, for example, I add a piece at the top about how and when to submit.
I also send details in an email that goes out to all jammers. And we communicate things well on the Discord.
The main thing that's tripping me up now is just weird stuff to do with the flow of submitting games through Itch. It should be easier to submit your game to the jam and harder to remove it - that way I wouldn't have to spend so much time explaining this to people!
I'm always working with Itch to implement new features, squash bugs, and improve the flow. I can't get everything I want - it's their website, not mine! - but they've been great at taking my requests on board. Hopefully we can get these last few issues resolved and make the whole process friction free.
What went right: Zero tolerance for issues
If there's one thing that makes the jam horrible for me, it's dealing with people who miss the jam, delete their game, or need help updating a broken game. It takes a lot of extra admin time, people get impatient and argumentative, and it makes me feel miserable to turn people down.
However, it's simply not feasible to help everyone, and not fair to pick and choose eihther.
So the best thing to do is simply have a zero tolerance policy of "sorry, better luck next year!". I also spend a lot of time stressing how important it is to upload your game on time - and make it clear before hand that there will be no late uploads.
This year there actually weren't that many people who missed the jam, considering the size, and they were mostly fine with my "sorry!, nothing we can do!" response. Another thing that really helped was pushing people to the Post Jam Jam, as a sort of consolation prize.
The more years that go by where it's clear there's a hard cut off date, the easier this will become.
What went right: More moderation
The GMTK Discord is the community hub for the Game Jam. Every year thousands of people enter the server to chat about the jam, get hyped for the theme, troubleshoot problems, and so on.
We've always done a pretty good job of moderating it, thanks to server stalwarts like Slam and Adam. But this year we went hyper mode and hired a bunch of extra helpers. That meant there were always at least 2 or 3 people watching the chat, 24/7. They could quickly answer questions and shut down spam, intolerance, and unhelpful arguments.
Getting new people up to scratch on how we like to moderate things is always tricky, so there were a few bumps and problems. But nothing major, and I think the server ran more smoothly than ever.
We'll definitely have a big team of mods going forward.
What went right: A good theme!
Last year's theme, Roll of the Dice, had its issues. It was difficult, limiting, and led to lots of same-y games. This year's theme, Roles Reversed, was much better I think.
It was definitely tough, but people rose to the challenge and made some incredible stuff. And while many games followed my lead from the theme announcement video (essentially - take a popular game or genre and then let us play as a different character), many others did wild and unexpected things.
This was the first year that I checked the theme with another game jam expert, to ensure it made sense. That was a good idea - I felt much more confident heading into the jam.
What went wrong: AI woes
Ah. Sigh. So, as the jam was approaching it suddenly dawned on me that I would need to make some kind of ruling on whether jammers could use generative AI models like Midjourney and ChatGPT.
It was tricky, because I was juggling lots of different arguments.
For one, generative AI does allow people to focus more on game design - the most important bit of the jam - and less on presentation or code. We already let people use preexisting assets, so why not AI-made stuff?
And also, it's really hard - if not impossible - to police this stuff. Can you know for sure if a background was made by Midjourney or a human? And, my word, what about code? Game Jam rules should be easy to follow and effortless to impose - AI really muddled that.
But then there was the more ethical side of things. Many people are wary or outright critical of AI. There are serious concerns about how these models learn from human-made stuff, and thorny questions about how unscrupulous publishers could use AI to replace coders and artists. Do we really want to support this stuff in the jam?
I'll cut a long story short - I initially announced that the jam would allow AI code but ban AI art. That made a lot of coders mad. So I change my mind and banned all AI stuff. It was a messy announcement and I regret it.
In the end it was fine - people followed the rules, there wasn't much moaning, and people were mostly happy for the jam to be AI free. But rules need to be crystal clear and consistent, and that was not the case, thanks to my flip flopping. Whoops!
Plans for next year
Next year things will likely stay largely the same! We've got GMTK Game Jam down to a science at this point, so it's only little twists and tweaks from year to year.
It's only weird curveballs - like the boom in generative AI - that mess things up. On that note, my plan for next year is to keep the rules as they are with a firm ban on AI-made stuff. But I'll keep an ear on the mood and get more feedback closer to the time.
One thing that I'm considering changing is the weights of the criteria, so presentation counts less towards the overall score.
The argument for this is that sometimes really clever games get shut out of the top 100 because the creator(s) are not brilliant artists. And some games can score highly because they look amazing - even if the gameplay is just so-so.
The argument against this is that while GMTK Game Jam's focus is always on design, I wouldn't want to devalue the efforts of artists and musicians, or dissuade them from entering the jam.
I've got a year to make up my mind, but any and all feedback will be useful!
I also want to do more to clear up broken games and submissions. It sucks to rate games and then get stuck with dead games and busted EXE files. I tried moderating this myself but it was an impossibly large task. I'm impressed by the efforts of the mod team on the Discord, so I could definitely get an army of volunteers to help me sort through game reports.
Stats
Let's finish with a look at some stats.
We can get an idea of team sizes, based on the number of people assigned as contributors to a game. This is optional, so it probably doesn't give the full image. But from the data we do have, we can see that...
The vast majority of games are made by a single person.
And having more people on your team will improve your rankings.
However, there probably is a limit! Look at Normal Activities - a game made by a whopping 24-member team, including pro game devs like Jonas Tyroller. It certainly did well, but puts paid to the idea that a huge team will guarantee game jam success!
Thanks to Jamalyzer for this info.
For game names, the most common words were dungeon, tower, reversed, reverse, hero, space, revenge, and simulator. The most common game name was Tower Attack (18 games), then Tower Offense (14), Dungeon Master (11), and Quest Giver (9).
The longest game names were...
- "I am a Monster Saving the Princess I Captured While Trying Not to Let My Boss Dragon Notice Because They Are Unpleasant"
- "PLEASE HELP ME I'm a lich in a dungeon but this knight wants to kill me and each time he dies he comes back stronger"
- "The night I went to convenience store won a lottery but got kidnapped by alien then started to escape or die"
Every game (barring the disqualified games) got at least 6 ratings. Making the Game had the most ratings at 1504. The average number of ratings per game was 23.5 and the median was 16.
Roughly 80% of developers made their own art (compared to 20% who used preexisting assets). As for music, the split was essentially 50/50.
And then in terms of engines being used, Unity is the big winner as always, followed by Godot, and then a small chunk of Game Maker and Unreal Engine games (plus a few other engines like Construct and PICO-8).
But what's interesting is to look at the historical data. I've been collecting this info since 2020, and so we can see how things have changed over the last four years.
Basically, Unity is slowly shrinking every year, and Godot is quickly growing. I saw someone on Twitter do a graph showing, if this trend continues, Godot could overtake Unity in 2028. But while Godot is slowly biting away at Unity, it is demolishing Game Maker.
Probably because Godot is in the same niche, and because Game Maker switched to a controversial subscription pricing system in recent years.
GMTK Alumni
One of the great things about running a game jam is seeing jam submissions take on a second life as fully-fledged games.
So I thought I'd add a new section to these post mortem posts where I can shout out new games that can be traced back to GMTK entries. These were released since the end of the last jam, in July 2022.
The most notable game is certainly Rollerdome, which came out shortly after the last jam, in August 2022. I covered this one in a GMTK video, so I'll let that do the talking for me.
Another big game was A Little to the Left, a cozy "puzzle game that lets players sort, stack, and organise household items into pleasing arrangements". It was released in November last year on Steam and Switch. It was even featured in the Wholesome Direct.
A 2021 winner, Factori, has been expanded into a massive new Steam release, as Word Factori, and will be released on August 3rd. Good luck to Star Garden games.
At GDC this year, I was thrilled to see that Slider - a GMTK Jam game - won "Best Student Game" in the Independent Games Festival. Good luck to them on turning it into a bigger game.
I try to list all of these games on Steam and on Backloggd.
Thanks for reading! I'll see you next year!
Mark
Comments
H.B.
The reversal on the stance of AI when it comes to code made no sense to me. E.g. check out the StackOverflow survey on AI tools being used (https://survey.stackoverflow.co/2023/#most-popular-technologies-ai-dev), more than half of respondents are using GitHub Copilot, a paid tool; that should give some indication how much people actually are in support of such tooling. It also is not like people use such tools in the vein of "write a game for me". Most of the time it's just a fancy auto-complete that writes *exactly* what you would have written yourself anyway. And as someone else noted, if the rule cannot be enforced it does not make much sense. Especially for code, it is virtually impossible to tell and it is not like the source is being published or reviewed by anyone.
Willhart
Like an option for it would have been nice.
Willhart
I think this jam could have been a great opportunity to discover more benefits and use cases of AI made code, art, and ideas for those who have been experimenting with the new tech a lot. I also read that at least one good game got banned for using AI art too (one where you forget the lines when going on a stage and have to improv).
Lauren N
While I understand the point about presentation scores causing good games with bad art to score lower than they perhaps should, I think it's important to remember that the jam is not a fair match and it likely never will be. There are plenty of unfair reasons one game idea does better than another. After all, you have teams of 20+ versus individuals. Even if you were to lower the weighting of presentation, people will still probably rate a game higher in all categories if they enjoy it's appearance, because it's hard (if not impossible) to separate the different parts of your experience with a game. So I doubt it will fix the problem much. Besides, it already feels like a loosing game as a solo artist. No need to dash our hopes even more. Anyway, good job and thanks to you and your team running the jam! It's always a lot of fun every year.
I think scoring should be changed so that the weakest score is weighted less heavily in the overall ranking so as to be fair to those who are about the art and those who are creative at the same time As for the AI debate, if you cannot enforce a rule you should not have it. A rule you cannot enforce just rewards cheaters. So if you cannot enforce AI bans you are hurting your honest participants.
