It reliably provides a certain experience.
To who? It didn't provide it to me.
Well....(thinks)....to most people.
Most people don't play RPGs, and of those, most of them haven't played this one.
Mmmmm...to most people who...want...to...have that experience....already? Aaaand...didn't get it with that other game? Aaaand who....
Here's the problem we have when we talk about "objectively good game design".
It isn't about providing a certain experience reliably.
Focused design is, at best, about:
Providing a certain experience reliably to a given audience.
Good game design, on the other hand, is about:
Providing any game tools the audience enjoys using to a given audience.
There's a difference because Focused Design assumes an audience so small and homogenous that you can actually reliably provide them an experience.
Let's take a look at a couple "successful" independent RPG products. Products that sold well in their categories and made money for their authors and won fabulous prizes.
Here's Dogs In The Vineyard. Its author says this:
"It's a game where you're a teenager with a gun and a book, and you're sent out to solve problems between adults that can't really be solved with either. If that sounds fun to you, you might enjoy the game, and if it doesn't, you won't."
So here we've got the experience it's supposed to provide and a description of an audience that it is supposed to provide that to. That audience seems to be very happy about it.
The experience is you're some western proto-fasicst religious weirdo passing judgment on rural towns, the audience is amateur game designers and people who hang out on their forums who are into westerns and/or judgment and/or dealing with soap operatic moral issues in games and/or the kind of people who go to gaming conventions and see that game description and are like "Yeah, I'm on that".
How big that audience is or could potentially be is unclear, but it seems fairly true that within that audience those people are happy with that thing and get that thing out of it. He sold his copies. They don't complain much from what I've seen.
They are people I do not understand who want a thing I do not understand why anyone would want it, but they get it and are happy with it. If we define "good design" as giving the experience people expect to a defined group of people, it succeeds.
So then we got a book that sells about the same rate, my book, Vornheim: The Complete City Kit (I would love to use another book instead of my own for this example, but I needed something where I knew a lot about the sales figures and audience response from a different audience that Vincent's.)
It provides a bunch of tools for making up an interesting, useful fantasy city in the middle of running an RPG to an audience that:
1. Knows how to play an RPG
2. Has a ruleset already
3. Is ok with improvising content
4. Does not want to memorize a lot of content
5. Wants to run an open sandboxy campaign
6. Is ok with a few weird results mixed in
7. Is adults
8. Likes the tone set by the fairly dense, fairly inky artwork,
9. Plays older versions of D&D or is used to/good at converting on the fly to their system of choice
10. Is into the same kind of weird sword and sorcery fiction old and new that inspired it, and
11. ...probably reads a lot about games on-line and probably reads this blog
That's a very specific audience. But that audience seems to like it. It sells well and there are few complaints in my mailbox.
It provides that thing to that audience. Its tools are "well designed" in that sense.
Point is: both of these different products succeed not necessarily because they are objectively well-designed in any way, really. They succeed largely because they were very good at finding the audience that wanted them--mostly by capitalizing on social networks where people talk about games that were already there and telling those networks who their authors were and what they were about.
Change the audience and suddenly the experience changes completely. I played Burning Wheel--which is supposed to be a game of beliefs being challenged in epic struggles--and it came out as the funniest light-comedy game I ever saw.
Hard Examples (Mainstream Design)
Now here's the rub for bigger games--Vampire, RIFTS, Call of Cthulhu, D&D etc.
If we are judging whether they are "well designed" we have to take into account that both parts of the "focused design" equation are (by necessity and design) waaaaay more ambitious than what Vincent and I were going for.
Like with Vampire, the focused design brief pretty much has to be:
Provide tools for much any kind of adventure conceivable where you play a vampire in a contemporary world full of secret, hidden vampires to an audience consisting of anybody in the world who thinks playing a vampire in a world full of secret, hidden vampires is a good time and which has maybe seen Lost Boys or read Anne Rice and liked it.
The number of bases to cover is bigger, the audiences are bigger.
This is why people accusing games of "bad design" (unless they are specifically talking about like math inconsistencies or obvious one-fix problems, which is rare) consistently, when pressed, end up focusing on the marketing. The marketing seemed to them (a fraction of an audience) to promise a certain experience that it didn't deliver.
Point is: Since the marketing for Dogs In The Vineyard and Vornheim consists (by default) largely of its authors talking about games online and because the audience consists largely of people who read those things they said, it's really motherfucking easy for us to provide an expected experience to an audience who wants that expected thing. They know us and what we like.
If, on the other hand, your "what are we promising" is a painting of some thing happening and a tagline ("Chill: A Scary Game For Scary People") and your audience consists, essentially, of anyone who likes what's in that ad, (which is what all mainstream games have to deal with) it's much harder.
One obvious reason why is that suddenly your audience consists of people with mutually exclusive skillsets and needs.
Like, providing "Gritty, fast-paced adventure in an imaginative setting" to a group whose GM has great social skills and no math skills and providing "Gritty, fast-paced adventure in an imaginative setting" to a group whose GM has great math skills and no social skills requires two very different rulesets and assumptions. This is the Sword Of Obviousness hanging over every would-be-mainstream game design team.
I've heard people argue D&D Type IV is objectively better-designed because it was built along the focused-design principles that were invented for indie games: if the GM is bad you can just fall back on the default experience the rules-as-written provides (exciting, heroic combat using the powers on your sheet to play a cooperative combat-centric game where you fight balanced monsters and many noncombat challenges are resolved via an always-balanced skill challenge mechanic).
What they don't seem to get is that the rules-as-written do not provide that experience to everyone or necessarily even most people in the audience (because all groups are different and the audience for D&D is bigger than that for indie games) and/or that that default experience is not even something most groups necessarily want (because not everyone imagines exactly the same possibilities when they see a dragon in a dungeon).
This audience-expansion is one thing the focused design crowd hasn't seemed to take into account yet--at least not en masse. It's way easier and makes more sense to do a focused design when the audience is itself focused. A lot of them seem kinda surprised they didn't like 4e: wait, I thought D&D was about fighting monsters in dungeons, 4e is the most fighting-monsters-in-dungeons-focused D&D yet!, focus is good!, why am I not enjoying this?
Just because you showed up with a sword and a spell to play D&D doesn't necessarily mean you have any interest in the kind of combat 4e's default-mode wants you to do.
And here's the rub with mainstream games: often that default, uncustomized experience is something most of the audience doesn't want and the design team knows that. The center is left empty on purpose.
OD&D was fairly up-front with the fact that you had to add your own stuff just to make the thing work at all.
There are people who look at a picture on the cover of a mainstream RPG and go "That is exactly what I want". I don't, in real life, know any of them. And yet my group consistently plays these mainstream games. Why? Because the fact that you gotta make stuff up and that most of the best content will be made up was part of the original design intent of the game. They are not recipes: they are grocery stores. Here's stuff: go make things.
Call of Cthulhu is full of rules I never use and the sample adventures in the book are things I would never play. Designwonks go "Isn't it then a poorly-designed game for you? Don't you want something more focused on the play you want" No. 90% of the things in that rulebook are just not for me-- they're for all the other Call of Cthulhu groups who play it like Hellboy or X Files or Deep Ones and Tommy Guns. And that's ok. That's what mainstream design does.
I have absolutely no hope that some magically sympathetic just-like-Zak guy will come along and write a horror game that is perfect for my group and I don't need them to. I have stats, SAN mechanics and % skills and I can make whatever I need out of that and so can about a jillion other people so we're good.
If the standard was "Reliably produces an adventure in a certain genre that I wanna play" then no game in the history of gaming has ever been "Well designed" so far as I can tell. Why? Because many people (including the game designers) are individuals. The design and content has to be tweaked to fit those individuals and the experience-desired has to be tweaked to give them a mood, a tone, an authorial voice that they are interested in. They need a tailored experience or they are bored.
This might even be most gamers. We don't know.
This is the rub:
Enthusiasts of focused design games point to games with no default experience hardwired in the center (or an experience they don't like or understand hardwired into the center) and say "this game doesn't deliver what it promises, it's poorly designed".
But this isn't so, because the game does not promise you can run it well without adding anything of your own because for that game's audience the idea that you could run a game well without adding stuff in yourself never turns out to be true no matter what game it is.
If you think any game can run even tolerably well without the players or GM adding something you might be right for you, but you are not in this game's target audience and so the it's "good design/bad design" equation can't be judged by what it gives you.
Now I know from the internet that there are actually people in the world who look at a module and see a shiny, squarejawed paladin smacking a hunchbacked kobold with a +2 sword forever and think "YES, EXACTLY THAT!" and for them a design which has a "default experience" running up its spine that they can work off of is what they will consider "good". "Why can't we have Mr Default Paladin Smacking Mr Default Orc right there and you can have your crazy expert-GM arty fringe lunacy on top of that?"
It is no small task to build that "default experience" into a game. It takes up space and time that could be devoted to other stuff and that you have to flip past in the books and in the statblocks and in the character generation session and in the graphic design and in the system itself. Building a fighter and a thief who mathematically balance out the same way in combat takes a lot of design space to undo if you don't like that (trust me I've tried). It is not necessarily a thing with no cost. So if the people who want that spine are not in the target audience, there are fine reasons to leave it out.
If someone wants to build a game with a default spine for whatever slice of the audience actually enjoys default things but that also provides no significant hurdles to the slice of the audience that cannot enjoy a game they have not significantly customized they should try. They should try and should know that it has never successfully been done before but I hope they succeed.
*P.S. Prescriptive Grammar nerds fuck off.