03-23-13 - AI Rambles

With GDC coming up I've been thinking generally about the state of game technology in general. First a bit of a rant on that.

I am so fucking bored of graphics. Graphics are not the damn problem. I'm completely appalled by the derivative repetitive boring games you all keep making. I don't want to play "Shoot People in the Face 227" or "Space Marines 154" or "Slide Blocks to Make them Go Bling N" or "Cute Creatures Jump Around on Blocks N". Barf, boring. And making them all shiny with new graphics is just gilding the turd. Stop working on graphics.

Games have huge tech problems that nobody seems to want to work on. One that I have wanted to work on for a long time is animation. And by "animation" I don't really mean playing back clips, which fundamentally looks like garbage, but making characters move naturally, able to transition movements the way their body should, respond to surface variations and so on. Game animation just looks so awful, and it's becoming more uncanny as the graphics get better.

(in fact if we were smart we would have done it the other way around. Every cartoonist for a hundred years has known that it's actually ok for the visuals to look unrealistic if the animation and sound are really good. Human perception cares more about motion than the static appearance of things.)

Anyhoo, the other big one is AI. And by "AI" I don't mean playing scripts, or moving to designer-placed cover spots. Even some of the more sophisticated game AI systems are really just fancy whack-a-mole. You can see the AI's run to one spot, do a pre-programmed routine, run to another spot, pop out of cover so the player can shoot me, pop back in cover. Now, certainly there are merits to whack-a-mole AI. If you're making a platformer you don't want the enemy to do surprising things, you just want them to walk back and forth on a set pattern that the player can pick up easily. They're not really AI at all, they're rigid bodies with an animal painted on them.

These AI's never surprise you, they never make you laugh, they never make you want to play again because they might do something new. They feed off your energy and don't give anything back, like a bad conversation partner.

So it made me realize that game AIs are actually more interesting when the game is very simple. It might naively seem like a big complex sandbox 3d world has got a more complex AI, but really that complex world means that the AI no longer understands what it's doing. Your only hope is to give it simple rules to follow about what it can do in that world.

In contrast, AI for simple game systems (chess, checkers, backgammon, poker) can do amazing things that the human programmer never anticipated. There's a funny thing that happens with computer algorithms where a cold rational scientific brute-force search of a mathematical problem space actually leads to behavior that's more human than the shitty heuristic decision-tree type programming that's explicitly trying to simulate human behavior.

For example, when I was writing poker AI, I was really amazed at the "creative" plays that a simulation-based bot makes. (for review : a standard UAlberta-style poker bot works by building a model of the opponent based on observation of previous action; it then simulates all the possibilities for future cards and imagines what the opponent will do in each situation; it sums the EV over all paths for each of its own actions, and chooses the action that maximizes EV).

At the simplest level, it figures out things like check-raising when you tend to bet checked flops too much. But it did even weirder things. For example the bot would very quickly become hyper-aggressive against an opponent that folds even slightly too much; it adjusted faster and way more severely than any human. I would play against it sometimes with our cards face up so that I could make sure it was doing sane things, and I would see it make a huge check-raise bluff on the river with junk. My first thought is "I have a bug" and I'd go looking into the stats of the model, and found that there was no bug, it's just that the AI had learned that I thought a big river raise meant strength, so I was folding to them a lot, and therefore the simulation will jam almost every hand.

This type of poker AI is not the game theoretic equilibrium solution. It's assuming that the opponent plays by some scheme which may not be optimal, and that its own strategy is not face up. That can lead it to make mistakes. One I've long been aware of is that it doesn't hedge correctly. Normal humans hedge all the time in their poker play, perhaps too much; you will often suspect that someone is bluffing a huge percent of the time, but you aren't sure. A non-hedging AI would immediately start making very light call-downs, but a cautious human will weight in some factor for the model being wrong and play with a blended strategy that's not disastrous if the model is wrong (like only doing the light call-down in small pots, or waiting for a call-down with a hand that has some chance of being best even if the model is wrong).

Continuing the random rambling train of thought, I just realized (re-realized?) that one of the flaws with this style of poker AI is that it doesn't anticipate the reaction to its moves. Of course it does anticipate the reaction just in terms of "if I bet, what hands will he call with or raise with", but it is evaluating based on the *past* model of the opponent. After you make your bet, the opponent sees it and adjusts their view of you, so you need to be anticipating how their play style changes. For example in the case I mentioned above - when someone is playing pretty weak/tight the bot rapidly becomes hyper-aggressive, which is mostly good, but the bot never gets the idea that "hey he can see I'm raising every single street of every hand, he's going to adjust and call me down more".

Anyway, bringing it back to games, it occurred to me that it would be interesting to try some really simple 2d games, and give them a mathematical solving AI, instead of the usual heuristic crap we do. Like, let's face facts - we can't actually make games in these big free form 3d worlds, it's too complex. Our ability to do the graphics has gotten way beyond every other aspect. We need to back up and go to like Ultima-style 2d tile-based games. Now you have a space where the AI can just explore future actions, and things like advancing on the player by moving from cover to cover just pops out of the behavior automatically because it maximizes EV, not because it was explicitly coded.

(I'm not contending that this is the "right way" to make games or that it will necessarily make good games, I just thought it was interesting)


Wes said...

I'm with you on the being appalled about new graphics on the same game thing.

We've been playing these for 30+ years now, lots of us are getting older. Seems like there should be a way to keep gamers on board by tackling some themes that start mattering as we age.

Neal Burns said...

The New York Times posted a rock paper scissors game you play against a computer, which is still up here. It was uncanny how badly it was beating me. In a sense, all games against an opponent reduce to rock paper scissors when the opponents are otherwise well-matched, don't you think? The way the game in the NYT article went, it convinced me that trying to win a bluffing game against a statistical model was kind of pointless. I've never been much into games, though.

old rants