Artificial Inteligence Tuning Howto


The computer opponents in this computer poker game are completely self taught.  They start out totaly clueless bumbling around randomly folding or raising without a care.  They end up as extremely capable opponents to any human player with the guts to try their luck.  In fact over time they become better and better. How is this possible?  Its simple: they evolve.



Genetic programming 101


In order to train anybody to do anything you have to allow them to make choices.  You then punish them when they choose incorrectly and reward them when they are right.  Training a computer is no different.  You simply setup the critieria for which it will be judged and then allow it to make choices.

Life on this planet has come up with a pretty nifty trick for training, it is called evolution.  You start off with a population of creatures that are capable of: 1. procreating and 2. dying.  Now in every population of creatures there will be differences in individuals.  Those individuals that do better than other individuals have a better chance of staying alive.  Those 'better' individuals then have a better chance of procreating and passing whatever traits they have along to their offspring.  Those offspring with the 'better' traits will then live an pass those traits onto their offspring and so on.  

To do this with a computer you need two things:
  1. A way of passing traits from one generation to the next.
  2. A way of deciding which individuals are 'better' than others.
    To pass traits along one simply encodes them into a 'genotype' which in our particular case is a three dimensional array consisting of 128,000 cells.  When individuals are then mated to produce offspring their genotypes are then combined in totally random way.  This means that the offspring will share many of their parents traits and will more than likely also exhibit totally new behaviours which may or may not be better.  Along with the random combination of genotypes there is also a very very small amount of purely random point mutations (usually on the order of 0.5%).  These random point mutations insure that the population will not grow stale through too much inbreeding.

    Deciding which individuals are better than others is actually fairly easy in a game like poker where money is involved.  Simply put: the more money you have the greater your chance of producing offspring, and the less chance you have of being exterminated.  You can somewhat control the strategies played by the individuals by tinkering around with factors such as how often they mate (once per round or once per hundred rounds?), and if they have the chance to play the same cards in all positions (how often the deck is shuffled).


The human factor


    So how does all this affect you?  If you allow the computer opponents to learn from you (by turning on that 'learn from human' feature) then you will begin to affect who lives and who dies in the computer opponent population.  Over time those players who are better at attacking the weaknesses of their competitors (including you) will survive.  They will turn into better and better competitors who will hone in any weaknesses you have in your game.  The best part is that as you change so will the oponents so you can try various strategies and see if they have any weaknesses.


Great things come to those who wait


    Since your computer opponents are grown more than forged they will need to take time to learn.  Fortunatly you can have them learn against each other.  Simply choose a brain set from the AI options dialog and then choose 'Run simulation' from the File menu.  On a 800 mhz box you can expect a fairly decent batch of players in just a few hours.  This is recommended any time you change any of the important game options (such as number of players, or maximum number of bets).

Learn what it feels like to play god


    If at any time you feel that a set of opponents is no longer worthy of your time then smote them via the 'Initialize' button located in the AI options dialog box.  Don't worry no perminant harm will be done.  You will however probably want to simulate a few hours of gameplay with this newly initialzed brain set if you want to play against them again (unless you just want some easy opponents!).