Math Menace


The use of machine learning to teach computers to play board games has had a lot of interest lately. Big companies such as Facebook and Google have both made recent breakthroughs in teaching AI the complex board game, Go. However, people have been using machine learning to teach computers board games since the mid-twentieth century. In the early 1960s Donald Michie, a British computer scientist who helped break the German Tunny code during the Second World War, came up  with Menace (the Machine Educable Noughts And Crosses Engine). Menace uses 304 matchboxes all filled with coloured beads in order to learn to play noughts and crosses.

How Menace works

Donald Michie's original Menace.

Donald Michie’s original Menace.
Menace “learns” to play noughts and crosses by playing the game repeatedly against another player, each time refining its strategy until after having played a certain number of games it becomes almost perfect and its opponent is only able to draw or lose against it. The learning process involves being “punished” for losing and “rewarded” for drawing or winning, in much the same way that a child learns. This type of machine learning is calledreinforcement learning.
The 304 matchboxes that make up Menace represent all the possible layouts of a noughts and crosses board it might come across while playing. This is reduced from a much larger number by removing winning layouts, only allowing Menace to play first and treating rotations and reflections as the same board.

Each of these layouts would be represented by a single matchbox, as they are all either rotations or reflections of one another.
Each of these matchboxes contains a number of coloured beads, each colour representing a valid move Menace could play for the
Coloured beads representing possible moves.

Coloured beads representing possible moves.
corresponding board layout. The starting number of beads in each matchbox varies depending on the number of turns that have already been played. In Donald Michie’s original version of Menace, the box representing Menace’s first turn had four beads for each different move. The boxes representing the layouts of the board for Menace’s second turn contained three beads for each different move; there were two beads each for Menace’s third; and one of each in the boxes representing Menace’s fourth go. There are no boxes representing Menace’s fifth move as there is only one space remaining and Menace is forced to take it.
To speed up the learning process even more, only beads representing unique possible moves are used. So even though all the spaces are free on an empty board, only three need to be represented: centre, side and corner. All other positions are equivalent to one of these three.
The beads shown are the only ones required for each scenario: all other positions on the board are equivalent to one of the positions marked.

The beads shown are the only ones required for each scenario: all other positions on the board are equivalent to one of the positions marked.
When Menace makes its move one must find the box representing the current board layout and take a bead at random from that box. The bead represents the space in which Menace wishes to place its counter. This process is repeated every time it is Menace’s go until either somebody wins or the board is filled.
After having completed a game, Menace is punished or rewarded depending on the outcome. If Menace lost, the beads representing the moves Menace played are removed. If it was a draw, an extra bead of the colour played is added to each relevant matchbox; while if Menace won, three extra beads are added. This means that if Menace played badly, it will have asmaller chance of playing the same game next time. However, if Menace played well, it is more likely to follow the same route the next time and win again.

An example gameimg4

When playing against Menace, Menace always starts, otherwise the number of matchboxes needed would be greatly increased.
1st move, Menace’s turn: The operator finds the matchbox that displays the empty board, opens it and takes out a random bead. In this case the random bead is red, which means that Menace wants to place its counter in the centre-top space.
2nd move, player’s turn: The human player places the counter in the desired spot, which is in the centre.
3rd move, Menace’s turn: The operator finds the matchbox that displays the current board layout, opens it and takes out a random bead. The random bead is now blue, which means that Menace wants to place its counter in the top-left corner.
4th move, player’s turn: The human player places the counter in the desired spot, here blocking Menace from getting three in a row.
5th move, Menace’s turn: The operator finds the matchbox that displays the current board layout, opens it and takes out a random bead. The random bead is green, which means that Menace wants to place its counter in the middle row and left-most column.
6th move, player’s turn: The human player places the counter in the bottom-left corner and obtains three in a row. The human player wins!
With the game over, Menace, since it lost, must be punished. Hence every bead that represented a move played is removed from its corresponding box, as shown below.

Building Menace

I built a physical implementation of Menace so that I could play against the real thing myself. I wanted to create an experience similar to that which Michie must have had when he came up with the idea. What is most striking is the time it takes to play one game: finding the right box, taking a bead out and then placing Menace’s piece in the corresponding space on the board is very time-consuming.
img6img7To make things faster I therefore decided to write a Python script to work in the same way that Menace does, with the computer doing all the work for me. Using the script, I could also save the state of the boxes and try out new reward systems and bead arrangements without losing the results from a previous set of data.
On top of this, I could now train Menace by playing it against already existing programmed strategies for noughts and crosses. I first had it play hundreds of games against a program that placed its counters randomly. You could see that after having played a certain number of games Menace had evolved. But when I played against Menace afterwards, it would sometimes make irrational moves. When Menace played against the random strategy, the random strategy would often place its counter somewhere other than where it could easily block Menace, letting Menace win and therefore reinforcing the sequence of moves that had been played. These moves, however, might not necessarily have been good ones Menace might just have got lucky.
I also wrote a program for a perfect strategy for noughts and crosses to train Menace against. This strategy only lets another player draw or lose against it. What struck me when training Menace against this program was the number of times it ran out of beads in certain boxes, having lost so many times that all of the beads from a particular box were removed. If there are no beads left in the box representing the current scenario, Menace is said to “resign” as it doesn’t “think” it can win in its current situation. This is OK if it’s a box representing a move that you rarely come across, but when it’s the box representing the empty board, the box that Menace always starts off with, there’s a problem! To fix this, all I needed to do was to add more beads to this box, although this had the consequence that it takes Menace longer to learn.


Graphs showing Menace against a perfect strategy. The y-axis shows the number of games Menace has drawn minus the number it has lost. On the top, we can see that after having added beads to the starting state of Michie’s original Menace, it takes longer to draw the same number of games as it has lost (cross the x-axis).
Machine learning is being used in many fields today including in the work of technology giants such as Facebook and Google, whom we have already mentioned have made breakthroughs in the more complex game of Go. The AI programs learning to play Go are not saving all the possible layouts of the game as we have done with noughts and crosses, as there are more of these than there are atoms in the universe. Instead, these programs detect similar patterns and use them as a starting point for their learning. However, the programs are then taught in a similar way to the way we taught  Menace to play against different strategies, although the programs learning Go play against themselves repeatedly. So ideas that emerged from simple machines in the past are still being used today for much more complicated tasks.”
#math instructors

What is wealth anyway

“It never really existed.
One of the big fantasies people have about wealth and money is that it all exists somewhere, piles of gold and cash stashed away in vaults.

That’s not true.  A lot of wealth is ownership.  Stocks are ownership.  Whether a stock is worth 10$ or 100$ is due to a lot of factors weighed and considered by “the market” .  The market is people interested in buying and selling.


As an analogy, let us consider a peaceful tropical island in which there is no money, but there are coconuts and seashells.  We can eat coconuts, so we tend to value everything in how many coconuts we are willing to trade for them.  You have 10 very pretty seashells, and seashells like that are usually worth about 5 coconuts.  We tally up your wealth, including counting your pretty seashells,  and we determine you have 100 coconuts of wealth- half of it in sea shells.

Then, something changes.  For some reason, people just don’t want sea shells as much as they used to.  Maybe they found something better.  Maybe times are harder, and seashells are pretty, but you cannot eat them, so people think they are less important.  For whatever reason, the seashells that traded yesterday at 5 coconuts each are now bringing in only 3 coconuts in the island market.

Again, we tally up your wealth, including the 10 pretty seashells you own.  This time, you have 80 coconuts worth of wealth.

You still have the same number of seashells.  You still have your hut, your fishing pole, and some coconuts.  Nothing has changed.  But 20% of your wealth has vanished!

Ah, but then you say “But that was just people’s subjective evaluation.  All a matter of opinion!  I haven’t actually lost anything.”

Aha!  Right.  

That’s all the market is.  A consensus of opinion.  But it remains the very best way of determining the relative value of pretty sea shells and coconuts.   “


“I’ll attempt to give a less technical definition that I hope is more accessible for people less mathematically inclined.

It may be easier to describe chaos theory than to define in a manner that is clear. 

Take the image below. There is an equation that will define this shape, ie, if you plot the graph of the equation, this is what you get:


What is interesting is that the line continues to swoop back on itself. In one instance it is in the lower left corner, and the very next, it is in the upper right corner, than lower right, then upper left, and finally back to lower left, and the cycle repeats. This is the butterfly of the so called “butterfly effect”. 

Now for a thought experiment, imagine that the above image is a weather prediction model.  Down is cold, Up is Hot, Left is Dry and Right is Wet. What makes this system chaotic is that if you plug in your variables (maybe pressure, wind speed, time of day, etc etc) into the equation, and you spit out a point in the lower left, so Cold and Dry is coming….. But wait… your variables were off by .0000001, hmm better plug them back in just to be safe. But low, now you’ve got a point in the upper right, so Hot and Wet….

So which is it? cold and dry, or hot and wet? 

That is chaos. Tiny, almost imperceptible changes to the input not only change the answer, but they change it categorically. It doesn’t just go from cold to being slightly colder… it completely flips cold to hot, dry to wet, or both! 

So what is the result of all this? to get reliable weather prediction we not only need a really accurate model/equation, we need to know our input variables to an incredible level of precision. This is why we can’t accurately predict weather out past a fairly short amount of time and why we still get it wrong sometimes when predicting things only mere moments away.”

Pay differences around the world

   Do people in Europe make more money than people in the United States? Salaries are different for jobs that are the same. Can anyone say why there are differences in salaries? Money does change some things but not everything.Being paid the correct amount based on the service that you offer is only fair.Who doesn’t want to get a fair days wage for a fair day of work?

Click Here!

Click Here!

Click Here!

For more information please fill out this form:

First Name:
Email Address:

Powered By: