“The machine does not isolate man from the great problems of nature but plunges him more deeply into them.”
― Antoine de Saint-Exupéry


AlphaGo DeepMind and AI


In the age of Xbox and Pokemon Go, when you think about classic board games, the images that come to mind may either be of wholesome families with doting parents spending quality time with their precocious children, with the hope that the kids may pick up a few life skills along the way, or that of intellectual dilettantes in professional clubs who obsess over mastering the game and one-upping their like-minded sparring partners.


Image: quatr.us

When an ancient Chinese board game recently gave the world a glimpse of the coming revolution in artificial intelligence, it created a tsunami of commentary, declaring “the bots have won”. The momentous event occurred early 2016 with the crushing defeat of a human at the hands of an AI algorithm. This event was similar to the historic defeat of Chess grandmaster Garry Kasparov at the hands of IBM’s DeepBlue or Ken Jennings capitulating to Watson in Jeopardy. Both events received world-wide-global-Kardashian-level media coverage.


The game of ‘Go’ has a tradition steeped in Chinese history dating back 2,500 years. A quick Wikipedia search will tell you that it’s:

“a zero-sum, perfect-information, partisan, deterministic strategy game, putting it in the same class as chess, draughts (checkers) and Reversi (Othello).”

The American GO Association’s website gives an even more intriguing description:


Go’s appeal does not rest solely on its Asian, metaphysical elegance, but on practical and stimulating features in the design of the game.

.. Go thinking seems more lateral than linear, less dependent on logical deduction, and more reliant on a “feel” for the stones, a “sense” of shape, a gestalt perception of the game. Beyond being merely a game, Go can take on other meanings to its devotees: an analogy for life, an intense meditation, a mirror of one’s personality, and exercise in abstract reasoning.


The AI algorithm that managed this feat is AlphaGo, a product of DeepMind – a company Google acquired in 2014 and founded by British wunderkind Demis Hassabis (@demishassabis). In a thrilling contest of 5 games, AlphaGo claimed the intellectual throne with a final score of 4-1. Facing off with AlphaGo was fellow human organism Lee Sedol, an 18-time international titlist and a 9-dan (highest rank) player from Seoul.



Small step or giant leap?

One of the key moments in the tournament was a move made by AlphaGo in the second game. Now etched in public memory as Move 37, the move not only bewildered Sedol, but many others watching the game via closed circuit. The move was cited as something out of the left field, so to speak, by the experts. This was not a move that was generated by AlphaGo’s brain by way of sheer brute force computation. It was in many ways, a move borne out of intuition.

This led three-time European Go champion, Fan Hui to say:


“It’s not a human move. I’ve never seen a human play this move,” he says. “So beautiful.” It’s a word he keeps repeating. Beautiful. Beautiful. Beautiful.


To understand this specific move and why AlphaGo’s triumph is different from previous feats of man vs. machine, it is critical to understand how ‘Go’ stands apart from other strategy board games like Chess, that have been solved before it. As often described, Go has more possible moves than there are atoms in the universe. You’d be forgiven to ask what all the fuss is about, given its seemingly simple rules. But out of its sheer simplicity arises epic complexity.

In logical games like Chess, the caliber of a good player is judged by the quality of decisions taken while making a particular move out of a set of possible moves – known as the ‘breadth’ or ‘branching factor’. Go is especially complex in this regard as there are about 10360 possible moves at any turn. Compared to 10123 for Chess, this number is too large to fathom or comprehend, let alone compute – even for powerful supercomputers, proving once again the cruelty of our mathematical reality and that it does not care about your feelings!


Lee Sedol

Lee Sedol | Image: yonhapnews.co.kr


Success in Go has been attributed to players having in their armory – skills like visualization, pattern recognition, along with broader strategic and tactical insights with respect to shape and empty space. What separates the greats from the amateurs are the hard-to-define and nebulous qualities of intuition and perception. It’s that thing that players may not be able to explain when talking about a particular move or the strategy behind it (“it felt right”).

It has been extremely hard, if not outright impossible to write algorithms and evaluation functions to build in such concepts. This is arguably why previous Go programs fell short and were not able to defeat their human overlords. This nature of the game also posed a salivating challenge for AI researchers, who use such games as test beds for inventing algorithms that can solve problems and learn, much like a human being does.

Thought to be revolutionary at the time, DeepBlue’s Chess playing algorithm relied on AI techniques that were predicated on the construction of a search tree with all possible moves for a board. It applied brute force to generate up to 200,000,000 positions per second to search for the best move working on top of a well-designed evaluation function with inputs from grandmasters and expert players. According to Wikipedia:


The evaluation function had been split into 8,000 parts, many of them designed for special positions. In the opening book, there were over 4,000 positions and 700,000 grandmaster games. The endgame database contained many six piece endgames and five or fewer piece positions. Before the second match, the chess knowledge of the program was fine-tuned by grandmaster Joel Benjamin. The opening library was provided by grandmasters Miguel Illescas, John Fedorowicz, and Nick de Firmian.



Under the hood

Owing to the extremely high branching factor of Go, techniques successfully used historically would fail miserably. The ingenuity of the technique eventually used for AlphaGo was the combination of a heuristic called Monte Carlo Tree Search (MCTS) with deep neural networks. It’s an advanced form of tree search and uses an alternative approach to search the game tree. It simulates playing out the game till the very end by selecting moves at random. The final result is then used to assign weights to the winning moves. These weights are used in selecting strong moves in further simulations.

What’s fascinating and ironic about this approach is that the use of probabilistic randomness ended up being the key technique for a game that is fully deterministic (with perfect information). As the game tree does not have to be expanded to account for all possibilities, it helps in working around the huge branching factor of Go. Using the Monte-Carlo technique was not new. But unlike previous programs that used the technique, AlphaGo used deep neural networks to guide the search.

Artificial neural networks are essentially a set of algorithms that mimic in design the neurological structure of the animal brain. They are composed of layer upon layer of artificial neurons and the outputs of neurons in one layer are fed to neurons in the next layer. Deep neural nets are distinguished from their more common cousins by their depth or the number of layers of neurons the data passes through. AlphaGo used two neural networks working in tandem with each other – a “policy network” gives the best move to play given the current state of the game, and a “value network” works on guessing the likelihood of a win for the given game state.

The second stage of training AI’s Rocky Balboa was letting AlphaGo’s policy network play against each other. This type of reinforcement learning used the results of these games as training input which led to AlphaGo improving itself and actually learn and discover strategies. This approach of splitting AlphaGo’s brain into Jekyll and Hyde and playing them against each other over millions of games resulted in dramatic improvements in its performance.

This type of learning, by trial-and-error and based on rewards and punishment has its origins in behavioral psychology. Russian physiologist Ivan Pavlov in the 1920’s may have been the one to introduce the word ‘reinforcement’ in the context of an organism’s behavior when it is preceded by a specific stimulus. In the context of machine learning and AI, it translates to how software algorithms take actions based on notional rewards – in this case, moves that kick ass.

This level of complex training compounded by millions of games worth of reinforced learning enabled AlphaGo to ‘learn’ on its own. Move 37 was the result of the machine thinking and discovering the best moves that will capture most territory on the board, rather than logic coded into it by a human.

In a post-match deconstruction in DeepMind’s control room, Wired reporter Cade Metz (@cademetz), in trying to understand the inexplicable move, interviewed David Silver who is a key researcher with the company:


Drawing on its extensive training with millions upon millions of human moves, the machine actually calculates the probability that a human will make a particular play in the midst of a game. “That’s how it guides the moves it considers,” Silver says. For Move 37, the probability was one in ten thousand. In other words, AlphaGo knew this was not a move that a professional Go player would make..

.. “It discovered this for itself,” Silver says, “through its own process of introspection and analysis.”




For most of us, an introduction to AI would most likely have been through the consumption of  pop culture artifacts, namely sci-fi and movies. In the 60’s, ground-breaking philosophical mind-benders like Stanley Kubrick’s 2001: A Space Odyssey, based on the book by British sci-fi legend Sir Arthur C. Clark explored the concepts of artificial intelligence and human evolution by way of HAL, an AI program and also the chief antagonist in the film. The 2015 sleeper indie hit ‘Ex Machina’ featured as one of its main characters, the wily Ava – a humanoid, sentient AI robot. Apart from the obvious reason of using it as a narrative device designed to thrill audiences, there is another, more valid reason in popularizing this notion of artificially intelligent entities with elements of humanness in the way they are represented. Us humans are prone to various cognitive biases. One of them is ‘anthropomorphism’. Coined by the Greek philosopher and poet Xenophanes, it refers to the attribution of human traits and emotions to non-human entities. Unsurprisingly, he is also known for the quote “Men create the gods in their own image”. This bias explains the depiction of not only divine beings (all religions!) throughout history and mythology, but in modern advertising as well – where companies create brand mascots with the intent of making the brand and its products both relatable and likable to consumers.



Ex Machina: the 2015 indie cult hit explored the idea of a sentient AI | Image: kanijoman


Owing to the very wide breadth of AI as a domain and due to its use as an umbrella term for disparate techniques, it has had limited understanding in the public domain and an established PR problemContrary to these popular conceptions of AI-powered super-robots coming to take our jobs (or wiping us from the face of the planet), we needn’t go too far from our daily life to see the practical and recurrent impact of AI, right from the moment we wake up in our beds.

Examples of Artificial Narrow Intelligence (ANI) or ‘Weak AI’ range from — recommendations on Amazon or Spotify, Siri giving you the day’s weather forecast with healthy doses of witticisms, to high-frequency trading algorithms trading securities in a matter of nanoseconds. Such forms of machine intelligence are bunched under ANI since it is designed and capable to perform only one narrow task. So if you ask Siri to chalk up a retirement plan for you, you are in for an awkward response.

Even though your smartphone may seem to be able to perform multiple tasks like a loyal genie-in-the-pocket, its intelligent capabilities are best described as products of machine learning – a sub-discipline of AI. Due to the sheer complexity of the space and only recent discussion of it being the next game changer, both ML and AI get conflated more often than not in the business media. Virtual assistants like the latest version of Apple’s Siri was trained using machine learning (and deep learning), enhancing its speech recognition and natural language processing juice, enabling it to respond in a more natural voice. Getting closer to Her.

Steven Levy, in his deeply reported piece for Wired writes:


The second Siri component Cue mentioned was natural language understanding. Siri began using ML to understand user intent in November 2014 and released a version with deeper learning a year later. As it had with speech recognition, machine learning improved the experience — especially in interpreting commands more flexibly.

As an example, Cue pulls out his iPhone and invokes Siri. “Send Jane twenty dollars with Square Cash,” he says. The screen displays a screen reflecting his request. Then he tries again, using a little different language. “Shoot twenty bucks to my wife.” Same result.

The impact of machine learning goes beyond Siri in the iPhone –

You see it when the phone identifies a caller who isn’t in your contact list (but did email you recently). Or when you swipe on your screen to get a shortlist of the apps that you are most likely to open next..

.. These are all techniques either made possible or greatly enhanced by Apple’s adoption of deep learning and neural nets.


Besides consumer devices, the hottest area is the elusive promise of autonomous vehicles. AI-powered tech may finally give us the flying cars (at least self-driving ones for the time being), the lack of which we have been complaining about.

With six out of eight founders coming from Stanford’s AI lab, Drive.ai is a startup that aims to make self-driving vehicles safer and friendlier. It differentiates itself from other tech companies by making a big bet on deep learning as opposed to a rules-based robotics approach as the only viable way to master autonomous driving, given innumerable complexities related to perception, decision making, motion planning, weather variability, elements of human cognition and common sense – the last of which we take for granted (more often than we should). For example, using deep learning, we can train the system to learn what a pedestrian looks like by feeding in reams of camera capture data. The pedestrians can be a couple walking down the street, a bicyclist walking with his bike, or a group of children crossing the road. Instead of trying to ensure every possible scenario of possible pedestrians is covered, this technique can give outstanding gains as the results can be applied to recognize objects it has not actually seen before. So based on its training, it will be able to also recognize Sourdough Sam walking down the street. Same concepts can be applied to other ‘perception’ problems like understanding traffic signals and road signs.

This cannot be done with a rules-based system alone. Just imagine all the ways we intuitively use non-verbal ways to communicate to others while driving – hand gestures, shouting and cursing, different types of honking etc.

Drive.ai plans to train the product to make sense of the context of the surroundings. An interesting sidenote is how Drive.ai is also applying deep learning to the problem of a labor-intensive process of annotating and labeling the raw data coming in from the wild for the algorithms to make sense out of. Kinda meta if you think about it.




The holy grail of AI is what science fiction has conditioned us to expect. It’s a machine that can pretty much perform any intellectual task as a human. As you can imagine, it intuitively seems a daunting task to create an intelligent agent that is able to plan and build cities, understand the intricacies of Keynesian economics, explore the cosmos or create viral internet memes.

Aaron Saenz wrote in Singularity Hub:


..These programs are far from the sentient, love-seeking, angst-ridden artificial intelligences we see in science fiction, but that’s temporary. All these narrow AIs are like the amino acids in the primordial ooze of the Earth. The ingredients for true human-like artificial intelligence are being built every day, and it may not take long before we see the results.


So while the questions and debate about the emergence — when, how, what if — of a truly sentient AI rages on, masters like Fan Hui and Lee Sedol, who lament the loss of some kind of human agency, can take solace in the fact that even though AlphaGo does not play Go like a human and starts playing it much better than a human, it was in fact created by humans, who like Xenophanes want to create these new super-gods in their own image.



Leave A Response