1.1 The Power of Chips, Revealed by the AI Era

We live in the age of AI.

AI answers our questions, creates text and images, and helps cars and robots move in ways that fit their surroundings. When AlphaGo defeated Lee Sedol at Go in 2016, it was received as a stunning event. Then OpenAI’s ChatGPT arrived in 2022, followed by AI assistants such as Anthropic’s Claude and Google’s Gemini, and now anyone can talk naturally with an AI and get help from it. Before we knew it, AI had made its way into our work and daily lives. Yet in the end, all of this is carried out by a computer. What is it, inside that computer, that actually does the work?

These capabilities were, in fact, goals people had been chasing for a very long time. Computers that converse with people (the 1960s), robots that assess a situation and act on it (the early 1970s), cars that drive themselves (the late 1970s) — countless studies and attempts toward these goals stretch far back into the past. Some succeeded in limited settings, but making them perform at the expected level amid the complexity of the real world was not easy.

Of the many approaches toward these goals, the one that became the foundation of today’s genuinely useful AI is the artificial neural network. It takes its inspiration from the way countless nerve cells in the human brain connect, exchange information, and make decisions, and it simplifies that process mathematically. A structure built on this approach, in which many mathematical operations are linked together, is called an AI model. Many kinds of AI models have been studied, and even this basic idea has a long history of research behind it.

To have a computer carry out the operations an AI model requires, we need a program that specifies the order in which those operations are performed. We create programs by writing code in languages that are easy for us to understand, and we usually call such programs software. The physical components, such as chips, that turn a program’s instructions into actual computation and action are called hardware. In other words, software decides what to process and how, and hardware actually carries out that work. The code that runs an AI model and the code we write every day both become programs and are executed inside the computer in the same way.

The problem with using AI models in the past was that the hardware of the day could not keep up with the computation the models demanded. The larger a model grew to tackle complex problems, the more its computation exploded, and the same operations had to be repeated countless times. Space was also needed to hold the numbers the model used and the intermediate results produced along the way. What was required was both the ability to compute quickly and the ability to store and retrieve large amounts of information. Scaling up a model meant computation took far too long, and sometimes it could not run at all for lack of memory. This was also a major obstacle to experimenting with and improving new models. In short, hardware struggled to support what software could do, and that in turn held software back.

What made it possible to break through this limit was the advancement of chip technology. A chip is a component made by gathering tiny electronic circuits onto a semiconductor, and inside a computer it takes on roles such as computing and remembering. The small black squares we often see are packages that protect the chip and connect it to the outside world. Figure 1-1 shows what such chips look like. Chips that handle computation have steadily advanced to process more operations at once, and faster. Memory chips, too, have come to hold more information and exchange it more quickly. As a result, the conditions were set for building and using AI models at scales that were previously out of reach, and today we encounter a wide range of AI technologies powered by AI model computation.

Advances in chip technology don’t stop at running existing programs faster; they also widen researchers’ opportunities to test larger models and new methods. Shorter experiments speed up the cycle of checking results and trying the next idea, and combined with richer data and accumulated research, software development has gained momentum as well. More powerful chips make new models possible, and those models in turn demand more computing power and memory — so software and hardware have come to drive each other forward.

It is true that advances in chips have brought remarkable changes to the latest AI technology. But whatever the application, the basic way a computer carries out work, and the principles operating inside it, have not changed much. Put simply, software and hardware mesh together on an electronic device called a computer, which contains chips. On the software side, code written by people decides what to process and how; on the hardware side, chips turn those instructions into actual computation and action. Whether running a complex, computation-heavy AI model or doing something as simple as adding two numbers, the way a computer handles the work with its chips — following a program written in code — and the principles behind it stay the same.

The chip is where the work defined by code turns into real, physical action. Yet from what we see on a computer screen alone, it is hard to tell what is happening underneath. How are the operations that code instructs carried out inside the computer? Where is the necessary information stored, and along what paths does it travel? Following these questions, this book sets out to look at what happens beneath the code by understanding the chip, which sits at the center of the computer’s operation. The goal is to understand how code written by people leads to the actual operation of a chip — in other words, the world beneath the code.