What Is a Processor? History, Architectures and How a CPU Works

What Is Programming? described programs as precise descriptions of a process, and What Is an Operating System? described the software that shares a machine between them. This article goes one level further down, to the part that does the work: the processor. It covers what a processor is, how it got from vacuum tubes to tens of billions of transistors, the main ways processors are designed, and, in some detail, how a modern CPU turns instructions into results.

What a processor is

A processor is the part of a computer that carries out instructions. It fetches them from memory one after another, works out what each one asks for, and does it, operating on values held in its registers and in memory.

Everything a computer does, from drawing a window to training a neural network, ends up as a long stream of very simple instructions: load a number from memory, add two numbers, compare them, jump to a different instruction if the result was zero, store a number back. The processor’s whole job is to run that stream correctly and as fast as possible.

The essential parts have barely changed since the 1940s:

CPU, core, processor, chip

These words overlap, and are often used interchangeably:

Architecture and microarchitecture

The most important idea for understanding processors is the difference between what a processor promises to do and how it does it.

• The instruction set architecture (ISA) is the contract between hardware and software: the instructions, the registers, how memory is addressed, how interrupts and privilege levels work. x86-64, ARM (AArch64) and RISC-V are ISAs. Compilers and operating systems are written against the ISA.
• The microarchitecture is one particular design that implements an ISA: how many instructions it works on at once, how big its caches are, how it predicts branches. Intel’s and AMD’s latest cores are completely different microarchitectures that run the same x86-64 programs.

IBM made this separation deliberately with the System/360 in 1964: one architecture for a whole family of machines at very different prices and speeds, so customers could move to a bigger model without rewriting their software.¹ Every major processor family since works the same way, which is why a program compiled years ago still runs on a processor designed this year.

Kinds of processors

A history of processors

Before the microprocessor

The first electronic computers of the 1940s built their processors out of vacuum tubes and relays, and the stored-program design that every processor still follows came out of that decade (the story is told in What Is Programming?). John von Neumann’s 1945 report described a machine with an arithmetic unit, a control unit and a single memory for both instructions and data.²

Two inventions made modern processors possible:

• The transistor, demonstrated at Bell Labs in December 1947 by John Bardeen and Walter Brattain, working in William Shockley’s group, replaced the vacuum tube with a small, cool and reliable switch.³⁴
• The integrated circuit put many transistors and their wiring on one piece of semiconductor. Jack Kilby built the first at Texas Instruments in 1958.⁵ In 1959 Robert Noyce at Fairchild conceived a silicon version built with the planar process, which could be manufactured in volume; the first working ones followed in 1960.⁶

Processor design advanced quickly on these parts. In 1951 Maurice Wilkes proposed microprogramming: implementing the control unit as a small internal program of simpler steps rather than as fixed logic, which made complex instruction sets practical to build.⁷ The CDC 6600 (1964), designed by Seymour Cray, used ten functional units working at once and a “scoreboard” to track which instructions could proceed, and was the fastest computer of its day.⁸ For the IBM System/360 Model 91 (1967), Robert Tomasulo devised an algorithm that let instructions run as soon as their inputs were ready, not strictly in program order.⁹ Both ideas are at the heart of every high-performance processor today.

1971: the microprocessor

By the late 1960s an integrated circuit could hold thousands of transistors, enough for a whole, simple processor. Intel was asked by the Japanese calculator company Busicom to build a set of chips for a calculator, and instead of many custom chips, Ted Hoff, Federico Faggin, Stanley Mazor and Busicom’s Masatoshi Shima designed a single programmable one. The Intel 4004, announced in November 1971, was a 4-bit processor of about 2,300 transistors.¹⁰¹¹

Then came a rush of 8-bit processors that started the personal computer industry: the Intel 8008 (1972) and 8080 (1974), the Motorola 6800 (1974),¹² the MOS Technology 6502 (1975), which powered the Apple II and, as the 6510 variant, the Commodore 64,¹³ and the Zilog Z80 (1976).¹⁴ In 1978 Intel’s 16-bit 8086 introduced the x86 instruction set,¹⁵ and IBM’s choice of its sibling, the 8088, for the IBM PC in 1981 made x86 the dominant architecture of personal computing.¹⁶

Moore’s law and Dennard scaling

In 1965 Gordon Moore, then at Fairchild, observed that the number of components that could be put economically on a chip was doubling every year, and predicted it would continue for at least a decade.¹⁷ In 1975 he revised the rate to a doubling about every two years.¹⁸ The prediction became a self-fulfilling industry roadmap and held, roughly, for half a century.

A second, less famous rule mattered as much. In 1974 Robert Dennard and colleagues at IBM showed that as transistors shrink, their voltage and current can shrink in proportion, so the power used per unit of chip area stays constant.¹⁹ Dennard scaling meant that each new generation of smaller transistors could also run at a faster clock without the chip getting hotter. For three decades, programs got faster every year without anyone changing them.

The 1980s: RISC

As compilers improved, researchers measured what instructions programs actually used and found that compilers rarely used the complicated ones that microprogramming had made easy to add. RISC (reduced instruction set computer) proposed the opposite: a small set of simple, fixed-length instructions, each doing one thing quickly, with only load and store instructions touching memory, and many registers.²⁰

• At IBM, John Cocke’s team began the 801 in the mid-1970s and had a prototype running by 1980, the first machine designed this way.²¹²²
• At Berkeley, the project led by David Patterson and Carlo Séquin, which coined the name RISC, produced its first RISC I chips in 1981.²³²⁴
• At Stanford, John Hennessy’s MIPS project followed the same lines.²⁴

Commercial RISC processors followed: MIPS, Sun’s SPARC, IBM and Motorola’s PowerPC, and, in 1985, the first ARM processor, designed at Acorn Computers in Cambridge by Sophie Wilson and Steve Furber.²⁵ Designed for simplicity and low power, ARM went on to dominate phones and embedded systems, and Arm describes it as the most widely used processor architecture in the world.²⁶

x86 survived the RISC era by borrowing its ideas. Starting with the Pentium Pro (1995), Intel’s processors translate x86 instructions internally into simple, RISC-like micro-operations and execute those out of order.

The 1990s: pipelines, superscalar and out-of-order

With more transistors to spend, designers made each processor do more work per clock tick. The Intel 80486 (1989) was pipelined and put the cache and floating-point unit on the chip. The Pentium (1993) was superscalar, able to start two instructions per cycle.²⁷ Out-of-order execution, Tomasulo’s idea from the 1960s, returned in microprocessors such as the Pentium Pro, the MIPS R10000 and the Alpha 21264. Clock speeds climbed from tens of megahertz to over a gigahertz by 2000.

The 2000s: 64 bits and the power wall

Two shifts defined the next decade.

64 bits. Intel’s attempt to replace x86 with a new 64-bit architecture, Itanium (2001), never caught on. Instead AMD extended x86 to 64 bits as AMD64, disclosed in 1999 and first shipping in the Opteron in April 2003, and Intel adopted the same extension in 2004.²⁸²⁹ This x86-64 is what desktops, laptops and most servers run today.

The power wall. Around 2004, Dennard scaling broke down. Transistors had become so small that they leaked current even when switched off, and voltage could no longer be lowered in step with their size. Higher clock speeds now meant much more heat than a chip could get rid of. Intel’s Pentium 4 topped out at 3.8 GHz,²⁷ and desktop clock speeds have grown only slowly since. The industry turned to multicore chips instead, putting several complete processors on one die. IBM’s POWER4 (2001) was the first commercial example;³⁰ AMD and Intel shipped dual-core desktop chips in 2005.³¹ Herb Sutter’s essay The Free Lunch Is Over (2005) warned programmers that from then on, faster programs would have to be parallel programs.³²³³

The 2010s and 2020s: phones, specialisation and open ISAs

Smartphones made energy efficiency the central design goal, and ARM-based SoCs took over mobile computing. Arm’s big.LITTLE design paired fast, power-hungry cores with slow, efficient ones on the same chip,²⁶ an idea now used in laptops and desktops as well. In 2020 Apple moved the Mac from Intel to its own ARM-based M1, a chip of 16 billion transistors.³⁴

As general-purpose cores stopped getting much faster, more work moved to specialised processors. GPUs, originally graphics chips, became general parallel processors once NVIDIA’s CUDA (2006) let programmers write ordinary code for them,³⁵ and they now do most machine-learning work. Google built the Tensor Processing Unit, deployed in its data centres from 2015, purely for neural-network arithmetic.³⁶

RISC-V, started at Berkeley in 2010, is an open instruction set: anyone may build a RISC-V processor without a licence or royalties.³⁷³⁸ It has spread quickly in microcontrollers and embedded chips and is moving into larger systems.

In January 2018, the Spectre and Meltdown attacks showed that the speculative execution processors had relied on for decades could leak secrets through cache timing.³⁹⁴⁰⁴¹ Processor security is now a design goal alongside speed and power.

A timeline at a glance

Instruction set architectures

von Neumann and Harvard

The first architectural choice is how instructions and data reach the processor.

In a von Neumann architecture, instructions and data share one memory and one path to it. This is flexible (a program is just data, so it can be loaded, generated or modified like any other data) but the processor cannot fetch an instruction and read data in the same instant. That single path is the von Neumann bottleneck. A Harvard architecture keeps instructions and data in separate memories with separate paths, so both can be accessed at once; many microcontrollers and DSPs work this way.

Modern CPUs are a hybrid, sometimes called modified Harvard: main memory is shared, as in von Neumann’s design, but the first-level cache is split into an instruction cache and a data cache, so the core can fetch and access data in the same cycle.⁴²

CISC and RISC

The distinction matters less than it once did. Large x86 cores decode their instructions into RISC-like micro-operations, and ARM has grown a large, sophisticated instruction set. What remains is that simple, fixed-length encodings are easier to decode quickly and with less power, while x86 carries decades of compatibility.

The main ISAs today

What an instruction looks like

An instruction is just a number whose bits the processor’s decoder splits into fields. In RISC-V every basic instruction is 32 bits, laid out in a handful of fixed formats.⁴⁵ The register-to-register format looks like this:

Classifying parallelism

Michael Flynn’s 1966 taxonomy classifies machines by how many instruction streams and data streams they handle at once:⁴⁶

A modern chip is all of these at once: several MIMD cores, each with SIMD vector units.

How a processor works

From transistors to logic

At the bottom, a processor is made of transistors used as switches: a voltage on one terminal turns the connection between the other two on or off. A few transistors make a logic gate (NOT, AND, OR, NAND). Gates combine into larger blocks:

• Combinational logic computes an output purely from its current inputs: adders, comparators, and multiplexers that choose between inputs.
• Sequential logic remembers: a flip-flop stores one bit and updates it only on a clock edge. Registers are rows of flip-flops.

The clock ties it together. On each tick, every register captures the output of the logic feeding it, and in the time before the next tick that new value ripples through the next stage of logic. The slowest path through the logic between two registers sets the fastest the clock can run.

The fetch–decode–execute cycle

Every processor, from the 4004 to the newest server chip, repeats the same cycle:

1. Fetch the instruction at the address in the program counter.
2. Decode it: which operation, which registers, which immediate values.
3. Execute it: compute a result, an address or a branch decision.
4. Access memory if the instruction is a load or store.
5. Write back the result to a register, and move the program counter to the next instruction, or to the branch target.

The classic RISC design maps each of those steps onto one stage of hardware:

Here is a short C function and the RISC-V assembly a compiler might produce for it. Every line of the assembly is one trip around that cycle:

long sum(const long *a, long n) {
    long s = 0;
    for (long i = 0; i < n; i++)
        s += a[i];
    return s;
}

# a0 = pointer to the array, a1 = n; the result goes back in a0
sum:
    li    t0, 0           # s = 0
    blez  a1, done        # nothing to add if n <= 0
loop:
    ld    t1, 0(a0)       # load the next element         (memory)
    add   t0, t0, t1      # s += element                  (ALU)
    addi  a0, a0, 8       # step to the next 8-byte element
    addi  a1, a1, -1      # one fewer to go
    bnez  a1, loop        # go round again while n != 0   (branch)
done:
    mv    a0, t0          # return s
    ret

Control: hardwired or microcoded

The control unit turns each decoded instruction into the signals that steer the datapath: which registers to read, what the ALU should do, whether to access memory. In a hardwired design this is fixed logic, which is fast and suits simple RISC instructions. In a microcoded design, as Wilkes proposed, each instruction runs a short sequence of micro-instructions from a small internal memory.⁷ Modern x86 cores use both: common instructions are decoded directly into micro-operations by hardware, and rare complex ones fall back to microcode. Because microcode can be updated, processor makers can also use it to fix some bugs after a chip has shipped.

Pipelining

Running one instruction through all five stages before starting the next would leave four of the five stages idle at any moment. Pipelining overlaps them, like an assembly line: while one instruction is being executed, the next is being decoded and the one after that fetched.

A pipeline does not make any single instruction faster; it raises throughput, the number of instructions finished per second. A deeper pipeline, with more and shorter stages, also allows a faster clock. The cost is hazards, situations where the next instruction cannot proceed:

• Data hazards. In the loop above, add t0, t0, t1 needs the value that ld t1, 0(a0) is still loading. Processors forward results straight from one stage to the next instead of waiting for them to reach the register file, and stall (insert a bubble) only when the value genuinely is not ready yet.
• Control hazards. At bnez a1, loop the processor does not know which instruction comes next until the branch is resolved, several stages later.
• Structural hazards. Two instructions need the same piece of hardware in the same cycle; designers avoid these by duplicating hardware, such as the split instruction and data caches.

Branch prediction

In typical integer code roughly one instruction in five to seven is a branch,⁴⁷ and a deep pipeline cannot afford to wait for each one. So processors predict the outcome and keep fetching down the predicted path; if the guess was wrong, they discard the work and restart from the right place.

Early schemes simply guessed that backward branches (loops) are taken. James Smith’s 1981 study showed that remembering each branch’s recent behaviour in a small table of two-bit counters predicts more accurately and more consistently than fixed rules.⁴⁸ Modern predictors combine many such tables indexed by the history of recent branches and are right well over 95 percent of the time on typical code.⁴⁷ They need to be: on a current desktop core a misprediction throws away around 15 to 20 cycles of work, during which the core could have started several instructions per cycle.⁴⁹

Superscalar and out-of-order execution

A pipelined processor finishes at most one instruction per cycle. A superscalar processor has several pipelines and can start several instructions in the same cycle; large modern cores can decode, rename and retire around eight per cycle.⁵⁰

To keep that many units busy, the processor cannot simply run instructions in program order: whenever one is waiting, for example on a load that missed in the cache, everything behind it would wait too. Out-of-order execution looks ahead in the instruction stream for independent work.

1. The front end predicts branches, fetches instructions and decodes them into micro-operations, in program order.
2. Register renaming maps each architectural register (such as t0) onto one of a much larger pool of physical registers. This removes false dependencies between instructions that merely reuse the same register name.
3. A scheduler holds waiting micro-operations and issues each one to an execution unit as soon as its inputs are ready, which is Tomasulo’s algorithm in modern form.⁹
4. The reorder buffer keeps every micro-operation in its original order until it completes, then retires them in that order, making their results officially visible. If a branch was mispredicted or an instruction faulted, everything after it is discarded as if it had never run.

Modern cores keep several hundred instructions in flight this way; Intel’s Lion Cove core, for example, has a 576-entry reorder buffer.⁵⁰ The illusion they maintain is the important part: the program sees exactly the results it would have seen if each instruction had run one at a time, in order.

Speculation and its side effects

Out-of-order execution and branch prediction together mean a processor constantly runs instructions it is not sure it needs, and throws away the results if it guessed wrong. The results are discarded, but not every trace of them: data that a discarded instruction loaded is still in the cache. Spectre showed that an attacker can trick the branch predictor into speculatively reading a secret and then detect, from cache timing, which cache line the secret touched.⁴⁰ Meltdown showed that some processors would speculatively read kernel memory from user code before checking permission.⁴¹ Fixes have combined microcode updates, operating system changes (such as keeping kernel memory almost entirely out of user page tables)⁵¹ and new hardware designs.

The memory hierarchy

A processor core can start several instructions every nanosecond, but reading main memory takes on the order of a hundred nanoseconds. William Wulf and Sally McKee called the widening gap between processor and memory speed the memory wall.⁵² The answer is a hierarchy of memories, each larger and slower than the one above it:

Caches work because programs have locality:

• Temporal locality. Data used recently is likely to be used again soon, such as a loop counter or the top of the stack.
• Spatial locality. Data near recently used data is likely to be used soon, such as the next element of an array.

Typical figures for current cores are an L1 hit in about four cycles and a trip to main memory of roughly a hundred nanoseconds.⁵⁰⁵³ Caches move data in cache lines, typically 64 bytes,⁵⁴ so reading one array element brings its neighbours along for free. When a line is needed and the cache is full, the cache evicts an older line, usually the least recently used. How you walk through memory can therefore matter more than how many operations you do. These two functions add up the same numbers:

#define N 4096
static double m[N][N];

double sum_rows(void) {       /* walks memory in order: uses every byte of each line */
    double s = 0;
    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
            s += m[i][j];
    return s;
}

double sum_columns(void) {    /* jumps 32 KiB per step: a new cache line every time */
    double s = 0;
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            s += m[i][j];
    return s;
}

C stores each row contiguously, so sum_rows reads memory sequentially, while sum_columns touches a different cache line, and with 4 KiB pages a different page, on every addition. On a typical machine the second is several times slower.⁵⁵

With several cores, each with its own caches, the hardware must also keep them coherent: when one core writes to a line, copies of it in other cores' caches must be updated or invalidated. Protocols such as MESI track whether each line is modified, exclusive, shared or invalid.⁴²⁵⁴

Vectors, cores and threads

Beyond running one instruction stream fast, processors find parallelism in three more ways:

• SIMD (vector) instructions apply one operation to several values packed in a wide register: four, eight or sixteen numbers in one instruction. x86 has SSE and AVX, ARM has NEON and SVE, and RISC-V has its V extension. Compilers use them automatically for simple loops like sum above, and libraries for graphics, media and machine learning use them heavily.
• Multiple cores run separate instruction streams, which is why multithreaded programs matter.
• Simultaneous multithreading (SMT), which Intel calls Hyper-Threading, lets one core hold the state of two threads and fill idle execution units from either, recovering some of the time one thread spends waiting.

Power and the clock

The power a chip burns switching its transistors grows roughly as C × V² × f: the capacitance being switched, the square of the supply voltage, and the clock frequency.⁴²⁵⁶ Since a higher frequency also needs a higher voltage, power climbs much faster than speed. That is why the clock race ended, and why every processor now constantly adjusts its own speed: dynamic voltage and frequency scaling lowers voltage and clock when there is little to do, and boost or turbo modes raise them briefly while there is thermal headroom. Chips with hybrid designs, such as big.LITTLE, schedule background work on small efficient cores and save the large ones for work that needs them.

Manufacturing process nodes are still named in nanometres (“7 nm”, “3 nm”), but the names began to drift away from physical dimensions in the late 1990s and are now labels for a generation of technology, not the size of any feature on the chip.⁵⁷

Putting it together: one trip round the loop

Following the sum loop through a modern out-of-order core shows most of the machinery at work in a few nanoseconds.

1. Predict. The branch predictor has seen bnez a1, loop taken many times, so it predicts taken again and tells the front end to keep fetching from loop without waiting.
2. Fetch and decode. The front end reads the next several instructions from the L1 instruction cache and decodes them, often a whole loop iteration per cycle.
3. Rename. t0, t1, a0 and a1 are mapped to fresh physical registers, so iteration k+1’s addi a0, a0, 8 does not have to wait for iteration k’s load to finish reading a0.⁴⁹ Several iterations are now in flight at once.
4. Execute out of order. The pointer and counter updates are ready at once and run immediately. Each ld goes to a load unit; if the element is in the L1 data cache, it arrives in about four cycles, and the matching add is issued the moment it does. If it misses, the core keeps working on later iterations whose data is already there, while the hardware prefetcher, having noticed that the loop walks memory in order, has probably already started fetching the next lines.
5. Retire. As each instruction completes, the reorder buffer retires it in program order, and its result becomes part of the official state.
6. Mispredict, once. On the last iteration the branch falls through. The predictor guessed wrong, the core discards the speculative instructions from the iteration that never happened, and fetching restarts at done.

To the program, all of this is invisible: it ran one instruction after another, exactly as written.

Why it matters

For most of computing history, software got faster for free as processors improved. That ended with the power wall, and what decides performance now is much more visible to programmers: whether code can use several cores, whether its loops can use vector instructions, whether its data is laid out so the caches and prefetchers can do their job, and whether its branches are predictable. Knowing how the processor really works, and the illusion of simple sequential execution it maintains, explains why two programs that do the same arithmetic can differ in speed by an order of magnitude.

References