1. The core mental model

The cloud is a giant shared computer that you control through an API. It exists because of one economic fact: most computers sit idle most of the time, and pooling them across thousands of customers makes each hour of use cheaper.

Why the cloud exists

Before the cloud, a company bought servers, waited weeks for delivery, and sized them for its busiest day. The rest of the year those machines sat mostly idle, but were still paid for, powered and cooled.

The cloud changes three things:

• Capital becomes operating cost. You rent by the hour instead of buying hardware that depreciates over 3–5 years.
• Demand gets pooled. Thousands of customers peak at different times, so the provider runs its fleet at higher average use than any single company could.
• Scale buys efficiency. Providers negotiate hardware in bulk, design their own chips and data centers, and spread operations staff across millions of machines.

The formal definition (from the US standards body NIST) names five properties. On-demand self-service: you create resources yourself through an API. Broad network access: reachable over the network. Resource pooling: many tenants share hardware. Rapid elasticity: capacity grows and shrinks in minutes. Measured service: you pay for what is metered.

The four primitives

Every cloud product is built from four things:

1. Compute - something that executes instructions: CPU cores, GPUs, and the memory attached to them.
2. Storage - something that keeps bytes when the power goes off: disks, SSDs, object stores.
3. Network - something that moves bytes between machines, and between the cloud and the outside world.
4. Control plane - the API layer that creates, configures, permissions and bills the other three. Identity and access live here.

Everything else is software running on these four, sold as a managed product. A managed Postgres database is a VM, a disk, a network endpoint, automated backups and someone else doing the patching. A model API such as Amazon Bedrock is GPUs, serving software, and a meter that counts tokens.

This is the first-principles move: when you see a product, ask which primitives it bundles and what work it removes from you. The price premium over the raw primitives is what you pay for that work.

Six constraints everything obeys

These never change, no matter which provider or product. Most “why is it designed this way?” questions trace back to one of them.

1. Physics. Light in fiber travels about 200,000 km/s, roughly 5 microseconds per km. Frankfurt to US East is about 6,000 km, so a round trip is at least 60 ms and usually 80–90 ms. No software makes that faster.
2. Power and heat. A modern AI server rack draws 40–130 kW, versus 5–10 kW for a traditional rack. AI data centers are now limited more by electricity and cooling than by land or chips.
3. The memory hierarchy. The closer data sits to a processor, the faster and smaller that storage is. Most AI performance problems are about moving bytes, not doing math.
4. Failure is normal. Across thousands of machines, disks, GPUs and links fail every day. Systems are designed to survive failure, not to avoid it.
5. Utilization economics. You pay for reserved capacity whether you use it or not. An idle GPU costs exactly as much per hour as a busy one.
6. Data gravity. Moving large data is slow and providers charge to move it out. So compute tends to move to where the data already lives.

The ladder of abstraction

You can rent computing at different levels. Each step up the ladder means less to manage and faster shipping, in exchange for less control and usually a higher price per unit of work.

Level	What you rent	You manage	AI example
Bare metal	A whole physical server	Everything from the OS up	A dedicated 8-GPU node from a GPU cloud
IaaS (VMs)	A virtual machine	OS, runtime, app, scaling	A GPU VM you SSH into and run PyTorch on
Containers / Kubernetes	A cluster that runs your containers	Containers, configs, cluster policies	vLLM pods on a managed Kubernetes GPU node pool
PaaS / serverless	A platform that runs your code	Code and settings only	A function on a serverless GPU platform
Managed AI service	A finished capability	Your data and prompts	Calling a model API and paying per token

Where you sit on this ladder is the most important architecture decision in AI systems work. A team of two shipping a product should usually start high (APIs, serverless) and move down only when cost, latency, privacy or control forces it.

Check yourself

• Why can a provider rent you a server for less than it would cost you to own one at 15% use? When does owning win?
• Which four primitives sit under a managed vector database, and what work does the managed version remove?
• A startup keeps 50 TB of training data in Google Cloud Storage. Which constraint explains why it will probably train on Google Cloud too?
• Your app in Frankfurt calls a model API hosted in Virginia. What is the minimum latency added per call, and why?