Cloud Computing for AI Systems from First PrinciplesHow to use this guide · 19 of 19 written
  1. How to use this guide
  2. 01The core mental model
  3. 02The physical layer
  4. 03Virtualization: how one machine becomes many
  5. 04Compute for AI: GPUs and accelerators
  6. 05Back-of-envelope math for AI workloads
  7. 06Storage and data
  8. 07Networking
  9. 08Identity, security and compliance
  10. 09Orchestration: running fleets, not machines
  11. 10Infrastructure as code and delivery
  12. 11Distributed training in the cloud
  13. 12Inference and serving
  14. 13Reliability and observability
  15. 14Cloud economics
  16. 15The provider landscape
  17. 16Fluency toolkit: the questions to ask
  18. 17Hands-on learning path
  19. 18Glossary
  20. 19Resources worth your time

Cloud Computing for AI Systems from First Principles

AI Systems Infrastructure · 19 posts1h 27m read

Cloud computing is renting someone else's computers through an API, by the hour or by the request. Every product you will meet is a combination of a few physical resources: processors, memory, disks, network links and power. Learn those resources and their limits, and a new service stops being a name to memorize and becomes a puzzle you can take apart.

How to use this guide

The guide is built bottom-up, so each part leans on the ones before it:

  • Parts 1–2 - what the cloud physically is, and the constraints nothing escapes.
  • Parts 3–8 - the building blocks: compute, GPUs, storage, networking, identity.
  • Parts 9–10 - how systems are assembled and automated.
  • Parts 11–14 - AI systems: training, inference, reliability, cost.
  • Parts 15–17 - the provider landscape, the questions to ask, and a hands-on path.

The one habit that makes you fluent

When you meet any cloud service, run it through seven questions. After a few weeks this becomes automatic, and it is what separates people who understand the cloud from people who know product names.

  1. What physical resource sits underneath? CPU, GPU, RAM, disk, or network.
  2. What is abstracted away, and what do I still own? The OS, scaling, patching, data?
  3. How is it billed? Per hour, per request, per GB stored, per GB moved out.
  4. What limits it? Throughput, latency, quotas, maximum sizes, available capacity.
  5. How does it fail? What breaks, how often, and what happens to my data.
  6. How does it scale? Bigger machine (up), more machines (out), or not at all.
  7. Who can touch it? Which identity, which permission, from which network.

A worked example, AWS Lambda (run a function without managing a server):

Question Answer
Underneath A slice of a shared server, isolated in a tiny VM (Firecracker)
You still own Your code, its dependencies, its permissions
Billing Per request plus memory × seconds of runtime
Limits 15-minute maximum run, about 10 GB memory, no GPU, cold starts
Failure Async calls are retried; your code must tolerate running twice
Scaling Out, automatically, up to an account concurrency quota
Access An IAM role attached to the function

That table tells you at once that Lambda is wrong for serving a large model (no GPU, run limit) and right for glue code around one.

Conventions

  • Numbers are approximate and from general knowledge as of mid-2026. Hardware and prices move fast, so check the provider’s page before relying on one.
  • AWS names are used as the default example because they are the most common in job posts and docs. Part 15 maps every term to Google Cloud and Azure.
  • Each part ends with Check yourself questions. If you can answer them out loud without notes, you own that part.