Test your understanding of high-throughput LLM inference architectures, GPU memory hierarchy limitations, KV cache VRAM footprint calculations, PagedAttention block allocation, FlashAttention online softmax tiling, and continuous batching iteration scheduling.
0 / 6
How do the prefill (prompt processing) and generation (autoregressive decoding) phases of LLM inference differ in arithmetic intensity, and why does decoding become memory bandwidth-bound?