AI Systems Performance Engineering: Optimizing Model Training and Inference Workloads with GPUs, CUDA, and PyTorch

by Chris Fregly

Elevate your AI system performance capabilities with this definitive guide to maximizing efficiency across every layer of your AI infrastructure. In today's era of ever-growing generative models, AI Systems Performance Engineering provides engineers, researchers, and developers with a hands-on set of actionable optimization strategies. Learn to co-optimize hardware, software, and algorithms to build resilient, scalable, and cost-effective AI systems that excel in both training and inference. Authored by Chris Fregly, a performance-focused engineering and product leader, this resource transforms complex AI systems into streamlined, high-impact AI solutions.


Inside, you'll discover step-by-step methodologies for fine-tuning GPU CUDA kernels, PyTorch-based algorithms, and multinode training and inference systems. You'll also master the art of scaling GPU clusters for high performance, distributed model training jobs, and inference servers. The book ends with a 175+-item checklist of proven, ready-to-use optimizations.


Codesign and optimize hardware, software, and algorithms to achieve maximum throughput and cost savings
Implement cutting-edge inference strategies that reduce latency and boost throughput in real-world settings
Utilize industry-leading scalability tools and frameworks
Profile, diagnose, and eliminate performance bottlenecks across complex AI pipelines
Integrate full stack optimization techniques for robust, reliable AI system performance

1906 pages, Kindle Edition

Published November 11, 2025