Source Coding Theory

by Robert M. Gray

Source coding theory has as its goal the characterization of the optimal performance achievable in idealized communication systems which must code an information source for transmission over a digital communication or storage channel for transmission to a user. The user must decode the information into a form that is a good approximation to the original. A code is optimal within some class if it achieves the best possible fidelity given whatever constraints are imposed on the code by the available channel. In theory, the primary constraint imposed on a code by the channel is its rate or resolution, the number of bits per second or per input symbol that it can transmit from sender to receiver. In the real world, complexity may be as important as rate. The origins and the basic form of much of the theory date from Shan­ non's classical development of noiseless source coding and source coding subject to a fidelity criterion (also called rate-distortion theory) [73] [74]. Shannon combined a probabilistic notion of information with limit theo­ rems from ergodic theory and a random coding technique to describe the optimal performance of systems with a constrained rate but with uncon­ strained complexity and delay. An alternative approach called asymptotic or high rate quantization theory based on different techniques and approx­ imations was introduced by Bennett at approximately the same time [4]. This approach constrained the delay but allowed the rate to grow large.

202 pages, Hardcover

First published October 31, 1989

About the author

Robert M. Gray, of Stanford University.

See also:
Robert M. Gray Jr., editor of Technology Integration in Higher Education.
Robert M. Gray, author of The Church and the Older Person.

Robert M. Gray was born in San Diego, Calif., on November 1, 1943. He received the B.S. and M.S. degrees from M.I.T. in 1966 and the Ph.D. degree from U.S.C. in 1969, all in Electrical Engineering. Since 1969 he has been with Stanford University, where he is currently the Lucent Technologies Professor of Engineering, Emeritus, and Professor of Electrical Engineering, Emeritus. His research interests are in information theory and signal processing, especially in the theory and practice of quantization, compression, and classification.
He was a member of the Board of Governors of the IEEE Information Theory Group (1974-1980, 1985-1988) and of the IEEE Signal Processing Society (1998-2001). He was Associate Editor for Source Coding (1977-1980) and Editor-in-Chief (1980-1983) of the IEEE Transactions on Information Theory. He is currently the Editor-in-Chief of Foundations and Trends in Signal Processing. He was Co-Chair of the 1993 IEEE International Symposium on Information Theory and Program Co-Chair of the 1997 and 2004 IEEE International Conference on Image Processing.

He is a Fellow of the Institute of Mathematical Statistics and the IEEE and has held fellowships from the Japan Society for the Promotion of Science at the University of Osaka (1981), the John Simon Guggenheim Foundation at the University of Paris XI (1982), and NATO/Consiglio Nazionale delle Ricerche at the University of Naples (1990). During spring 1995 he was a Vinton Hayes Visiting Scholar at the Division of Applied Sciences of Harvard University. He is a Faculty Affiliate of the Clayman Institute for Gender Studies at Stanford University, where he was a Faculty Research Fellow during the 2008-2009 academic year.

He was corecipient of the 1976 IEEE Information Theory Group Paper Award and the 1983 IEEE ASSP Senior Award. He was awarded an IEEE Centennial medal (1984) and an IEEE Third Millennium Medal (2000). He received the 1993 Society Award, the 1998 Technical Achievement Award, the 2005 Meritorious Service Award, and the 2009 Education Award from the IEEE Signal Processing Society. He received a Golden Jubilee Award for Technological Innovation (1998) and the 2008 Shannon Award from the IEEE Information Theory Society. He received the 2008 IEEE Jack S. Kilby Signal Processing Medal. He received a 2002 Presidential Award for Excellence in Science, Mathematics and Engineering Mentoring (PAESMEM) and the 2003 Distinguished Alumni in Academia Award from the University of Southern California. He is a member of the National Academy of Engineering (2007). He holds an Advanced Class Amateur Radio License (KB6XQ).