ECE 515
Information Theory

Course Materials and References

• Course Notes by Jean-Yves Chouinard

• Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas

• Information Theory Lecture Notes by Stefan M. Moser

• A First Course in Information Theory by Raymond W. Yeung

• Information Theory, Inference, and Learning Algorithms by David J.C. MacKay

• "A Mathematical Theory of Communication," by Claude E. Shannon

• "Transmission of Information," By R. V. L. Hartley

• "Data Compression," by Debra A. Lelewer and Daniel S. Hirschberg

• "The ABCs of Linear Block Codes," by Bernard Sklar and Fredric J. Harris

Course Slides

• Introduction

• Joint Entropy, Equivocation and Mutual Information

• Decision Trees

• Logistic Regression

• Distortionless Source Coding 1

• Typical Sequences

• Distortionless Source Coding 2

• Channel Capacity and Coding

• Differential Entropy

• Review