ECE 515
Information Theory

Course Materials and References

• Course Notes by Jean-Yves Chouinard

• Elements of Information Theory, 2nd Edition, Thomas M. Cover and Joy A. Thomas

• Information Theory Lecture Notes by Stefan M. Moser

• A First Course in Information Theory by Raymond W. Yeung

• Information Theory, Inference, and Learning Algorithms by David J. C. MacKay

• Principles of Data Science, OpenStax

• "A Mathematical Theory of Communication," by Claude E. Shannon

• "Transmission of Information," by R. V. L. Hartley

• "Data Compression," by Debra A. Lelewer and Daniel S. Hirschberg

• "The ABCs of Linear Block Codes," by Bernard Sklar and Fredric J. Harris

Course Slides

• Introduction

• Joint Entropy, Equivocation and Mutual Information I

• Decision Trees

• Joint Entropy, Equivocation and Mutual Information II

• Logistic Regression

• Perplexity

• Distortionless Source Coding 1