MTHE 474 Information Theory Units: 3.50
Topics include: information measures, entropy, mutual information, modeling of information sources, lossless data compression, block encoding, variable-length encoding, Kraft inequality, fundamentals of channel coding, channel capacity, rate-distortion theory, lossy data compression, rate-distortion theorem.
(Lec: 3, Lab: 0, Tut: 0.5)
(Lec: 3, Lab: 0, Tut: 0.5)
Offering Term: F
CEAB Units:
Mathematics 6
Natural Sciences 0
Complementary Studies 0
Engineering Science 23
Engineering Design 13
Offering Faculty: Faculty of Arts and Science
Course Learning Outcomes:
- Computing Shannon's information measures (entropy, Kullback-Leibler distance and mutual information).
- Computing the capacity of communication channels.
- Reasoning about the properties of Shannon's information measures (entropy, Kullback-Leibler distance and mutual information).
- Using mathematical tools to infer properties of coding and communication systems.
- Probabilistic modeling of communication systems for source and channel coding purposes.
- Using tools from probability theory to analyze communication systems.
- Metric assessment of data compression code designs.