Copyright © Antonio D'Angelo, Univ. of Udine

Multimedia Systems Syllabus

1. Aims

   The aim of the course is that to acquaint a student with a number concepts about the signal processing exploited in the field of multimedia applications. Some basic tools, such as Fourier and Cosine transforms, are presented but also the Nyquist's theorem and a short introduction to digital filter design and usage. The general notion of waveform is introduced and discussed along with a short prosentation of typical filter arrangements: low-band, high-band, pass-band, etc... Also the Radon transform is mentioned as an application on computed tomography (CT) applications. Different color models are also discussed and the concepts of Chrominance, Luma and Luminance are introduced within the general mechanism of human vision system. The most popular compression algorithms are widely discussed as well as audio systems such as DIMI, MPEG and MPEG4. The JPEG compression is presented as an instance of the so called Discrete Cosine Transform even if the complete compression cycle is built on the entropy-based compression algorithms.

2. Method

   The course is intended to understand the signal processing usage within their application in multimedia systems. The staring point is a short introduction to waveforms, continuous and digital signals, filtering, Nyquist's theorem, Fourier transform with its discrete Fast Fourier Transform (FFT). The notion of modulus and phase spectra are also considered in the frequency domain. The principles of data compression are introduced starting from the basic concept of entropy, as it has been introduced by Shannon. Different coding algorithms are presented and discussed. The course terminates with a short presentation of the descrete cosine transform (DCT) and the JPEG compression.

3. Contents

   The topics are primarily based on signal processing, starting from the basic notions of waveform, frequency, amplitude, phase and spectra. Afterwards digital filters and Fourier Transforms are introduced along with the Nyquist's theorem. After having considered the general properties of audio and video data, a number of compression algorithms are presented and discussed.

   In the following you can find a schematic program of the course. A more precise scheduling of the lectures are available as detailed outlines of their contents.

   • Digital Signal Processing

     • Waveforms
     • Amplitude, Frequency and Phase
     • Signal to Noise ratio and Decibel Scales
     • Signal Flow Graphs
     • Digital Signal Filtering
     • Finite Impulse Response

   • Fourier Transform

     • Frequency components of Audio and Video Data
     • Frequency Domain
     • Fourier's theorem: 1D and 2D transforms
     • Magnitude and Phase Spectra
     • Time-Frequency Representation

   • Digital Filters

     • Low Pass Filters: Ideal, Butterworth
     • Filtering Noisy Images
     • High Pass and Band-Pass Filters
     • Fourier Transform and Convolution

   • Multimedia Data

     • Discrete and Continuous Media
     • Analog and Digital Signals: Analog­/­Digital Converter
     • Text and Static Data
     • Audio: digitising Sound
     • Graphics, Images and Video
     • Digital Sampling: Nyquist’s theorem

   • Digital Audio: MIDI and MPEG-4

     • Nyquist Sample Rate and Bit Size
     • MIDI: definition, components, hardware aspects
     • Messages, Channels
     • Structured Audio
     • MIDI Semantics and Control
     • Common Digital Audio Formats

   • Graphics, Images and Videos

     • Graphic/Image File Formats: 24-bit and 8-bit colors
     • Bit­Maps, Gray Scale and Dithering
     • Graphic Formats: GIF, JPEG, TIFF, PNG, EPS
     • RGB and CMY Color Models
     • Chrominance, Luma, Luminance and Gamma Correction
     • Color Vision: Photoreceptors, Cone Sensitivity and Color Properties
     • Color Video Signals: NTSC and PAL
     • Chroma Subsampling, Aliasing
     • Temporal Aliasing

   • Compression Algorithms

     • Need for Compression: Basics of Information Theory
     • Shannon and Kolmogorov: Lossless and Loss Compression
     • Lossless Compression Algorithms: Repetitive Sequence Suppression, Run-Length Encoding, Pattern Substitution, Entropy Encoding; Shannon-Fano Algorithm; Huffman Coding; Arithmetic Coding; Simple Arithmetic Coding
     • Decoding
     • Binary Arithmetic Coding: Code generation
     • Arithmetic Coding
     • Lempel-Ziv-Welch Algorithm: Compression and Decompression;
     • Source Coding Techniques: Transform and Differential Coding
     • Frequency Domain
     • Vector Quantization
     • K-Means

   • Discrete Cosine Transform

     • Moving into the Frequency Domain
     • Relationship between DCT and FFT
     • 1D DCT and 2D DCT
     • Performing DCT Computations
     • Compression with DCT
     • Separability
     • 2D DCT on Image Blocks
     • 2D DCT Basis Functions

   • JPEG

     • Basic JPEG Compression Framework
     • Major Coding Algorithms
     • Quantization
     • Zig-zag Scan
     • Differential Pulse Code Modulation
     • Run Length Encode
     • Huffman Coding
     • Artefacts and Gibb's Phenomenon

4. Suggested Readings

   • Fundamentals of Multimedia: Ze-Nian Li & Mark S. Drew, Pearson Prentice Hall, 2004
   • Digital Signal Processing: Steven W. Smith, California Technical Publishing, 1999
   • Digital Signal Processing System-Level Design Using LabVIEW: Nasser Kehtarnavaz & Namjin Kim, Elsevier, 2005
   • Digital Signal Processing: a Filtering Approach: Steve White, Delmar Cengage Learning, 2000
   • Design of Digital Filters: J. Fessler, excerpt from lectures, 2004
   • Digital Filters: Saeed Vaseghi, excerpt from "Multimedia Signal Processing: Theory and Applications in Speech, Music and Communications", 2007, Wiley publisher
   • Introduction to Data Compression: Guy E. Blelloch, excerpt from "Algorithms in the Real World", 2001
   • Introduction to Nuclear Magnetic Resonance Imaging: J. Fessler, excerpt from lectures, 2009
   • Lectures from the Course, available via Web