M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Unit 2: MULTIMEDIA INFORMATION REPRESENTATION: Introduction, digital principles, text, images, audio, video. 7 Hrs Multimedia Information Representation Introduction Digital form - store and process form in the computer - for all types of multimedia information Textual inforamtion: contains strings of characters - keyboard through codeword: each character represented - by a unique combination of fixed number of bits complete text: hence, can be represented by strings of codewords Image: computer-generated graphical graphical images: made up of a mix of o f lines, circules, squares, and so on - each represented in a digital form Ex.: line - represted by start and end coofrdinates of the line - each coordinate being defined in the form of a pair of digital d igital values - relative to the complete image Audio and video: microphone and video cameras: produce electrical signals - whose amplitude varies continuously with time - amplitude indicating the magnitude of the soundwave/image-intensity at that instant Analog: signal whose amplitude varies continuously with time Store and process analog signal type of media in a computer: to covert any time-varying analog signals into oa digital form - is necessary for speech and audio - in like, loud speakers, and for display of digitized images in like, computer monitors - digital values of media types must be converted back again into a corresponding time-varying analog form on output - from the computer for a particular media type: Signal encoder: electric circuit - converts analog signal into a digitla form it involves: 1. sampling: having the amplitude of analog signasl - at repetitive time intervals
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
2. quantization: converting - amplitude of each sample - into a corresponding digital value Signal decoder: electric circuit - converts - stored digitized samples - to their corresponding time-varying analog form all media types: associated with the various multimedia applications stored and processed within a computer in an all-digital form so, different media types can be readily integrated together - resulting integrated bitstream can be transmitted over a single all-digital communication network Digitization principles Analog signals Fig. shows - general g eneral properties relating to any time-varying analog signal Fig. a - amplitude of signals varies cotinuously with time Fourier analysis: mathematical tenchique - used to show that any analog signal is made up of - a possibly infinite number of single-frequency sinusoidal signals - whose amplitude and phase vary continuosuly - with w ith time relative to each other Ex.: highest and lowest frequency components of t he signal shownw in Fig. may be those shown in Fig. signal bandwidth: range of frequencies of the sinusoidal components - that make up a signal Fig. shows two examples - relate to an audio signal first - relate to a speech signal second - relate to a msic signal produced by - say, an orchestra Speech - humans produce sounds, which are converted into electrical singals - by a microphone - made up of a range of sinusoidal signals varying in frequency between 50Hz and 10kHz For music: range of signals is wider - varies between 15kHz to 20kHz - being comparable with the limits of the sensitivity of the ear Analog signal - when, being transmitted through a network BW of transmission channel (range of frequencies, channel will pass) >= BW of the signal if BW of channel < BW of signal - some low and/or high frequency components - will be lost, thereby degrding degrding the quality of th received signal - such, a channel is called the bandlimiting channel - as in Fig.
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Encoder design Encoder: electronic circuit - converts, time-varying analog signals to digital form Fig. shows - principles of an encoder consists of 2 main circuits - bandlimiting filter and ADC (Analog to Digital Converter) ADC - contains sample-and-hold sample-and-hold and a quantizer waveform set: Fig. shows the typical waveform set - for the signal encoder bandlimitng filter: remove selected higher-frequency components from the source signal A sample-and-hold: got output of bandlimiting filter, B signal used to sample - amplitude of the filtered signal - at regular time intervals - C signal, and to hold the sample ampitude constant between samples - D signal Quantizer circuit - got signal D converts ech sample amplitude amplitude - into - a binary value - known as as a codeword codeword - like E signal polarity (sign) of sample: positive or negative - relative to the zero level indicated by - most significant bit of each codeword binary 0: indicates - a positive value binary 1: indicates - a negative value From Fig. from - time-related set of waveforms: to represent the amplitude amplitude of a time-varying analog signal signal precisely precisely require: 1. signal should be sampled at a rate > maximum m aximum rate of change of signal amplitude 2. number of quantization levels - used - to be - as large as possible sampling rate Nyquist sampling theorem: states that - for f or an accurate representation of a time-varying anlaog signal - it's amplitude must bee sampled at a minimum rate tha tis sequal to or greater than twiwce th highest sinusodial frequency component that is present in the signal - known as Nyqusit rate, normally represented as either Hz o r, or correctly, samples per second (sps) samplin signal at a rate < Nyquist rate: results, in additional frequency components - being generated that are not present in the original signal - which, in turn cause original signal to become distorted Fig. shows - effect e ffect of undersampling single-frequency sinusoidal signal caused by sampling
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
a signal at a rate lower than the Nyquist rate Ex.: original signal is assumed to: 6kHz sinewave sampling rate(8ksps)
) include - commands to start: new paragraph (), start and end boldface ( text ), present in the form of a bulleted list ( list ), include an image (
), and so on Hypermedia - other media types, such as sound and video clips - can also be included hypermedia and hypertext: terms - often used u sed interchangeably - when referring to pages created in HTML specification of a hyperlink: made by specifying both the URL - where, the require page is located together with the textual name of the link Ex.: specification of a hyperlink to a page containing - 'Further details' would have the form Further details so, hypertext string - is similar to formatted text string - except, tht at the linkage points within a page additional text strings are found, tha define the URL of the linked page
Images Images include - computer-genera computer-generated ted images computer graphics(or simply, graphics):are digitized images of both documents and pictures 3 types of images: 1. Graphics 2. Digitized documents 3. Digitized pictures are displayed and printed - in 2-D matrix form - of individual i ndividual picture elements
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Pixels (Pels): individual picture elements each type - is represented - differently within the computer memory, or more generally, in a computer file each type - of image - is created differently
1. Graphics computer graphics: creation for - a range of software packages and programs - are available provide - easy-to-use tools - to create graphics - which, are composed of all kinds of visual objets including - lines, arcs, squares, rectangles, circles, ovals, diamonds, stars, and so on as well as - any form of hand-drawn (normally referred to as freeform) objects produced by drawing - desired shape on the screen - by means of a combination of a cursor symbol on the screen - Ex.: pencil or paint brush, and the mouse facilities are also provided - to edit these objects - Ex.: to change their shape, size, or color - and, to t o introduce complete predrawn images, either previously created by the author of the graphic or clip-art (selected from a gallery of images that come with the package) - better packages provide many hundreds of such images Textual information: can include a graphic - together with - precreated tables and graphs and digitized pictures and photographs - which have been previously obtained objects - can overlap - each other with - selected object nearer to the front than another Additon of with - add fill and add shadows - to objects - to give the complete 3-D effect
computer's display screen: can be cosidered - as made up of - 2-D matrix of individual pixels - each of which can have a range of colors - associated with it Ex.: VGA (Video Graphics Array): common type t ype of display Fig. shows - a matrix of 640 horizontal pixels by 480 vertical piexels For Ex.: 8 bits/pixel - which allows each pixel to have one of 256 different colors All objects: including the free-form objects - made up of a series of lines - connected to each other curved line - as what may appear - in practice, is a series of very short lines - each
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
made up of a string of pixels - which, in the limit, have the resolution of a pair of adjacent pixels on the screen Fig. shows - some examples Attributes: each object - has a number of attributes - associated with it include - its shape - a line, a circle, a square, and so on - its size - in terms of - pixel p ixel positions of its border coordinates, color of border, its shadow, and so on Editing of an object - involves - simply, changing selected attributes associated with the object Ex.: Fig as in, square can be moved to different location on the screen - by simply, changing its border coordinates and leaving the remaining attributes unchanged Object shape: said to be either open or closed Open object: start of the first line and end of the last line - that make up the object's border are not connected - i.e., they do not start and end on the same pixel closed object: start of the first line and end of the last line - that make up the object's border are connected - i.e., they start and end on the same pixel closed object: color-fill:can be done -i.e., - pixels enclosed - by its border - can can all be assigned the same color - to rendering (creation of the solid objects, by color-fill) - as in Fig. All obejects - are drawn on the screen by the e user simply specifying - name of the objects and its attributes - including - its color-fill and shadow effect if required set of more basic lower-level commands: are then used - to determine bth the pixel locations that are involved - and the color that should be assigned to each pixel
Representation of a complete graphic - is analogous to - the structure of a program written in a high-level h igh-level programming language language Ex.: program consists of a main body together with a number of prcoedures/functions: prcoedures/function s: each of of which has a set of parameters associated with it and performs a specific function
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Graphic: consists of the set of commands (each with attributes) - necessary to draw the different objects - that make up the graphic procedures/functions procedures/function s in the program: can be the mix of those - created by the writer of the program ad those available as library procedures/functions procedures/functions so, objects creatded by the graphic - can be either those created by - the author or those selected from - set of standard objects or the clip-art gallery procedurs/funtions in the program: may in turn call - a number of lowerlevel funtions so, the commands associated with objects use the lower-level commands - to display the objects on the screen Main body of the t he program:concerned with - invoking the various procedures/functions procedures/function s - in the order nencessary to implement - a particular computational task - so, main body of the graphic representation is concerned with - invoking the different object commands - in the correct sequence - to create the desired graphic taking into account any overlapping objects 2 forms of representation of computer graphic 1. high-level version (similar to the source code of a high-level program) 2. actual pixel-image of the graphic (similar to the byte-string, corresponding to the low-level machine code of the program - known generally, as bit-map format) graphic can be transferred - over a network in either form 1. high-level program form: much more compact requires less memory - to store s tore the image requires less BW - for its transmission destination must - be able to interpret various high-level commands Bit-map form- used to avoid above requirements there are a number of standardized forms - of representation such as GIF (Graphical Interchange Format) an TIFF (Tagged Image File Format) SRGP (Simple Raster Graphics Package): convert the high-level langauge into a pixel-image form
2. Digitized documents Ex.: document produced by the scanner associated with a facsimile (fax) machine Fig. shows - principles of facsimile (fax)
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
scanner - associated with with the fax machine - operated operated by - scanning each complete page from left to right - to produce - a sequence of scan lines that start at the top of the page and end at a t the bottom vertical resoultion - of scannin procedure - is either 3.85 or 7.7 lines/mm - which is equivalent to approximately - 100 or 200 lines/inch As - Each line is scanned - output of the scanner - is digitized to a resolution - of approximately 8pels with fax machines/mm Fax machines: use just a single binary digit to represet each pel - 0 for white pel is used 1 for black pel is used Fig. shows - digital representation of the scanned page, which produces a stream of about two million bits printer of fax: then, reproduces reproduces - original image image received stream bits to a similar s imilar resoultions Use of fa single binary digit per pel - means, m eans, fax to scanning bitonal (black-and-white) images - such as printed mainly textual information
page - for a typical
by printing out the
machines are best suited documents comprising
Digitized pictures scanners: used for didgitizing continuous-tone monochromatic images (such as, printed picture, or scene) - normally, more than a single bit is used to digitize each pel Ex.: good quality black and white pictures - can be obtained - by using 8bits/pel - yields 256 different levels of gray per element - varying between white and black which gives substantially increased picture quality - over a facsimile image - when reproduced for color images - to understand - digitization format used: to obtain an understanding of the principles of how color is produced how the picture tubes - used in computer monitors (on which the images are eventually displayed) operate color principles known that - human eye - sees just a single color - when a particular set of 3 primary colors -
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
are mixed and displayed simultaneously simultaneously color gamut: whole sprecrum of colors - is produced by - different prorportions of 3 primary colors red (R), green (G), and blue (B) Fig. shows - this principle principle - together together with some some examples of of colors - that can be produced Fig. a: mixing techique - used is called additive color mixing additive color mixing: called as, since - black is produced when all three primary colors are zero particulary useful - for producing a color image - on a black surface, as is the case in display appication subtractive color mixing: complementary to additive color mixing produces - similar to additive color mixing range of colors Fig. b shows - as - in i n subractive color mixing white is produced, when the 3 chosen primary colors cyan (C), Magneta (M), and Yellow (Y) - are all zero - these t hese colors are particularly useful - for producing a color i mage on a white surface - as in, printing apications color television (with 3 primary colors R, G, and B) picture tubes: same principle is used In most computer monitors: in general, since, those used with personal computers use the same picture tubes - as are use din television sets for comatibility - with the computer monitors on which digital pictures are normally viewed: digitization process used yields a clor image - which can be directly displayed - on the screen of either a television set or a computer monitor Fig. shows - general g eneral principles - associated with the process Raster-scan principles Raster-scan: picture tube - used in most television sets - operate using raster-scan involves - raster - a finely-focussed electron beam raster - scan over the complete screen progressive scan:each complete scan - comprises a number of discrete horizontal lines - first of which starts at the top left corner of the screen - and, the last of which w hich ends at thee bottom right corner - at this point, the beam is defelected back aain to the top left corner c orner - scanning operation repeats in the same way type of the scanning - is progressive scannig Fig. shows - progressive p rogressive scanning
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Frame: each compete set of horizontal scan line made up of N individual - scan lines N is either 525 (North and South America and most of Asia) or 625 (Europe and number of other countries) Inside of the display screen of the picture tube: coated with - a lightsensitive phosphor - which, emitts light - when energized by the electron beam brightness: amount of light emitted determined by - the power in the electron beam - at that instant During: each horizontal(line) and vertical(frame) vertical(frame) retrace period - electron beam is turned off To create - an image on the screen - level of power in the beam is changed as each line is scanned black-and-white picture tubes: in - a single electron beam - is used with a white-sensitive phosphor color tubes - use three separate - closely located beams, and a 2-D matrix of pixels Each pixel - comprises - set of 3 related clor-sensitive clor-sensitive phosphors - one each for R,G, and B signals Phosphor triad: set of 3 phosphors - associated with each pixel Fig. shows - typical arrangement of the triads - on each scan line In theory - each pixel represents - an idealized rectangular area - which is independent of its neighboring pixels Spot: shape of each pixel, in practical cases - which merges with its neighbours typical size - 0.025 inches (0.635 mm) when viewed from a sufficient distance - a continuous color image is seen Television picture tubes: designed to display moving images persistence of light/color: produced by the phosphor - is designed to decay very quickly - so, continuous refresh of thee screen is needed for moving image: light signals associated with each frame - change to reflect the motion - that has taken place during the time - required to scan the preceding frame for a static/still image:same image:same set of light sisgnals are used for each frame Frame refresh rate: must be high enough - to ensure the eye is not aware the display is continuously being refreshed Flicker - led by a low refresh rate caused by - the previous image - fading from the eye retina before the following image is displayed to avoid - a refresh rate of at least 50 times/s is required Frame refresh rate: determined by - frequency of the mains electricity supply - which is
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
either 60Hz in North and South America and most of Asia and 50 Hz in Europe and a number of other countries current picture tubes - current - operate in analog mode - i.e., amplitude of each of 3 color signals is continuously vrying - as each line is scanned Digital television - in - digitized pictures stored - within the computer memory - color signals are in the digital form - comprise a string of pixels with a fixed number of pixels per scan line To display the stored image - pixels that make up each line - are a re read from memory in time-synchronis time-synchronism m with the scanning process - and, converted into a continuously varying analog fom by means of DAC Video RAM: separate block of memory - used to store the pixel image Area of computer memory that holds the sting of pixels that make up the image - the pixel image must be accessed continuously - as each line is scanned graphics program: needs to write the pixel images into video RAM - whenever, either selected pixels or the total image changes Fig. shows - the architecture of various steps - involved used to create - high-level version of the image interactively - using, either the keyboard or a mouse Display controller (frame/display/refresh (frame/display/refresh buffer): part of the program interprets sequences of display commands converts them into - displayed objects - by writing w riting appropriate pixel values into the video RAM Video controller: hardware subsystem - that reads the pixel values stored in the video RAM - in time-synchronism time-synchronism with the scanning process converts - for each set of pixel values - into equivalen set of R, G, and B - analog signals for output to the display Pixel depth pixel depth: number of bits/pixel determines the range of different colors - that can be produced Ex.: 12 bits - 4 bits per primary color - yielding 4096 different colors and 24 bits - 8 bits per primary color - yielding in excess of 16 million colors Eye cannont - distinguish - such a range of colors - so, in some instances a selected
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
subset of this range of colors been used procedure: 1. selected colors in the subset - are then stored in the table 2. CLUT (Color Look-Up Table):each pixel - value is used - as an address to a location within the table (color look-up table, CLUT) - which, contain the corresponding 3 color values Ex.: if each pixel - is 8 bits and the CLUT contains 24 bit entries, then, CLUT had 24 bit entries - which will provide - a subset of 256 (28) different colors - selected from the palette of 16 million (224) colors Advantage: amount of memory required to store an image can be reduced significantly
Aspect ratio: ratio of the screen width to screen height depends on: 1. number of pixels/ scanned line 2. number of lines/frame l ines/frame current television tubes': aspect ratio - is 4/3 of older tubes (on which the PC monitors are based) is 16/9 of the wide-screen television tubes Standards for color television 1. US: Standard for color television: defined by NTSC (National Television Standards Committee) NTSC: uses 525 scan lines/frame some lines carry information and some lines c arry control all lines - are not displayed on the screen 2. Europe: 3 color standards exists a. PAL: of UK b. CCIR: of Germany c. SECAM: of France PAL,CCIR,SECAM: uses 625 scan lines some lines carry information and some lines c arry control all lines - are not displayed on the screen Number of visible lines/frame = vertical resolution in terms of pixels 480: for NTSC monitor 576: with the othe 3 standards Lattice structure - Fig. shwos - diagarammatic form of square lattice structure To produces, square picture - for: To avoid distortion on the screen - with 4/3 aspect ratio: it is necessary - for displaying a square of (N X N) pixels p ixels to have 640 pixels (480 * 4/3) per line, with an NTSC monitor
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
to have 768 pixels (576 * 4/3) per line, with a European monitor associated factors for screen resolutions - in a common computer monitor: 1. television pictue tubes 2. amount of memory required - to store the corresponding image (as in Table. ) Memory requirements: to store a single digital image - can be high - vary between 307.2 kbytes for an image: displayed on a VGA screen with 8bits/pixel through to approximately 2.36Mbytes for a SVGA (Super VGA) screen - with 24 bits/pixel computer monitors of expensive computes, since, not based on television picture tubes: 4/3 aspect ratio for need not to constrained Ex.: 1280 X 1024 X 24 - may have the refresh rate - as high as 75 frames/s for sharp image
Digital cameras and scanners Fig. shows - typical arrangement - used to capture and store a digital image produced - by a scanner or a digital camera - either, a still-image camera or a video camera It is assumed - captured image is transferred - to the computer directly as it is produced For digital cameras: set of digitized images - can be stored within thee camera itself then, downloaded into the computer - at a later time Image sensor: image capture - for in - camera/scanner camera/scanner - solid-state device photosites: 2-D grid of light-sensitive cells type of image sensor - used in digital cameras stores the level of intensity of the light that falls on it - when w hen the camera shutter is activated CCD (Charge-Coupled Device): widely used image sensor comprises - of an array of photosites - on its surface operates - by conerting the level of light intenssity - that falls on each photosite into - an equivalent electrical charge level of charge (light intensity) - stored at each photosite position - is read out then, it is converted into - a digital value into an ADC Scanners: CCD of the technique only used - except image sensor - comprises, a single s ingle line of photosites photosites - exposed in time-sequence - with the scanning operation
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
each row of stored - charges are read out and digitized - before the next scan occurs For color images: color associated - with each photosite so, the pixel position - obtained in 3 methods: 1. Surface of each photosite: coated with either R, G, or B filter - so, that its charge is determined only by the level of R, G, and B lights that falls on it coatings - are arranged on the 3 X 3 grid structure - as in Fig. then, color associated with each photosite/pixel: determined by amount of output of photosite - R, G, and B - together with each of its 8 immediate neighbors levels of two other colors - each pixel - are estimated by - interpolation procedure - of 9 values uses: used with most consumer-friendl c onsumer-friendly y cameras 2. involves - use of 3 separate exposures - of a s ingle image sensor - R, G, and B filters from color - associated with each pixel postion - is then, determined by the charge obtained with each of 3 filters - R, G, and an d B cannot be used with video camears - since, 3 separate exposures - ae required for each image used - primarily with high-resolution high-resolution still-image cameras - in locations such as photographic studios - where, cameras can be a ttached to a tripod 3. uses - 3 separate image sensors - one with all photosites coated - with 3 filters - R, G, and B single exposure - is used - with the incoming light - split into 3 beams each of which - exposes a separate image sensor used - in professional-qu professional-quality-high-resoluti ality-high-resolution on still and moving image camers - since, they are more costly owing to - use of the 3 separate sensors and associated signal processing circuits Each image/frame: once, captured andd stored on t he image sensor(s) charge stored at each photosite - location is read and digitized CCD: set of charges - on the matrix of photosites - are read a single row at a time
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
readout register: register for which - first set of charges on the first row of photosites are transferred each of the photosites - in a row is coupled to the - corresponding photosites - in the two adjoining rows each row is transferred - to the readout register set of charges on each of the other rows in the matrix - move down to the next row of photosite positions readout register in - once - charge on each photosite position, is shifted out ADC from amplification and digitizaion will process above - repeates - until, set of charges on - all rows have been read out and digitized d igitized Low-resolution image (640 X 480 pixels) and pixel depth of 24 bits - 8 bits each for R, G, and B: amount of memory - required to store - each image is 921600 bytes output of this - directly to computer - bit-map can be loaded - straight into the frame buffer ready to be displayed if required to store - within the camera: multiple images sof this sixe need to be stored - prior to them - being output to a computer set of images - are stored in an IC I C memory - either on in of: 1. removal card 2.fixed within camera if 1. - card is removed simply inserted into - PCMCIA slot of the computer if 2. - contents of the memory - are a re downloaded - to the computer - by means of cable link once - within the computer - software can be used to insert - digital d igital image(s) into a document sent it by e-mail - and so on or photo-editing software - can be used - to manipulate a stored image - Ex.: to change its size or colors number of file formats: are used - to store - s ets of images TIFF/EP: most popular version of tagged image i mage file format many different types of image data - to be stored - in the image i mage file including data (date, time, and other camera settings) associated with each image Audio
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Two types of audio signals: 1. speech signal: used in a variety of interpersosnal interpersosnal applications including: telephony video telephony music-quality audio - as used in appliciations suc as CD-on-demand and broadcast television Audio: can be produced either naturally - by means of microphone generates - a time-varying analog signal audio signal encoder - from analog signals converted to digital form to store signals in the memory of the computer and to transmit them over a digital network by means of electronically using some form of synthesizer audio - is created in the digital form can be readily stored in the computer memory output of all digitized - audio signals, the stream of digitized values must be converted back again into its analog form - using audio signal decoder BW of a typical speech signal: from 50Hz to 10kHz music signal:15Hz to 20kHz sampling rate: used for 2 signals - must be in excess of their Nyquist rate - which is 20ksps (2 X 10kHz) - for speech 40ksps (2 X 20kHz) - for music Number of bits/sample - must be chosen - such that, quantization noise generated by the sampling process - is at an acceptable level relative to the minimum signal level for speech: assumption: linear (equal) quantization intervals: tests have shown - that - it dictates - use of minimum of 12 bits/sample and for music 16 bits Applications: involving music stereoscopic (stereo) sound - since, utilized - it resluts in bit rate double that of a monaural (mono) signal In practice - both thee sampling rate used and number of bits/sample < these values for speech: ex.: BW of the network used in many interpersonal applications < BW of source signal so, lower sampling rate - with fewer bits/sample is needed
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
for music: sampling rate is lowered - to reduce - the amount of memory - required to store the particular passage of music
PCM speech Interpersonal applications applications - involving speech - use - PSTN - for communication purpose PSTN: operated with analog signal throughput - source speech signal being transmitted and switched (routed) unchanged in its original analog form Progressively - older analog transmission circuits - were replaced by digital circuits it was carried out - over a number of years - and because, of the need to interwork between eariler analog and newerr digital equipments - during the trnasition period - design of digital equipment - was based on - operating parameters of the earlier analog network BW of the speech circuit - in this network - was limited to 200Hz through to 3.4kHz Nyquist rate is 6.8kHz - poor quality of the bandlimiting filters used meant that a sampling rate of 8kHz was required to avoid aliasing To minimize the resulting bit rate: 7 bits/sample - were selected for use - in North America and Japan - with bit rate of 56kbps 8 bits/sample - in Europe (both including a sign bit) - with bit rate of 64kbps Modern systems: moved to usig 8 bits/sample in each case - much improved performance - than that of by 7bits/sample PCM (Pulse Code Modulation): digitization procedure international standard relating to this: ITU-T Recommendation G.711 Fig. shows - circuits that make up - a PCM encoder and decoder Fig. shows - compressor c ompressor (encoder) and expander (decoder) circuits quantization operation: determines role of these circuits Linear quantization intervals: quantization intervals are equal irrespective of the magnitude of the input i nput signal - same level of quantization noise is produced
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Effect: noise level is the same for - both low amplitude (quite) signals and high amplitude (loud) signals Ear: more sensitive to noise - on quite signals than it is on loud l oud signals To reduce the effect of quantization noise: with 8 bits/sample - in a PCM system quantization intervals are made non-linear (unequal) with narrower intervals used for smaller amplitude signals - than, for larger signals these can be achieved by compressor and at the destination, the reverse operation by - the expander circuits - overall operation is known as - companding Fig. shows - input/output i nput/output relationship relationship - of both circuits characteristic curve for compression - compression characteristics characteristics and characteristic curve for expansion - expansion characteristics Prior to - input signal - being sampled and converted into a digital form by ADC - it is passed through compression circuit - which effectively compresses the amplitude of the input signal Level of compression and hence, quantization intervals - increases - as the amplitude of the input signal increases resulting compressed - signal is passed - through ADC ADC - performs - linear quantization of the compressed signal at the destination: each recieved codeword is first fed into a linear DAC Analog output - from the DAC - is then, passed to the expander circuit which perform the reverse operation of the compressor circuit Modern systems: perform compression and expansion operations digitally same principles applying For histroical reason: 2 different compression-expansion characteristics in use: 1. mu-law: used in North America and Japan 2. A-law: used in Europe and some other countries Fig. - as from, it is necessary - to carry out - conversion operation - when communicating between the two systems In bot mu-law, and A-law case: use of companding gives - a perceived level of performance with 8 bits - that is, comparable with the performance obtained with 12 bits and uniform quantization intervals CD-quality audio
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Discs: used in CD players and CD-ROMs - are digital storage devices - for stereophonic music and more general multimedia information streams CD-DA (CD-digital audio): standard associated with these devices Music: has an audible BW of from 15Hz through 20kHz - so, minimum sampling rate is 40ksps In the standard: actual rate used > this rate - to 1. allow for imperfections in the bandlimiting filter used 2. so that - resulting bit rate is then, compatible with one of the higher transmission channel bit rates - available a vailable with public networks Sampling rate - used - one of is 44.1ksps 4 4.1ksps - means, signal is sampled at 23 microsecond intervals High number of samples - can be used - since, BW of a recording channel - on a CD is large standard - defines - 16 bits/sample - tests have shown to be the minimum required with music to avoid the effect of quantization noise for this number of bits: linear quantization - can be used - which yields 65536 equal quantization intervals Recording of stereophonic total bit rate required Hence, bit rate/channel = 705.6 kbps and Total bit rate = 2
music requires: 2 separate channels - so, is double that for mono sampling rate X bits/sample = 44.1 X 103 X 16 = X 705.6 = 1.411 Mbps
CD-ROMs: also uses same bit rate - which, are widely used - for distribution of multimedia titles to reduce the access delay - multiples of this rate are used - within a compuer Not feasible - to interactively access a 30s portion of a multimedia title over a 64kbps channel with 1.5Mbps channel - the time is - high for i nteractive purposes
Synthesized audio Once digitized - any form of audio - can be stored within a computer amount of memory - required - to store a digitized audio waveform - can be very large evan for relatively short passsages synthesized audio - is often - hence, used in multimedia applications since, the amount of memory required to be between two and three orders of maginitude < that required to store the equivalent digitized waveform versions Much easier - to edit - syntesized audio - and mix, several passages together
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Fig. shows - components - that make up - audio syntesizer Syntehsizer components: 1. computer, with various application programs - takes input commands from the keyboard outputs commands to sound generator 2. keyboard, based on that of a piano - provides p rovides input commands to the computer 3. set of sound generators - produces the corresponding sound waveform - of - output of computer via DACs - to drive the speakers
pressig a key on the keyboard of a synthesizer - has similar effect - of pressing a key on the keyboard of a computer - in asmuch as - for each key that is pressed a different codeword - known as a message with a synthesizer key board is generated and read by the computer program Messages indicate - such things - as which key on the keyboard has been pressed and the pressure applied control panel - contains - a rnge of different switches and sliders - that collectively allow the user to indicate to the computer program - additional information such as the volume of the generated output and selected sound effects - to be associated with each key Seconday storage interface: allows - sequence of messages - including those associated with the control panel - relating to a particular piece of audio to be saved on secondar storage - such as a floppy disk There are programs - to allow - the use to t o edit - a previously entered passage and, if required, to mix the synthesizer - then, ensures that the resulting r esulting integrated sequence of messages - are synchronized and output to the sound generators to create - the merged passage other possible possible - input input instruments instruments other than (piano) (piano) keyboard keyboard - electric guitar type will be - which generate messages -similar to those produced by the keyboard To discriminate between - different possible sources - a standardised set of messages - have been defined for both input and for output - to the corresponding set of sound generators - defined in the standard known as - MIDI (Music Interface Digital Interface) MIDI - define the format of the standardized set of messages -
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
used by a synthesizer by the connectors, cables, and by electrical signals - used to connect - any type of device to the synthesizer format of message: cosists of a - status byte - which defines - particular event - that has caused the message to be generated, followed - number of data types which collectively - define a set of parameters - assciated with the event Ex.: event for is - key being pressed on the board and typical parameters would then be the identity of the key pressure applied, and so on there can be a variety of instrument types - used for input and output necessary to identify - type of instrument - that generated the event - so that, when a corresponding message is output to the sound generators - appropriate type of sound is produced so, different types of devices have - many codes associated with them Ex.: piano - has a code of 0 and vioin - has a code of40 some codes - have been assigned for specific special effects - such as, sound froma cannon and applause from an audience
Passage of audio - produced from the synthesizer - consists of - a very compact sequence of messages - each comprising of bytes - which can be played by sequencer program directly - heard by the composer - or, saved in a file on a floppy disk Typically, in many interactive applications: involving - multitimed pages comprising text and a passage of music, a synthesizer is first used to - create the pasage of music - which, is then saved in the file Author of pages - links the file contents - to the text at the point - where music to be played Sound card: needed in the client computer to interpret the sequence of messages and to generate the appropriate sounds - since, the music is in the eform of a sequence of MIDI messages Sound generators: use either 1. FM synthesis techniques or 2. wavelet synthesis - by using samples of sound produced by sreal instruments Video
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
M u l t i m e d i a Co m m u n i c at at i o n s ( 1 0 E C8 C8 4 1 )
U n i t 2 : M u l t i m e d i a I n f o r m a tit i o n R e p r e se se n ta ta tit i o n
Applications, Networks, Protocols and Standards, Fred Bibliography: Multimedia Communications: Applications, Halsall, Pearson Education, Asia, Second Indian Indian reprint 2002.
Ram Ram esh esh S
Asst. Asst. Prof.(EC Prof.(ECE E Dept .), Bengalur Bengalur u
ram isuniverse@gm isuniverse@gm ail. ail.com
cell: cell: +91 94 49 85 191 3
