1. OTHER RESEARCH IN SIGNAL PROCESSING AND COMMUNICATIONS IN ECE Richard Stern Carnegie Mellon University (with Dave Casasent, Tsuhan Chen, Vijaya Kumar, José Moura) rms@ece.cmu.edu (412) 268-2535 January 18, 2006

2. Carnegie Mellon Slide 2ECE Communications What is Signal Processing & Communications? Solving problems involving “signals” … –Images –Sound –Other types of data –Everything together Some of the things we do: –Get rid of noise and distortion –Analyze signals for content –Organize information –Design communications networks –Facilitate human communication

3. Carnegie Mellon Slide 3ECE Communications Some Research Foci in ECE Processing, analysis, and compression of static and moving video Optical signal processing techniques Signal processing for digital data storage Speech recognition and understanding Multimedia fusion of video and audio information Architecture and protocols of telecommunications and computer networks

4. Video: Codec and Representation (Moura) Applications: 2-D Rigid Motion Analysis Video Synthesis Object 1 Motion Analysis Object 2 Shape + Texture Motion script for Object 2 + Shape + Texture Motion script for Object 1 + Video Synthesis General Concept:

5. Articulated Motion              ModelingRecognitionCapture 3-D Rigid Motion Source images of object Shape Reconstructed 3-D object Texture Source images of outdoor scene Reconstructed 3-D scene Video: Codec and Representation (2)

6. X-Ray Agricultural Inspection (Casasent) 100% Non-destructive inspection: locate infested pistachio nuts GoodInfested Results: Infested Nuts reduced to 1.6-2.5%. Only 2-1% of Good Nuts Rejected InputSegment Non-linear feature extraction Neural Net Classifier

7. SAR Imagery and Remote Sensing (Casasent) Geological ground cover classification and change detection identify/measure forested/non- forested areas, vegetation types, wetlands Synthetic Aperture Radar (SAR) EOS Ground Cover Map Hyperspectral Sensors: AVIRIS

8. Carnegie Mellon Slide 8ECE Communications Research: –Multimedia Processing and Applications »Multimedia Communication »Audio-Visual Interaction »Standards: JPEG, G.7xx, H.26x, H.32x, MPEG –Facial Image Analysis and Synthesis –Networked Collaborative Environment Text VideoAudio Multimedia Processing and Applications (Chen)

9. Before... Multimedia Content Classification, Retrieval, and Protection Audio-Visual Speech Human Computer Interface Scene Analysis/Synthesis Now... Networking Issues: Rate Control, Traffic Analysis, Error Resilience Source/Channel Coding Audio/Speech Image/Video Text Translation Natural Language Proc. Coding and Processing Analysis, Coding and Representation Speech Recognition Speech Synthesis

10. Carnegie Mellon Slide 10ECE Communications Summary Voice, video, images, and text From signals to bits to packets to information streams Understanding information, translating information, transporting information