SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones -Hong LU, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury and Andrew T.

Slides:

Advertisements

Similar presentations

Darwin Phones: the Evolution of Sensing and Inference on Mobile Phones Emiliano Miluzzo *, Cory T. Cornelius *, Ashwin Ramaswamy *, Tanzeem Choudhury *,

Advertisements

Outline Activity recognition applications

Voiceprint System Development Design, implement, test unique voiceprint biometric system Research Day Presentation, May 3 rd 2013 Rahul Raj (Team Lead),

Entropy and Dynamism Criteria for Voice Quality Classification Applications Authors: Peter D. Kukharchik, Igor E. Kheidorov, Hanna M. Lukashevich, Denis.

Ear-Phone: An End-to-End Participatory Urban Noise Mapping System -Rajib Kumar Rana, Chun Tung Chou, Salil S. Kanhere, Nirupama Bulusu, Wen Hu -School.

Acoustic Model Adaptation Based On Pronunciation Variability Analysis For Non-Native Speech Recognition Yoo Rhee Oh, Jae Sam Yoon, and Hong Kook Kim Dept.

THE JIGSAW CONTINUOUS SENSING ENGINE FOR MOBILE PHONE APPLICATIONS Hong Lu,† Jun Yang,! Zhigang Liu,! Nicholas D. Lane,† Tanzeem Choudhury,† Andrew T.

Pitch Prediction From MFCC Vectors for Speech Reconstruction Xu shao and Ben Milner School of Computing Sciences, University of East Anglia, UK Presented.

Urban Sensing Systems: Opportunistic or Participatory?

Dual-domain Hierarchical Classification of Phonetic Time Series Hossein Hamooni, Abdullah Mueen University of New Mexico Department of Computer Science.

D u k e S y s t e m s Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application Emiliano Miluzzo†, Nicholas.

DARWIN PHONES: THE EVOLUTION OF SENSING AND INFERENCE ON MOBILE PHONES PRESENTED BY: BRANDON OCHS Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy,

NEUROPHONE: BRAIN- MOBILE PHONE INTERFACE USING A WIRELESS EEG HEADSET Andrew T. Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu, Matthew K. Mukerjee!,

A Survey of Mobile Phone Sensing

Feature Vector Selection and Use With Hidden Markov Models to Identify Frequency-Modulated Bioacoustic Signals Amidst Noise T. Scott Brandes IEEE Transactions.

Toward Semantic Indexing and Retrieval Using Hierarchical Audio Models Wei-Ta Chu, Wen-Huang Cheng, Jane Yung-Jen Hsu and Ja-LingWu Multimedia Systems,

Chapter 1: Introduction to Pattern Recognition

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

SENSING MEETS MOBILE SOCIAL NETWORKS: THE DESIGN, IMPLEMENTATION AND EVALUATION OF THE CENCEME APPLICATION Emiliano Miluzzo†, Nicholas D. Lane†, Kristóf.

A Practical Approach to Recognizing Physical Activities Jonathan Lester, Tanzeem Choudhury, and Gaetano Borriello In Proceedings of the Fourth International.

A Practical Approach to Recognizing Physical Activities Jonathan Lester Tanzeem Choudhury Gaetano Borriello.

AdvAIR Supervised by Prof. Michael R. Lyu Prepared by Alex Fok, Shirley Ng 2002 Fall An Advanced Audio Information Retrieval System.

Computer Science Department A Speech / Music Discriminator using RMS and Zero-crossings Costas Panagiotakis and George Tziritas Department of Computer.

1 Music Classification Using SVM Ming-jen Wang Chia-Jiu Wang.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Authors: Anastasis Kounoudes, Anixi Antonakoudi, Vasilis Kekatos

A Survey of Mobile Phone Sensing Michael Ruffing CS 495.

Representing Acoustic Information

Audio Processing for Ubiquitous Computing Uichin Lee KAIST KSE.

SoundSense: Scalable Sound Sensing for People-Centric Application on Mobile Phones Hon Lu, Wei Pan, Nocholas D. lane, Tanzeem Choudhury and Andrew T. Campbell.

Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application Emiliano Miluzzo†, Nicholas D. Lane†, Kristóf.

Kinect Player Gender Recognition from Speech Analysis

“SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones” Authors: Hong Lu, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury and.

Design, Implementation and Evaluation of CenceMe Application COSC7388 – Advanced Distributed Computing Presentation By Sushil Joshi.

SoundSense by Andrius Andrijauskas. Introduction  Today’s mobile phones come with various embedded sensors such as GPS, WiFi, compass, etc.  Arguably,

SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones -Hong LU, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury and Andrew T.

Dan Rosenbaum Nir Muchtar Yoav Yosipovich Faculty member : Prof. Daniel LehmannIndustry Representative : Music Genome.

MUMT611: Music Information Acquisition, Preservation, and Retrieval Presentation on Timbre Similarity Alexandre Savard March 2006.

Introduction to Smart-Phone Sensing 1. Reference Shamelessly lifted from the following paper : A Survey of Mobile Phone Sensing ◦ By Nicholas D. Lane,

Nicholas D. Lane, Hong Lu, Shane B. Eisenman, and Andrew T. Campbell Presenter: Pete Clements Cooperative Techniques Supporting Sensor- based People-centric.

Feature Vector Selection and Use With Hidden Markov Models to Identify Frequency-Modulated Bioacoustic Signals Amidst Noise T. Scott Brandes IEEE Transactions.

The Second Life of a Sensor: Integrating Real-World Experience in Virtual Worlds using Mobile Phones Mirco Musolesi, Emiliano Miluzzo, Nicholas D. Lane,

Network Community Behavior to Infer Human Activities.

© 2005, it - instituto de telecomunicações. Todos os direitos reservados. Arlindo Veiga 1,2 Sara Cadeias 1 Carla Lopes 1,2 Fernando Perdigão 1,2 1 Instituto.

Performance Comparison of Speaker and Emotion Recognition

MSc Project Musical Instrument Identification System MIIS Xiang LI ee05m216 Supervisor: Mark Plumbley.

BY KALP SHAH Sentence Recognizer. Sphinx4 Sphinx4 is the best and versatile recognition system. Sphinx4 is a speech recognition system which is written.

Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application Emiliano Miluzzo†, Nicholas D. Lane†, Kristóf.

PHYSICS CLASS ACTIVITY. CLASS ACTIVITY: TUNING FORK FREQUENCY.

Chapter 7 Speech Recognition Framework  7.1 The main form and application of speech recognition  7.2 The main factors of speech recognition  7.3 The.

1 Electrical and Computer Engineering Binghamton University, State University of New York Electrical and Computer Engineering Binghamton University, State.

Pocket, Bag, Hand, etc. - Automatically Detecting Phone Context through Discovery Emiliano Miluzzoy, Michela Papandreax, Nicholas D. Laney, Hong Luy, Andrew.

ADAPTIVE BABY MONITORING SYSTEM Team 56 Michael Qiu, Luis Ramirez, Yueyang Lin ECE 445 Senior Design May 3, 2016.

A Survey of Mobile Phone Sensing Nicholas D. Lane Emiliano Miluzzo Hong Lu Daniel Peebles Tanzeem Choudhury - Assistant Professor Andrew T. Campbell -

Recognition of bumblebee species by their buzzing sound

University of Rochester

Traffic State Detection Using Acoustics

ARTIFICIAL NEURAL NETWORKS

Robust Data Hiding for MCLT Based Acoustic Data Transmission

Pattern Recognition Sergios Theodoridis Konstantinos Koutroumbas

Speech Processing Speech Recognition

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Department of Electrical Engineering

AUDIO SURVEILLANCE SYSTEMS: SUSPICIOUS SOUND RECOGNITION

John H.L. Hansen & Taufiq Al Babba Hasan

Endpoint Detection ( 端點偵測)

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Presenter: Shih-Hsiang(士翔)

Measuring the Similarity of Rhythmic Patterns

Music Signal Processing

Presentation transcript:

SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones -Hong LU, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury and Andrew T. Campbell -Department of Computer Science Dartmouth College, 2009, number of pages 14 Presented By: Rene Chacon

Current use of Sound Sensing in mobile phones. SoundSense, a scalable framework for modeling sounds events on mobile phones. Implemented on Apple iPhone, supervised and unsupervised learning techniques for classification. Sound Sense (Abstract) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Sounds used to dictate a person’s surrounding environment. Audio stream degrades more gracefully (phone in pocket) and as such is still useful. Sound Sense (Introduction) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Audio Sensing Scalability Problem -Voice, Music, Ambient sounds -hierarchical classification architecture “Goal of SoundSense is to support robust sound processing and classifications under different phone context conditions.” Sound Sense (Section 2) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Privacy Issues and Resource Limitations -SoundSense processes all audio data on the phone. -Consider the accuracy and cost tradeoff. -Microphone: 4kHz Sound Sense (Section 2) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Sound Sense (Section 3) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Preprocessing Framing (64ms) Frame Admission Control rms threshold & entropy threshold Sound Sense (Section 4) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Coarse Category Classification [voice, music, ambient] -Zero Crossing Rate, human voice has high variation while music and ambient sounds have low. -Spectral Flux (SF) measures the changes in the shape of the spectrum. -Spectral Rolloff (SRF), Spectral Centroid (SC), and more. Sound Sense (Section 4) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Multi Level Classification – Use decision tree classifier -Utilizing algorithms, Gaussian Mixture Models (GMMs), Hidden Markov Models (HMMs) and more. Sound Sense (Section 4) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Finer Intra-Category Classification Unsupervised Ambient Sound Learning -utilize Mel Frequency Cepstral Coefficient (MFCC) in depth details of frequency bands sensitive to human ear. -Sound Rank and Bin Replacement Strategy -User Annotation of New Sounds Sound Sense (Section 4) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Sound Sense (Section 5) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones PCM data: 8kHz, 16 bit Frame: 512 samples ; lack of acoustic event: long duty cycle state Either an intra-category classifier or unsupervised adaptive classifier

Determining buffer sizes -Ex. Output of the decision tree classifier before input to Markov model recognizer. -Ex. MFCC frame length Sound Sense (Section 5) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

SoundSense on iPhone. <60% of the CPU utilized roughly 5MB compare to 30MB Coarse Category Classifier Finer Intra-Category Classifier Sound Sense (Section 6) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Unsupervised Adaptive Ambient Sound Learning -Future Work, incorporate other types of contextual information such as GPS to further clarify sounds. Sound Sense (Section 6) 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Audio Daily Diary based on Opportunistic Sensing 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Audio Daily Diary based on Opportunistic Sensing 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones 7dhc4

Discover events with music-like sounds (parties, concerts, music in a room). Camera is used to take picture or location and associate. Music Detector based on Participatory Sensing 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Sound Processing and classification is performed real- time as oppose to off-line / external. SoundSense works solely on the mobile phone. Tackles Classification issues with Ambient sounds. Significance of Work 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Paper: SoundSense: Scalable Sound Sensing for People-Centric Application on Mobile Phones Link: obisys09.pdf obisys09.pdf MetroSense Project that explores SoundSense References 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones

Questions 1 Paper: SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones