0x1A Great Papers in Computer Security Vitaly Shmatikov CS 380S

Slides:

Advertisements

Similar presentations

Robust Speech recognition V. Barreaud LORIA. Mismatch Between Training and Testing n mismatch influences scores n causes of mismatch u Speech Variation.

Advertisements

Building an ASR using HTK CS4706

ONLINE ARABIC HANDWRITING RECOGNITION By George Kour Supervised by Dr. Raid Saabne.

A Novel Approach for Recognizing Auditory Events & Scenes Ashish Kapoor.

An Overview of Machine Learning

Introduction The aim the project is to analyse non real time EEG (Electroencephalogram) signal using different mathematical models in Matlab to predict.

Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.

December 2006 Cairo University Faculty of Computers and Information HMM Based Speech Synthesis Presented by Ossama Abdel-Hamid Mohamed.

Feature Vector Selection and Use With Hidden Markov Models to Identify Frequency-Modulated Bioacoustic Signals Amidst Noise T. Scott Brandes IEEE Transactions.

Natural Language Processing - Speech Processing -

Speech Recognition. What makes speech recognition hard?

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

COMP 4060 Natural Language Processing Speech Processing.

1 Security problems of your keyboard –Authentication based on key strokes –Compromising emanations consist of electrical, mechanical, or acoustical –Supply.

Optimal Adaptation for Statistical Classifiers Xiao Li.

Why is ASR Hard? Natural speech is continuous

A PRESENTATION BY SHAMALEE DESHPANDE

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Introduction to machine learning

Statistical Natural Language Processing. What is NLP?  Natural Language Processing (NLP), or Computational Linguistics, is concerned with theoretical.

CS Machine Learning. What is Machine Learning? Adapt to / learn from data  To optimize a performance function Can be used to:  Extract knowledge.

CS-424 Gregory Dudek Today’s Lecture Neural networks –Backprop example Clustering & classification: case study –Sound classification: the tapper Recurrent.

EE513 Audio Signals and Systems Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky.

Eng. Shady Yehia El-Mashad

CEN 592 PATTERN RECOGNITION Spring Term CEN 592 PATTERN RECOGNITION Spring Term DEPARTMENT of INFORMATION TECHNOLOGIES Assoc. Prof.

Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University

Isolated-Word Speech Recognition Using Hidden Markov Models

1 7-Speech Recognition (Cont’d) HMM Calculating Approaches Neural Components Three Basic HMM Problems Viterbi Algorithm State Duration Modeling Training.

1 Robust HMM classification schemes for speaker recognition using integral decode Marie Roch Florida International University.

Artificial Intelligence 2004 Speech & Natural Language Processing Natural Language Processing written text as input sentences (well-formed) Speech.

This week: overview on pattern recognition (related to machine learning)

Midterm Review Spoken Language Processing Prof. Andrew Rosenberg.

7-Speech Recognition Speech Recognition Concepts

Side Channel Attacks through Acoustic Emanations

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

A brief overview of Speech Recognition and Spoken Language Processing Advanced NLP Guest Lecture August 31 Andrew Rosenberg.

Minimum Mean Squared Error Time Series Classification Using an Echo State Network Prediction Model Mark Skowronski and John Harris Computational Neuro-Engineering.

User Authentication Using Keystroke Dynamics Jeff Hieb & Kunal Pharas ECE 614 Spring 2005 University of Louisville.

Lecture 10: 8/6/1435 Machine Learning Lecturer/ Kawther Abas 363CS – Artificial Intelligence.

Csc Lecture 7 Recognizing speech. Geoffrey Hinton.

LML Speech Recognition Speech Recognition Introduction I E.M. Bakker.

Jun-Won Suh Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Speaker Verification System.

22CS 338: Graphical User Interfaces. Dario Salvucci, Drexel University. Lecture 10: Advanced Input.

Artificial Intelligence 2004 Speech & Natural Language Processing Natural Language Processing written text as input sentences (well-formed) Speech.

NOISE DETECTION AND CLASSIFICATION IN SPEECH SIGNALS WITH BOOSTING Nobuyuki Miyake, Tetsuya Takiguchi and Yasuo Ariki Department of Computer and System.

Side-Channel Attacks Vitaly Shmatikov CS Acoustic Information in Typing uDifferent keystrokes make different sounds Different locations on the supporting.

Feature Vector Selection and Use With Hidden Markov Models to Identify Frequency-Modulated Bioacoustic Signals Amidst Noise T. Scott Brandes IEEE Transactions.

Overview ► Recall ► What are sound features? ► Feature detection and extraction ► Features in Sphinx III.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Reestimation Equations Continuous Distributions.

Audio processing methods on marine mammal vocalizations Xanadu Halkias Laboratory for the Recognition and Organization of Speech and Audio

Speech Recognition with CMU Sphinx Srikar Nadipally Hareesh Lingareddy.

Probabilistic reasoning over time Ch. 15, 17. Probabilistic reasoning over time So far, we’ve mostly dealt with episodic environments –Exceptions: games.

Performance Comparison of Speaker and Emotion Recognition

ICASSP 2007 Robustness Techniques Survey Presenter: Shih-Hsiang Lin.

Predicting Voice Elicited Emotions

Message Source Linguistic Channel Articulatory Channel Acoustic Channel Observable: MessageWordsSounds Features Bayesian formulation for speech recognition:

1 7-Speech Recognition Speech Recognition Concepts Speech Recognition Approaches Recognition Theories Bayse Rule Simple Language Model P(A|W) Network Types.

Speech Recognition through Neural Networks By Mohammad Usman Afzal Mohammad Waseem.

BIOMETRICS VOICE RECOGNITION. Meaning Bios : LifeMetron : Measure Bios : LifeMetron : Measure Biometrics are used to identify the input sample when compared.

Speech Recognition

Deep Learning Amin Sobhani.

ARTIFICIAL NEURAL NETWORKS

HUMAN LANGUAGE TECHNOLOGY: From Bits to Blogs

Dynamic Authentication of Typing Patterns

EE513 Audio Signals and Systems

Introduction to Radial Basis Function Networks

Speech Recognition: Acoustic Waves

Neural networks (3) Regularization Autoencoder

Artificial Intelligence 2004 Speech & Natural Language Processing

Listen Attend and Spell – a brief introduction

Presentation transcript:

0x1A Great Papers in Computer Security Vitaly Shmatikov CS 380S

L. Zhuang, F. Zhou, D. Tygar Keyboard Acoustic Emanations Revisited (CCS 2005)

Acoustic Information in Typing uDifferent keystrokes make different sounds Different locations on the supporting plate Each key is slightly different uFrequency information in the sound of the typed key can be used to learn which key it is Observed by Asonov and Agrawal (2004) slide 3

“Key” Observation uBuild acoustic model for keyboard and typist uExploit the fact that typed text is non-random (for example, English) Limited number of words Limited letter sequences (spelling) Limited word sequences (grammar) uThis requires a language model Statistical learning theory Natural language processing slide 4

Sound of a Keystroke uEach keystroke is represented as a vector of Cepstrum features Fourier transform of the decibel spectrum Standard technique from speech processing slide 5 [Zhuang, Zhou, Tygar]

Bi-Grams of Characters uGroup keystrokes into N clusters uFind the best mapping from cluster labels to characters uUnsupervised learning: exploit the fact that some 2-character combinations are more common Example: “th” vs. “tj” Hidden Markov Models (HMMs) slide “t”“h”“e” [Zhuang, Zhou, Tygar]

Add Spelling and Grammar uSpelling correction uSimple statistical model of English grammar Tri-grams of words uUse HMMs again to model slide 7 [Zhuang, Zhou, Tygar]

Recovered Text _____ = errors in recovery = errors corrected by grammar slide 8 Before spelling and grammar correction After spelling and grammar correction [Zhuang, Zhou, Tygar]

Feedback-based Training uRecovered characters + language correction provide feedback for more rounds of training uOutput: keystroke classifier Language-independent Can be used to recognize random sequence of keys –For example, passwords Representation of keystroke classifier –Neural networks, linear classification, Gaussian mixtures slide 9 [Zhuang, Zhou, Tygar]

Overview Initial training Unsupervised Learning Language Model Correction Sample Collector Classifier Builder keystroke classifier recovered keystrokes Feature Extraction wave signal (recorded sound) Subsequent recognition Feature Extraction wave signal Keystroke Classifier Language Model Correction (optional) recovered keystrokes [Zhuang, Zhou, Tygar] slide 10

Experiment: Single Keyboard uLogitech Elite Duo wireless keyboard u4 data sets recorded in two settings: quiet and noisy Consecutive keystrokes are clearly separable uAutomatically extract keystroke positions in the signal with some manual error correction [Zhuang, Zhou, Tygar] slide 11

Results for a Single Keyboard slide 12 Recording lengthNumber of wordsNumber of keys Set 1~12 min~400~2500 Set 2~27 min~1000~5500 Set 3~22 min~800~4200 Set 4~24 min~700~4300 Set 1 (%)Set 2 (%)Set 3 (%)Set 4 (%) WordCharWordCharWordCharWordChar Initial Final [Zhuang, Zhou, Tygar] uDatasets uInitial and final recognition rate

Experiment: Multiple Keyboards uKeyboard 1: Dell QuietKey PS/2 In use for about 6 months uKeyboard 2: Dell QuietKey PS/2 In use for more than 5 years uKeyboard 3: Dell Wireless Keyboard New slide 13 [Zhuang, Zhou, Tygar]

Results for Multiple Keyboards u12-minute recording with app characters Keyboard 1 (%)Keyboard 2 (%)Keyboard 3 (%) WordCharWordCharWordChar Initial Final [Zhuang, Zhou, Tygar] slide 14

Defenses uPhysical security uTwo-factor authentication uMasking noise uKeyboards with uniform sound (?) slide 15