Cosc 4/5730 Android Text to Speech And Speech To Text.

Slides:



Advertisements
Similar presentations
Google Android Introduction to Mobile Computing. Android is part of the build a better phone process Open Handset Alliance produces Android Comprises.
Advertisements

CE881: Mobile and Social Application Programming Simon M. Lucas Menus and Dialogs.
Cosc 5/4730 Input Keyboard, touch, and Accelerometer.
Cosc 5/4730 Game Design. A short game design primer. A game or animation is built on an animation loop. – Instance variables of “objects” are updated.
Cosc 4/5730 Android and Blackberry Near Field Communications (NFC)
Android 02: Activities David Meredith
Manifest File, Intents, and Multiple Activities. Manifest File.
Lets Talk 9+ Emulator e-Tech for Tots CS590 - Ashok Sahu.
Cosc 5/4730 Game Design. A short game design primer. A game or animation is built on an animation loop. – Instance variables of “objects” are updated.
Spik v1.0 Voice Commands Execution in a Windows Environment Dekel Abelson Eliran Dahan Instructor: Ari Todtfeld.
Google Android as a mobile development platform T Internet Technologies for Mobile Computing Olli Mäkinen.
Cosc 4755 Phone programming: GUI Concepts & Threads.
Cosc 5/4730 Android SMS. A note first Depending on the API level, an import changes because of a deprecated API 3 uses – import android.telephony.gsm.SmsManager;
CS371m - Mobile Computing Audio.
Cosc 4730 Android TabActivity and ListView. TabActivity A TabActivity allows for multiple “tabs”. – Each Tab is it’s own activity and the “root” activity.
Android Development (Basics)
CS378 - Mobile Computing Speech to Text, Text to Speech, Telephony.
PROG Mobile Java Application Development PROG Mobile Java Application Development Event Handling Creating Menus.
 Understanding an activity  Starting an activity  Passing information between activities  Understanding intents  Understanding the activity lifecycle.
CS5103 Software Engineering Lecture 08 Android Development II.
Favorite Twitter® Searches App Android How to Program © by Pearson Education, Inc. All Rights Reserved.
Speech Recognition ECE5526 Wilson Burgos. Outline Introduction Objective Existing Solutions Implementation Test and Result Conclusion.
Cosc 5/4730 Introduction: Threads, Android Activities, and MVC.
11.10 Human Computer Interface www. ICT-Teacher.com.
Iteration. Adding CDs to Vic Stack In many of the programs you write, you would like to have a CD on the stack before the program runs. To do this, you.
Cosc 5/4730 Broadcast Receiver. Broadcast receiver A broadcast receiver (short receiver) – is an Android component which allows you to register for system.
DUE Hello World on the Android Platform.
1 Announcements Homework #2 due Feb 7 at 1:30pm Submit the entire Eclipse project in Blackboard Please fill out the when2meets when your Project Manager.
LiveCycle Data Services Introduction Part 2. Part 2? This is the second in our series on LiveCycle Data Services. If you missed our first presentation,
© Copyright by Deitel & Associates, Inc. and Pearson Education Inc. All Rights Reserved. 1 Tutorial 27 - Phone Book Application Introducing Multimedia.
Overview of Android Application Development
CS378 - Mobile Computing Intents. Allow us to use applications and components that are part of Android System – start activities – start services – deliver.
Cosc 5/4730 Android Communications Intents, callbacks, and setters.
Linking Activities using Intents How to navigate between Android Activities 1Linking Activities using Intents.
Cosc 4730 Android Fragments. Fragments You can think of a fragment as a modular section of an activity, which has its own lifecycle, receives its own.
Mobile Application Development Options most popular platforms/ application software: -BREW-Android -Symbian-JME -Palm OS-Windows Mobile -iPhone-Linux -BlackBerry.
First Venture into the Android World Chapter 1 Part 2.
Activity Android Club Agenda Hello Android application Application components Activity StartActivity.
Understand the difference between applets and applications Identify meaningful applet resources for academic subjects Create and demonstrate a basic Java.
Fall 2002CS 150: Intro. to Computing1 Streams and File I/O (That is, Input/Output) OR How you read data from files and write data to files.
Select (drop-down list) Inputs. Insert/Form/List Menu.
Applications with Multiple Activities. Most applications will have more than one activity. The main activity is started when the application is started.
Speech Recognition Yonglei Tao. Voice-Activated GPS.
Introducing Intents Intents Bind application components and navigate between them Transform device into collection of interconnected systems Creating a.
Intents 1 CS440. Intents  Message passing mechanism  Most common uses:  starting an Activity (open an , contact, etc.)  starting an Activity.
Martin T. Press.  Main Method and Class Name  Printing To Screen  Scanner.
SourceAnatomy1 Java Source Anatomy Barb Ericson Georgia Institute of Technology July 2008.
Lecture 2: Android Concepts
Cosc 5/4735 YouTube API. YouTube The YouTube Android Player API enables you to incorporate video playback functionality into your Android applications.
Cosc 4735 LocationAware API. Previous on … Before we looked at GPS location. – d-gpslocation.pptx
Cosc 5/4735 Voice Actions Voice Interactions (API 23+)
© Copyright by Deitel & Associates, Inc. and Pearson Education Inc. All Rights Reserved. 1 Outline 28.1 Java Speech API 28.2 Downloading and.
Technische Universität München Services, IPC and RPC Gökhan Yilmaz, Benedikt Brück.
Intents and Broadcast Receivers Dr. David Janzen Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution.
Java for android Development Nasrullah Khan. Using instanceof in Android Development the classes such as Button, TextView, and CheckBox, which represent.
Cosc 4735 Nougat API 24+ additions.
Google VR (gvr) CardBoard and DayDream With OpenGL
Intents and Broadcast Receivers
Lecture 2: Android Concepts
several communicating screens
Linking Activities using Intents
Android Speech Recognition
Android Introduction Camera.
CIS 470 Mobile App Development
Android Sensor Programming
Activities and Intents
Objects First with Java
Lecture 2: Android Concepts
Cosc 4730 An Introduction.
Objects First with Java
Presentation transcript:

Cosc 4/5730 Android Text to Speech And Speech To Text

TEXT TO SPEECH Android

Text to Speech In Android 1.6+ there is a native Text-to- speech built into the Android OS. – In 2.2, Menu-> Voice input & output-> Text-to- speech settings – You can use the “Listen to an example” to see how it works.

How it works. The Text-to-Speech (TTS) uses a the Pico engine – It sends the “speech” to the audio output. There is only one TTS engine and it is share across all the activities on the device. – Other activities maybe using it – The user may have overridden the settings in the preferences as well.

Using the TTS First we need to check if the TTS engine is available. – We can do this with a Intent to with ACTION_CHECK_TTS_DATA Using startActivityForResult, we then find out if the TTS engine is working and avialable.

Android.speech.tts To use the TTS we need get access to it using the constructor TextToSpeech(Context context, TextToSpeech.OnInitListener listener) – The constructor for the TextToSpeech class. mTts = mTts = new TextToSpeech(this, this); – First this, use the context of our application – Likely US-EN – Second this, the listener. … Activity implements OnInitListener public void onInit(int status)

OnInitListener onInit(int status) – Called to signal the completion of the TextToSpeech engine initialization. – Status is either TextToSpeech.SUCCESS – You can use it. – or TextToSpeech.ERROR – Failure, you can’t use it.

Using the TTS To have it speak words – speak(String text, int queueMode, HashMap params) To stop, call stop() Shutdown() to release everything

Example mTts.speak(“Test”, TextToSpeech.QUEUE_ADD, null); – You should hear the word test spoken.

Other methods. You can change the pitch and speech rate with – setPitch(float pitch) – setSpeechRate(float speechRate) To find out if “it” is still speaking – Boolean isSpeaking() To have the speech written to a file – synthesizeToFile(String text, HashMap params, String filename) Remember permission for writing to the file system.

Note In the OnPause() method – You should put at least a stop() call – You app has lost focus

Example code The txt2spk.zip example – Simple text box and button. Type in the words you want to speak and then press play. – If you are running the example on a phone For fun, use the voice input (microphone on the keyboard) for the input and then have it read it back to you.

Blackberry Text to speech is supported to through the JSR 113: Java Speech API 2.0 speech. – But RIM limits Text-To-Speech API to accessibility API – For app’s for blind of visually impaired.

SPEECH TO TEXT Android

Speech To Text Like Text to speech, we are going to call on another Google's voice recognition software. – Android.speech package – The simple version uses an intent and there is a dialog box for the users to know when to speech. RecognizerIntent – With a onActivityResult A Note speech recognition doesn’t work in the emulators.

Simple version code First get the recognize intent Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH); Specify the calling package to identify your application (this one is generic for any class you use) intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, getClass().getPackage().getName()); Display an hint to the user in the dialog box intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "Say Something!"); Given an hint to the recognizer about what the user is going to say intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM); Specify how many results you want to receive. The results will be sorted where the first result is the one with higher confidence. In this case max of 5 results intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS, 5); Now launch the activity for a result startActivityForResult(intent, VOICE_RECOGNITION_REQUEST_CODE);

Simple version code (2) When the recognition is done, results are returned to onActivityResult protected void onActivityResult(int requestCode, int resultCode, Intent data) { if (requestCode == VOICE_RECOGNITION_REQUEST_CODE && resultCode == RESULT_OK) { Fill the list view with the strings the recognizer thought it could have heard, there should be at most 5, based on the call ArrayList matches = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS); Now you deal with results in matches array. } lastly send other results to the super since we are not dealing with them. super.onActivityResult(requestCode, resultCode, data); }

SpeechRecognizer class A second version is more complex, but also removes the dialog box Which many people want implement their own or just not one. You will need record_audio permission – Get the speech recognizer and a RecognitionListener – This still uses an intent as well. Remember the recognition is done by Google's “cloud’.

SpeechRecognizer First get the recognizer sr = SpeechRecognizer.createSpeechRecognizer(this); Set your listener. sr.setRecognitionListener(new Recognitionlistener()); – Listener is on the next slide.

RecognitionListener create a Recognitionlistener and implement the following methods – void onBeginningOfSpeech() The user has started to speak. – void onBufferReceived(byte[] buffer) More sound has been received. – void onEndOfSpeech() Called after the user stops speaking. – void onError(int error) A network or recognition error occurred. Error codes are covered herehere – void onEvent(int eventType, Bundle params) Reserved for adding future events. – void onPartialResults(Bundle partialResults) Called when partial recognition results are available. – void onReadyForSpeech(Bundle params) Called when the endpointer is ready for the user to start speaking. – void onResults(Bundle results) Called when recognition results are ready. – void onRmsChanged(float rmsdB) The sound level in the audio stream has changed.

RecognitionListener (2) onResults methods – This is where you would pull out the results from the bundle – ArrayList results = results.getStringArrayList(SpeechRecognizer.RESUL TS_RECOGNITION);

Start the recognition As in the simple version we need an intent to start the recognition, but we are sending the intent through the SpeechRecognizer object, we declared in the beginning. – get the recognize intent Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH); – Specify the calling package to identify your application intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,getClass().getPackag e().getName()); – Given an hint to the recognizer about what the user is going to say intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LA NGUAGE_MODEL_FREE_FORM); – Specified the max number of results intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS,5); – Use our SpeechRecognizer to send the intent. sr.startListening(intent); The listener will now get the results.

Code Examples The txt2spk will speak text Speak2Text demo shows you more information on using other languages for voice recognition, plus will speak the results back to you. speech2txtDemo is simplified voice recognition speech2txtDemo2 is uses the RecognitionListener.

iSpeech There have SDK and API for blackberry, android, and iphone as well. – Text to speech With many voice options as well – Speech to text Limited to 100 word demo key per application launch. – License key removes the 100 word limit.

References /ApiDemos/src/com/example/android/apis/app/T extToSpeechActivity.html /speech/SpeechRecognizer.html /ApiDemos/src/com/example/android/apis/app/ VoiceRecognition.html w-can-i-use-speech-recognition-without-the- annoying-dialog-in-android-phones

Q A &