1. Microsoft Cortana for App Developers
Microsoft Virtual Academy
Header
Mastering Cortana Development, Part 1
Microsoft Cortana for App Developers
Scott J. Peterson
Produced by

2. Cortana Development
Cortana, as an extensible, programmable, actionable personal digital assistant, is available on Windows, Android, Xbox, and the web.

3. The Agenda Setting Up the Solution
Understanding Cortana Flow and Response
Understanding Command Definitions
Working with App Services
Structuring Your Command Logic
Handling Navigate Commands
Handling App Service Commands
Putting It All Together
Tips & Tricks

4. Setting Up the Solution
Create a new solution with the following:
UWP App Project
UWP Class Library
UWP Runtime Component Library
Add one or more App Services (optional)
Add Package Manifest declarations
Add Voice Command Set XML file
Install Command Set (at runtime)

5. Setting Up the Solution

6. Cortana Logic Flow Checks to see if App exists by App Name
Instantiates App
Checks to see if App Command Set contains verbiage
Instantiates App Voice Command Service
Checks to see if App exists by Command Prefix
Instantiates App
Processes App Service logic
Processes OOTB Commands

7. Cortana Response Flow Checks the Command Definition for a match
Passes Recognition Result to Navigate or Voice Command Service
Interrogate the Recognition Result and alternatively interrogate Semantic Interpretation information
Handles as if nothing happened
Process information
Show immediate results
Show list of options

8. Command Definition Elements
AppName OR CommandPrefix The word or phrase used to “activate” your App or App Service
Command The primary group of actions and activities for interaction
Example A textual response Cortana will show to a user as an “example” of what to say
ListenFor Words or phrases for Cortana to “listen for” when evaluating actions
Feedback A message for Cortana to show a user during initial processing
Navigate OR VoiceCommandService The type of “hand off” action for the Command, if the rules are met
PhraseList An optional list of possible choices to enhance Cortana interaction
Item A specific entry in a PhraseList
PhraseTopic An optional “category” used to improve the relevance of the recognition results

9. Command Definitions

10. Cortana Response Flow Checks the Command Definition for a match
Passes Recognition Result to Navigate or Voice Command Service
Interrogate the Recognition Result and alternatively interrogate Semantic Interpretation information
Handles as if nothing happened
Process information
Show immediate results
Show list of options

11. Handling Navigate Commands
The Navigate element of a Command is the simplest form of responding to a request from Cortana. When a rule is met, the App gets activated, and (optionally) send to a page with arguments.
Is always connected to showing an actual UI element (typically a Page) in your App
Can pass parameters in Target, but would need to be parsed
Needs to be evaluated in the OnActivated() method in App.xaml.cs

12. Handling App Service Commands
The VoiceCommandService element of a Command is the more robust and powerful form of responding to a request from Cortana. When a rule is met, a pre- defined App Service gets activated.
Is typically connected to an IBackgroundTask bundled with your App
Interrogates the AppServiceTriggerDetails to determine information being sent from Cortana
Can perform any task or run any method not requiring a UI context
Can “hand off” information back to the App UI

13. Programmatic Responses to Cortana
All interaction with Cortana is handled through a reference to the VoiceCommandServiceConnection she passes to you in an App Service.
//METHODS TO REQUEST MORE INFORMATION OR ACTIONS FROM CORTANA [VoiceCommandServiceConnection].RequestAppLaunchAsync(); [VoiceCommandServiceConnection].RequestConfirmationAsync(); [VoiceCommandServiceConnection].RequestDisambiguationAsync(); //METHODS TO RESPOND TO CORTANA TO SEND BASIC INFORMATION [VoiceCommandServiceConnection].ReportProgressAsync(); [VoiceCommandServiceConnection].ReportFailureAsync(); [VoiceCommandServiceConnection].ReportSuccessAsync();

14. Handling Navigate & App Service Commands

15. Putting It All Together
App (UI)
App Class Library (Shared Logic)
App Service
Voice Commands
Cortana Interactions

16. Mastering Cortana Development Microsoft Cortana for App Developers
Scott J. Peterson, MCSD, MCPSB, MCT
Senior Enterprise Architect & Developer

17. Tips & Tricks
Put as much logic and classes/models in a “Core” or “Common” project as possible.
Stay away from using images when sending Cortana a list of tiles to disambiguate.
Get used to slightly different behavior on ARM devices.
Some things just can’t be done or work well in a background task. Send the information to the foreground and process it there.
You may have issue on Insider Builds when calling “InstallCommandDefinitionsFromStorageFileAsync”.