1. Welcome to the
Boston

2. A BIG thank you to the 2018 Global Sponsors!

3. Computer, Veronika Kolesnikova Willy Ci Developer, Rightpoint
Sr. Frontend Developer, UMass Medical School
Level: Introductory to Intermediate

4. Agenda Artificial intelligence and machine learning
Virtual/Digital Assistants. What? When? How?
Most popular digital assistants
Development environments
Principles of bot design
Microsoft Cognitive Services
Cortana skills creation
Demo

5. Veronika Kolesnikova BS in Applied Computer Science and Economics
MS in Information Technology
3.5 years QA Experience
5+ years development experience
C#, .NET, SQL, Xamarin, .NET-based CMSes like EPiServer, Sitecore
PHP, MySQL, Drupal, RoR
Hobbies: dancing, travelling, aerial yoga

6. Willy Ci Graduated from UMass Amherst
Trained by IBM, mold by few startups, end up at UMass Medical School
More than 20 years of experience in front end development
Microsoft Hololens, Unity, 3D printing, WebGL
Hobbies: panorama photography, VR/AR applications
Has a lot of ideas, never has enough time to sleep

7. What is virtual assistant?
It can be a human, but we’re not talking about that here
Several most popular assistants
Millions skills and applications
Can be used on any devices like: mobile (Apple and Android)
Virtual assistants make work via:
Text
Voice
By taking and/or uploading images
A virtual assistant, also called AI assistant or digital assistant, is an application program that understands natural language voice commands and completes tasks for the user

8. 1980s automated attendant and medical digital dictation software
Where it came from?
1961 IBM Shoebox
1970s “Harpy”
1980s automated attendant and medical digital dictation software
1990s new feature for PCs
1994 Smart-phone IBM Simon
2011 Siri by Apple
IBM Shoebox (William C. Dersch in the Advanced Systems Development Division Laboratory)
1970s – “Harpy” (Carnegie Mellon University in Pittsburgh, Pennsylvania with support of the United States Department of Defense and its DARPA agency)
The earliest virtual assistants, which applied speech recognition software were automated attendant and medical digital dictation software
1990s – became a new feature for PCs
1994 – Smartphone IBM Simon
2011 – Siri by Apple

9. Most popular digital assistants
Siri (2011)
Google Now (2012)
Cortana(2014)
Facebook M (2017)
Alexa (2014)

10. Which one is better? Google Now Siri Cortana Alexa 3.1 2.9 2.3 1.7
Travel and commute related tasks
Productivity tasks: calendar appointments and
Search and Office tasks
Music and video
Google Home, GH Mini, GH Max
HomePod
Invoke
Amazon Echo, EchoDot, Echo Show

11. How to start designing bots
Explore different bot use-cases and design best practices
Understand bot anatomy—such as brand and personality, conversations, advanced UI controls—and their associated design patterns

12. Google Now development environment
Google Developer Console
Dialogflow
Google Cloud Speech API
Node.js
Firebase Tools
Dialogflow is an end-to-end development suite for building conversational interfaces for websites, mobile applications, popular messaging platforms, and IoT devices. You can use it to build interfaces (e.g., chatbots) that are capable of natural and rich interactions between your users and your business. It is powered by machine learning to recognize the intent and context of what a user says, allowing your conversational interface to provide highly efficient and accurate responses.

13. Siri development environment
SiriKit Xcode Swift and Objective-C
An Intents app extension receives user requests from SiriKit and turns them into app-specific actions. For example, the user might ask Siri to send a message, book a ride, or start a workout using your app.
An Intents UI app extension displays branding or other customized content in the Siri or Maps interface after your Intents app extension fulfills a user request. Creation of this extension is optional.

14. Alexa development environment
Create function in AWS
Configure the skill in Amazon developer portal
Use JSON code
Test either on an actual device or using online emulator Or
Use Alexa Skills Kit: Custom Skill, Smart Home Skill API, Video Skill API, Flash Briefing Skill API

15. Cortana Development environment
Cortana Skills Kit
Bot Service (provides .NET SDK and Node.js SDK)
Bot Builder (Web App Bot, Functions Bot, Bot Channels Registration)
Bot Framework
Microsoft Knowledge Graph exchange
Cortana Skills Dashboard
Microsoft Cognitive Services (LUIS!!!)
Cortana Development environment

16. Microsoft Cognitive services
Get Key
Build
EASY
FLEXIBLE
TESTED
Simple to add: just a few lines of code
Integrate into the language and platform of your choice
Built by experts in their field from Microsoft Research, Bing and Azure Machine Learning
Get access with REST APIs
Breadth of offerings helps you find the right API for your app
Quality documentation, sample code, and community support

17. Language understanding Intelligent services(LUIS)
Language Understanding (LUIS) allows your application to understand what a person wants in their own words. LUIS uses machine learning to allow developers to build applications that can receive user input in natural language and extract meaning from it. A client application that converses with the user can pass user input to a LUIS app and receive relevant, detailed information back.

18. Principles of bot design
Minify number of steps user needs to get his\her problem solved
Determine devices users will connect with bot through
Make replies as short and precise as possible

19. Principles of bot design
Add easy way to return to the start, exit and\or talk to a human or search articles online
Keep bot updated using feedback or failure log

20. Bot failure
Alexa was randomly laughing because it could mistakenly hear “Alexa, laugh”, so Amazon had to change the utterance to “Alexa, can you laugh?” and then alexa responds first before actually laughing (apparently the fix was already rolled out. Anyone noticed anything like that with your Echo devices?)

21. Demo find Doctor

22. Voice to bing speech api to luis (Language understanding Intelligent services) to bot to app

24. Useful links Microsoft Cognitive Services Documentation
Amazon developers portal
Microsoft developers portal
Follow Veronika on
Follow Willy on
Code