Welcome to the Boston

A BIG thank you to the 2018 Global Sponsors!

Computer, Veronika Kolesnikova Willy Ci Developer, Rightpoint Sr. Frontend Developer, UMass Medical School Level: Introductory to Intermediate

Agenda Artificial intelligence and machine learning Virtual/Digital Assistants. What? When? How? Most popular digital assistants Development environments Principles of bot design Microsoft Cognitive Services Cortana skills creation Demo

Veronika Kolesnikova BS in Applied Computer Science and Economics MS in Information Technology 3.5 years QA Experience 5+ years development experience C#, .NET, SQL, Xamarin, .NET-based CMSes like EPiServer, Sitecore PHP, MySQL, Drupal, RoR Hobbies: dancing, travelling, aerial yoga

Willy Ci Graduated from UMass Amherst Trained by IBM, mold by few startups, end up at UMass Medical School More than 20 years of experience in front end development Microsoft Hololens, Unity, 3D printing, WebGL Hobbies: panorama photography, VR/AR applications Has a lot of ideas, never has enough time to sleep

What is virtual assistant? It can be a human, but we’re not talking about that here Several most popular assistants Millions skills and applications Can be used on any devices like: mobile (Apple and Android) Virtual assistants make work via: Text Voice By taking and/or uploading images A virtual assistant, also called AI assistant or digital assistant, is an application program that understands natural language voice commands and completes tasks for the user

1980s automated attendant and medical digital dictation software Where it came from? 1961 IBM Shoebox 1970s “Harpy” 1980s automated attendant and medical digital dictation software 1990s new feature for PCs 1994 Smart-phone IBM Simon 2011 Siri by Apple 1961 - IBM Shoebox (William C. Dersch in the Advanced Systems Development Division Laboratory) 1970s – “Harpy” (Carnegie Mellon University in Pittsburgh, Pennsylvania with support of the United States Department of Defense and its DARPA agency) The earliest virtual assistants, which applied speech recognition software were automated attendant and medical digital dictation software 1990s – became a new feature for PCs 1994 – Smartphone IBM Simon 2011 – Siri by Apple

Most popular digital assistants Siri (2011) Google Now (2012) Cortana(2014) Facebook M (2017) Alexa (2014)

Which one is better? Google Now Siri Cortana Alexa 3.1 2.9 2.3 1.7 Travel and commute related tasks Productivity tasks: calendar appointments and email Search and Office tasks Music and video Google Home, GH Mini, GH Max HomePod Invoke Amazon Echo, EchoDot, Echo Show

How to start designing bots Explore different bot use-cases and design best practices Understand bot anatomy—such as brand and personality, conversations, advanced UI controls—and their associated design patterns

Google Now development environment Google Developer Console Dialogflow Google Cloud Speech API Node.js Firebase Tools Dialogflow is an end-to-end development suite for building conversational interfaces for websites, mobile applications, popular messaging platforms, and IoT devices. You can use it to build interfaces (e.g., chatbots) that are capable of natural and rich interactions between your users and your business. It is powered by machine learning to recognize the intent and context of what a user says, allowing your conversational interface to provide highly efficient and accurate responses.

Siri development environment SiriKit Xcode Swift and Objective-C An Intents app extension receives user requests from SiriKit and turns them into app-specific actions. For example, the user might ask Siri to send a message, book a ride, or start a workout using your app. An Intents UI app extension displays branding or other customized content in the Siri or Maps interface after your Intents app extension fulfills a user request. Creation of this extension is optional.

Alexa development environment Create function in AWS Configure the skill in Amazon developer portal Use JSON code Test either on an actual device or using online emulator Or Use Alexa Skills Kit: Custom Skill, Smart Home Skill API, Video Skill API, Flash Briefing Skill API

Cortana Development environment Cortana Skills Kit Bot Service (provides .NET SDK and Node.js SDK) Bot Builder (Web App Bot, Functions Bot, Bot Channels Registration) Bot Framework Microsoft Knowledge Graph exchange Cortana Skills Dashboard Microsoft Cognitive Services (LUIS!!!) Cortana Development environment

Microsoft Cognitive services Get Key Build EASY FLEXIBLE TESTED Simple to add: just a few lines of code Integrate into the language and platform of your choice Built by experts in their field from Microsoft Research, Bing and Azure Machine Learning Get access with REST APIs Breadth of offerings helps you find the right API for your app Quality documentation, sample code, and community support

Language understanding Intelligent services(LUIS) Language Understanding (LUIS) allows your application to understand what a person wants in their own words. LUIS uses machine learning to allow developers to build applications that can receive user input in natural language and extract meaning from it. A client application that converses with the user can pass user input to a LUIS app and receive relevant, detailed information back.

Principles of bot design Minify number of steps user needs to get his\her problem solved Determine devices users will connect with bot through Make replies as short and precise as possible

Principles of bot design Add easy way to return to the start, exit and\or talk to a human or search articles online Keep bot updated using feedback or failure log

Bot failure Alexa was randomly laughing because it could mistakenly hear “Alexa, laugh”, so Amazon had to change the utterance to “Alexa, can you laugh?” and then alexa responds first before actually laughing (apparently the fix was already rolled out. Anyone noticed anything like that with your Echo devices?)

Demo find Doctor

Voice to bing speech api to luis (Language understanding Intelligent services) to bot to app

Useful links Microsoft Cognitive Services Documentation https://docs.microsoft.com/en-us/azure/cognitive-services/ https://azure.microsoft.com/en-us/services/cognitive-services/?v=17.29 Amazon developers portal https://developer.amazon.com/alexa Microsoft developers portal https://developer.microsoft.com/en-us/Cortana Follow Veronika on Twitter @breakpointv16 Follow Willy on Twitter @willy_ci Code https://github.com/willyci/botFindProfessorC