Real-time sentiment built on the Azure IoT stack 5/8/2018 3:08 AM P4044 Real-time sentiment built on the Azure IoT stack Joel Narwold Developer – Microsoft Pulse © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Microsoft Pulse – Real-time Voting Platform The Technical Challenge: Handling Large-Volume Scale Walkthrough Pulse by the Numbers The Pipeline Testing the Architecture Lessons Learned/Things to Consider

Joel Narwold Full-stack developer on the Microsoft Pulse team. Worked with the pulse team for the last two years and supported multiple large scale events (including the State of the Union 2017, Presidential Debates, etc)

Microsoft Pulse Real-time Voting Platform

Microsoft Pulse Microsoft Pulse is the leading self-service audience engagement technology for real-time feedback. Capture audience sentiment in real-time and display feedback from participants through live voting and polls.

Microsoft Pulse The Technical Challenge

Handling Large Scale Volume Accept data from hundreds of thousands of concurrent users Tally the data by demographics Report results back to users in real-time

Microsoft Pulse Walkthrough

Producer Screen The producer screen is a self service tool for creating pulses, surveys, polls and quizzes. The screen is targeted to educators, marketers and broadcasters. From the dashboard producers can gather valuable insights regarding the number of viewers pulsing, how many votes have come in. This data can be broken out by demographic and sliced and diced with our external API.

Microsoft Pulse The Numbers

Live Events CNN – 2016 Iowa Caucus CNN – 2016 State of the Union 20,750 unique active 29,473 unique 315,000 total votes 124,327 unique active 167,210 unique 5,916,923 total votes

Microsoft Pulse The Pipeline

The Gateway

Gateway Pros Always open event hub connection Better load dispersal with azure traffic manager Cons Additional complexity 16-50 ms delay

Event Hub

Event Hub Pros High data ingestion Multiple consumer groups Cons Checkpointing Painful scaling No local debugger

Aggregation Worker

Aggregation Worker Pros Low latency Cons Monolithic design No local debugger Full deployment required Less stable

Data Storage Paths

SQL Pros Relational database Cons Relational database

Redis Pros High performance Pub-sub mechanics Cons Difficult debugging

SignalR

SignalR Pros Best available option in .NET Easy to setup Easy to consume Cons Small packet size Fewer messages sent Receiving order inconsistent State handling

Storage

Pulse Architecture

Microsoft Pulse Testing the Architecture

Headless Testers // The Plan

Headless Testing Architecture

Microsoft Pulse Final Thoughts

Resources Event Hub (http://bit.ly/1PjaZQr) Azure Redis Cache (http://bit.ly/1mR0RCv) SignalR (http://bit.ly/2oqs4xI) NodeJs (http://bit.ly/1LjG5AH) PhantomJs (http://bit.ly/1kzC14S)

Handling Large Scale Volume Plan ahead…but not too much Test your architecture in the real world Log early, log often

Thanks. Joel Narwold GMR Digital Solutions Developer jnarwold@gmrmarketing.com