1. Global Consumer Insights
Automating business insights through machine learning & artificial intelligence
March 6, 2018

2. General Mills

3. Purpose of this talk
1. Share our experience and maybe you can find ways to apply it in your space
2. Challenge us as practitioners to figure out how to make automation a reality in business
3. When people think about jobs in this space, don’t only think of Start-ups and tech firms

4. How do we help over 1,000 people in marketing with Data Analysis & Analytics?

5. Our original problem DATA EVERYWHERE OTHER PEOPLES DATA
SILO DATA OWNERS
Each of these prevent analytics and business intelligence to be built across data

6. So we organized in Marketing
Formed a dedicated team
ANALYTICS & DATA SCIENCE
DATA STEWARDS
DATA VISUALIZERS
Ensure quality of our data and maintain connections
Main point for training and inquiries
Develop easy way to use the data
Engineer new data sources and access
Purpose built analytics

7. And we partnered with IT
FROM TO

8. And we made an impact Availability
Enable access to data people could not access before
Documentation to educate
Connections
Data speaks to each other
Align the data to the way marketers think
New Insights
Purpose built visuals to answer common business questions
Analytics to answer advanced questions
New possibilities on what we can answer
Reduce time to prep analysis
Reduce time to answer or enable ability to answer questions

9. For example…Way to track new products
Built a visual to track New Product launches
50+ charts, 120 metrics, all important
15-20 minutes for a user to see what is going on with each new item
Data warrants watching it weekly
A marketer can have 5-8 new items at any given time

10. But our marketers are busy
Time
Priority
8am
Finish what they did not yesterday
9am
Prepping for a retailer conversation
10am
Got an urgent request from leadership
11am
Still scrambling to get an answer 12
Talk to media buying group about an ad in flight
12:55
Try to grab something to eat 1
Meet with Finance 2
Work with product development on a future flavor 3
Discuss what to do about plant restrictions 4
Finalize an answer to the urgent request from leadership 5
Run and grab the kids
9pm….
Answer all the s from the day
Jane
Marketing Manager on Nature Valley
Repeat

11. Time to think differently
How do we identify what products have something unique going on?
Quickly guide the marketer to what is unique about that product.
Increase speed to action.

12. Wanted to identify the stuff the marketer should look at
Types of Anomalies: Point: Individual data point can be considered anomalous with respect to the rest of the data. Collective / Contextual: instance is anomalous in a specific context (but not otherwise), then it is termed as a contextual (conditional) outlier. If group of points, then termed collective outlier. Global: Anomalous data points are defined by measuring the global deviation of a given data point with respect to its neighbors, globally. Local: Anomalous data points are defined by measuring the local deviation of a given data point with respect to its neighbors. Business Transactions: Anomalous data points related to some business transaction often measured via a KPI ($’s or Volume) over time (think time series). Reference Data: Anomalous data points related to the metadata or reference data about some entity.

13. How about machine leaning?
Approach
Example Algorithms
Considerations
Rule-Based
zscore, frequency, hard-coded rules
Must be able to define abnormal
Must define rules
Does not adapt to change
Supervised
Classification, Logistic Regression
Need data labeled as abnormal
No need to define rules
Adapts to change
Unsupervised
Clustering (KNN, DBSCAN), Isolation Forest, Deep Learning Autoencoder
No need to have labeled data

14. Algorithms worked
We had 2 products that were meant to be limited edition:
They were similar but 2 different flavors.
PRODUCT A
PRODUCT B

15. What happened with these products?
PRODUCT A
PRODUCT B
GOOD, not AMAZING Let Limited Edition run and discontinue
A CLEAR WINNER
Let’s keep it

16. Our algorithms alerted us Consumers were reacting differently
Right away we could tell a difference in sentiment between Product B and Product A
Product B
“I hope it will become a permanent product”
Product A
“Is there any way I can exchange it”

17. So our marketers should have said let’s keep Product B very early on
Start having discussions with retailers about keeping it
Alert manufacturing to keep sourcing this
Decide on a strategy to keep this as a permanent product
But we had a problem

18. We deployed the algorithms in our visualizations
1. A business user logs into the visual and sees what launches are anomalous
2. They see what dates where there is something worth their attention
3. They can then see what business metrics are being flagged for their attention

19. But…it requires a business user to investigate
They have to proactively go out and look
You are competing with all their other priorities
And this is not simple enough

20. Our big aha… We are competing for the business user time
We have to bring automation of insights to where they already are
As much as it pains me to say, an alert on what they need to know is what my user base needs

21. Key Takeaways It’s a journey
Treat business user experience like a consumer experience
We are never done

22. Global Consumer Insights
Thank You