2014 Mobile UI Performance: Dealing with lots of data Tina Wen – iOS Dropbox 10/10/2014 #GHC

What’s my background?  BS & MS from MIT, 2009  Developed for Mac desktop, iOS, and web professionally  Joined Dropbox 3 years ago, largely focused on iOS development

2014 What’s your background?  Have built mobile app?  Have heard of Dropbox?  Have heard of Carousel?

2014 Quick Intro to Carousel

2014 Carousel Core Principle Your data is there, when you want it be it a picture from 10 years ago, yesterday, or one your mom sent you a second ago from the east coast!

2014 In order to load lots of data fast… Local Database UI Dropbox Persistent but slow

2014 In order to load lots of data fast… Local Database UI Dropbox Today, we’re going to focus here…

2014 …and here conversation post

2014 Two basic requirements 1. Minimize latency necessary to show photos to the user 2. Smooth animation when needed as data loads / changes

2014 Two basic requirements 1. Load data as quickly as possible from disk 2. Keep UI smooth as data loads / changes Simple, right? not with THOUSANDS of posts

2014 Main thread + drawing model MAIN THREAD (UI THREAD) Draw on screen Take input Smooth scrolling == 60 fps ≈ 16 ms/cycle Can’t load from local database on main thread

2014 Breaking it down Load from Database Propagate to UI Process data

2014 Breaking it down Load from Database Propagate to UI Process data

2014 Platform-standard solutions Android: Loaders and Cursors Problem: Cursor interface performs poorly with more than 2MB of data iOS: Core Data Problem: Often slow CUSTOM

2014 Load Benchmark Loading & processing 1,000 post metadata (content without thumbnails) from SQLite at once takes 1.55 seconds – NO GOOD!

2014 Load in Chunks Textbook solution: load in chucks  Need cursor based API for chunk loading  Can have UI load with a single cursor in chucks… …but then UI has to wait for results  Instead, use: in background keep track of cursor, load data, then notify UI

2014 Load in Chunks For each chunk: BACKGROUND 1. Load [some] data from database 2. Process for UI MAIN THREAD 3. Propagate to UI

2014 Smart Chunking Load ONLY what user sees first For us, that’s the most recent 10 posts Switch to larger chunks after Increase chunks to 1,000 posts after initial load

2014 Breaking it down Load from Database Propagate to UI Process data

2014 Database UI Data in Database is rarely exactly the same as what the UI needs Can’t process on main thread postspost photos

2014 DB UI Data in DB is rarely exactly the same as what the UI needs Obviously can’t process on main thread postspost items Main Thread = BLOCKING

2014 In memory data source  Store data source for UI in memory  UI queries only data source synchronously at render time – super fast

2014 Data Source posts 0A 1B 2C 3E 4F 5G …… Simplified Data Source Local Database UI Dropbox

2014 Data Source UI OBJECTIVE-C - (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath; JAVA public abstract View getView (int position, View convertView, ViewGroup parent)

2014 But how is Data Source created?  Introducing the Accumulator (keeps in memory data source up to date as it receives updates from database)

2014 Accumulator + Data Source Individual Update Batch Update Accumulator UI posts 0A 1B 2C 3E 4F 5G …… Simplified Data Source

2014 How to deal with many sources? Simplify! API for Accumulator is just Add, Remove, and Commit One-Off: Add  Commit OR Remove  Commit Chuck load: Many Adds  Commit

Database of Posts Listener Building posts data source BACKGROUND THREAD UI (MAIN THREAD) Add Post A Append posts with Post A Add Post B Append posts with Post B Add Post A Replace old Post A with new Post A Remove Post B Remove Post B from posts Commit Sort posts array Generate posts array data source Publish to UI Set new posts array Call reloadData: … Post A Post B posts

2014 Accumulator API C++ void add_post_metadata(const DbxPostInfo &post_info); void remove_post_metadata(const string & post_id) void commit();

2014 Processing data for UI  Most apps do the processing natively  We use C++ in a library called libdropbox for cross-platform support

2014 Why C++? Why do things 2 times if I only have to do them 1 time?

2014 Introducing Djinni JNI Accumulator + Data Source UI More info at project page: Objective-C UI Libdropbox (C++) Objective-C++ JAVA

2014 Breaking it down Load from Database Propagate to UI Process data

2014 Propagating data to UI Have updated data source for UI Now just need UI to reload…right?

2014 Sure, but no animation!

2014 Sure, but no animation! Animations require context

2014 Calculating changes in data sources… ??? Old data source of Posts 0A 1B 2C 3D 4E 5F …… New data source of Posts 0A 1C 2E 3F 4D 5- ……

2014 This sounds familiar… Calculating changes in data sources…

2014 We already found deltas for Accumulator! Calculating changes in data sources…

Database of Posts ListenerOps Queue Add Post A Add Post B Add Post A Remove Post B TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB Commit … posts BACKGROUND THREAD

Ops Queue Database of Posts Listener Add Post A Add Post B Add Post A Remove Post B Commit BACKGROUND THREAD TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB … Post A Update UI Main Thread posts

Ops Queue Database of Posts Listener Add Post A Add Post B Add Post A Remove Post B Commit BACKGROUND THREAD TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB … Post A Post B Update UI Main Thread posts

Ops Queue Database of Posts Listener Add Post A Add Post B Add Post A Remove Post B Commit BACKGROUND THREAD TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB … Post A Post B Update UI Main Thread posts

Ops Queue Database of Posts Listener Add Post A Add Post B Add Post A Remove Post B Commit BACKGROUND THREAD TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB … Post A Post B Update UI Main Thread posts

Ops Queue Database of Posts Listener Add Post A Add Post B Add Post A Remove Post B Commit BACKGROUND THREAD TypeInsertion PostA TypeInsertion PostB TypeReload PostA TypeDeletion PostB … Post A Update UI Main Thread posts

2014 The smooth animation

2014 Conclusions 1. Chunk load from disk for lots of data 2. Pre-process data in background to avoid blocking UI 3. Using deltas in datasets to incrementally update UI simplifies APIs and makes animating changes easy 4. For multi-platform apps, writing data model code in C++ can save a lot of time

2014 Got Feedback? Rate and Review the session using the GHC Mobile App To download visit