Research Issues in Adapting Computing to Small Devices fcmlau@cs.hku.hk Research Issues in Adapting Computing to Small Devices Francis C.M. Lau Department of Computer Science The University of Hong Kong APPT 2005 • Hong Kong

Popular Computing 70’s: one computer, many users 80’s: one computer, one user 90’s: many (connected) computers, many users – distributed computing Now: many computers, one user What next? Ubiquitous computing? Grid computing? Service-oriented computing? … Do we need that many computers? Do we need a “computer”? APPT 05 (27/10/05, HK)

Windows Panic Continue or quit or kill the window or details? How would my mother know!? The best computing is without the computer APPT 05 (27/10/05, HK)

Imagine … APPT 2015 APPT 2015 (Disneyland, Hong Kong) … APPT 05 (27/10/05, HK)

My Future Handheld All in one: phone, PDA, wallet, game console, remote terminal, ad-hoc networking … That means a lot of software – big and fat!? But what I really want is lean and thin and mean Lean – simple software which does only what I want Thin – that’s all the space in my pocket, or around my neck! Mean – affordable LTM APPT 05 (27/10/05, HK)

LTM Computing The (thin-) client-server paradigm, works but in limited ways Too thin: a browser-only device, a remote terminal … Future devices are more powerful than just that But how powerful is powerful? APPT 05 (27/10/05, HK)

O my god! Software complexity APPT 05 (27/10/05, HK) Ravi Subramanian, 3G Spells Deconstruction for the Wireless Industry, IEEE CommSoc SCV, 1999 Subramanian, 1999 APPT 05 (27/10/05, HK)

The Network is the Computer ™ Rich Computing My hard disk and VM can be as big as the Internet Not just contents, but software come and go on demand Pay-per-use subscription model Browser plug-in’s and Java applets – not lean enough, and some require just too much resources Software in terms of “functionalities” The Network is the Computer ™ APPT 05 (27/10/05, HK)

The Network Behind Every Device Future digicams Snap and go The network is the memory The network is the photo album (The network is NOT the power) Context aware APPT 05 (27/10/05, HK)

Edge Computing Person .. device .. middleware (proxies) .. Internet The abstract cloud moves with the client – personalized “cuddleware”, nomadic computing Internet proxies united metropolis client APPT 05 (27/10/05, HK)

The continuum: WHAT SHOULD BE DONE WHERE? GRID EDGE DEVICE The continuum: WHAT SHOULD BE DONE WHERE? APPT 05 (27/10/05, HK)

A Continuum Your device .. other devices (peers) in the vicinity .. server(s) nearby .. the grid “Neighborhood” is still important network bandwidth can’t be infinite; latency will never be 0; network will never be trouble-free What should be done where? The end-device: completely passive (HAS) to completely active (DIY) APPT 05 (27/10/05, HK)

“Hire a Server” Completely passive (remote terminal) A waste of bandwidth Responsiveness and interactivity (esp. for games) Does it make sense? MP3, camera, PDA, … Something must be done locally Adaptation APPT 05 (27/10/05, HK)

Limits You Can’t Beat Power? Screen size: foldable, rollable, wearable, projectable? Or do without the screen (audible?) Convenience and privacy; and power consumption Resolution helps a little bit Why I don’t use a PDA (I just ordered an 24” LCD; but I still look at my watch) Ergonomics is very important Scrolling is a nuisance (now) User experience! APPT 05 (27/10/05, HK)

Wearable Computers server display APPT 05 (27/10/05, HK)

Adaptation Data Content UI Code = not code A special kind of data Text, paged, non-text Pixel level and/or higher UI Higher level, structural, interactive Buttons, icons, menus, panels and panes …  content Code APPT 05 (27/10/05, HK)

Componentized Software Executable code as puzzle pieces Fast start, like contents Software download pipelining – first-come-first-execute, the most needed functionalities first APPT 05 (27/10/05, HK)

The Need for Code On Demand How many functions have you ever used? A simplified view of Microsoft Word APPT 05 (27/10/05, HK)

Remember UNIX? Small is beautiful Make each program do one thing well Choose portability over efficiency Store data in flat ASCII files Avoid captive user interfaces Make every program a filter … APPT 05 (27/10/05, HK)

UNIX Philosophy Applied Small software components implementing single functionalities Bigger functions runtime-composable from small components Download on demand, and disposable Standard formats and interfaces (eg. XML) Shared GUI APPT 05 (27/10/05, HK)

Service-Oriented Computing Can Web Services Scale Down? - for small device apps APPT 05 (27/10/05, HK)

Adapting X to Y Many issues Context awareness Psychology Aesthetics Readability, convenience, privacy … Economics Context awareness Because small devices are likely mobile The user moves around APPT 05 (27/10/05, HK)

Awareness and Adaptation An intelligent supporting architecture Aware of the come and go of mobile users Choices of functions, depending on situation: the location, the device, the user, … Fast reacting nearby proxy Proxies united, within certain locality APPT 05 (27/10/05, HK)

Summary 1: Basic Problems (I think) Worth Pursuing Edge computing → “seamless” New protocols for the edge The continuum → the network is the computer Collaborative models and mechanisms, esp. at the edge The global grid → invisible, “PC” disappearing The device Adaptation On-demand code composition The SOC approach? Content HTML UI description languages New paradigms for user interaction in small devices (input and output) APPT 05 (27/10/05, HK)

Some HKU Projects The Sparkle Project – “code adaptation” PDF document content adaptation system – “content adaptation” HTML adaptation – “UI/content adaptation” APPT 05 (27/10/05, HK)

Sparkle Application = “facet” composition Mobile code on demand Proxy to device, device to device Features Code On Demand Context Based Adaptation Proxy Caching on Code Code & State Migration Ontology Sparkle View & Universal Browser APPT 05 (27/10/05, HK)

Universal Browser APPT 05 (27/10/05, HK) Time (ms) 500 1000 1500 2000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Time (ms) Add Noise Add Shadow Find Edges Flip Horizontal Vertical Gaussian Blur Open Image Save as Jpeg Sharpen APPT 05 (27/10/05, HK)

PDF Document Adaptation APPT 05 (27/10/05, HK)

APPT 05 (27/10/05, HK)

APPT 05 (27/10/05, HK)

APPT 05 (27/10/05, HK)

Web Document Adaptation APPT 05 (27/10/05, HK)

Only the bottom level is “compressed” The original content Only the bottom level is “compressed” (c) The two lowest levels are “compressed” APPT 05 (27/10/05, HK)

Handling Segments APPT 05 (27/10/05, HK)

Summary 2: When I Grow Older I want a simpler life (others can enjoy all the complexities) One device, LTM No computer please; terminal is fine; plug and play Single data storage by hire, access anywhere Single sign-on, single environment (adaptive) APPT 05 (27/10/05, HK)

APPT 05 (27/10/05, HK)