Brief Tour of TACC CS444I Internet Services Spring 00 © Armando Fox

© 1999, Armando Fox TACC Conceptual Data Flow C $ W W W A W W W T FE User request To Internet n Front end accepts RPC-like user requests n Original data fetched from cache or Internet n User’s customization profile retrieved n Aggregators/transformers operate on data according to customization profile

© 1999, Armando Fox What TACC Servers Give You n Uniform API’s to workers, caches l Many cache nodes, one virtual cache l Name workers symbolically, not by their location l Transparent management of customization DB n Management of partial failure, scaling, load balancing, operator notification, logging n An “Internet service construction kit”

© 1999, Armando Fox TACC API’s n Worker API n Inter-worker API n Worker libraries C $ W W W A W W W T FE User request To Internet n Service layer API n Data store API

© 1999, Armando Fox TACC Worker API n Languages supported: C/C++, Perl, Java(?) l Worker receives: original data, original headers, original MIME type, user profile l Worker returns: return code, new data, new headers, new MIME type n Special return codes l Redispatch: allows construction of pipelines l Routing hints may accompany redispatch n Workers can call each other l No built-in checking for mutual recursion, etc. l “Top level” worker timeout

© 1999, Armando Fox Data Store API n Cache subsystem l Get, Put, Query, Post Form (to server) l Asynchronous fetch (for prefetch and parallel-fetch) l One logical cache, many physical caches n Customization database l User profile automatically passed to each worker l Generate profile-change HTML form l Modify profile directly n Use cache/DB for intermediate state n Future: Distributed Hash Table API support

© 1999, Armando Fox A Keyword Highlighter In TACC # Red-hilite for keywords identified by arg 9000 use TACCutils; sub DistillerMain { my($data,$headers,$type,%args) my($data,$headers,$type,%args) $pattern = $args{’s9000’}; $pattern = $args{’s9000’}; $data = &html_regsub($data, $data = &html_regsub($data, "s!($pattern)! \$1 !ig"); "s!($pattern)! \$1 !ig"); $data =~ y/a-z/A-Z/; $data =~ y/a-z/A-Z/; $hdrs = "HTTP/ OK\r\nX-Route: transend/text/html\r\n\r\n"; $hdrs = "HTTP/ OK\r\nX-Route: transend/text/html\r\n\r\n"; return($data, $hdrs, "text/html", 10); return($data, $hdrs, "text/html", 10);} < 10 lines of Perl; could also be C, C++, Java, Tcl Data, headers, and user profile info from initial request Library of commonly-used routines (HTML munging, etc.) Return type of result; could also have returned an error, or “chained” (pipeline style) to another filter What you don’t see: scalability, replication, reliability, load balancing, failure handling...

© 1999, Armando Fox Other TACC API’s n Distill() and AsyncDistill(): subroutine and coroutine to another worker n MakePrefsForm(): return HTML form which, when submitted, will allow user to change user prefs n MonitorClientSend(): send messages, status, etc. to SNS monitor n Various useful HTML parsing functions in TACCutils library

© 1999, Armando Fox TACC Applications n TranSend n Top Gun Wingman (now ProxiWeb) Class projects: n Top Gun Mediaboard n RTSP Archive Server Front End n Group Annotation n Anonymous Rewebber n Support for location-dependent services

© 1999, Armando Fox TranSend Web Accelerator n Lossy on-the-fly Web image compression, extensively parameterized (per user, device, etc.) n Service layer: HTTP Proxy l HTTP/HTML used for error reporting (as with browsers) l Most errors result in bypass direct to client n Worker calls l Single worker selected, based on MIME type l Bypass If no worker desired delivered

© 1999, Armando Fox TranSend, cont. n Cache support l Fetch originals through cache l Optionally store distilled versions in cache Moral: a simple transformation-only service (trivial TACC example)

© 1999, Armando Fox Example: Info Aggregators n “Virtual sites” named by magic URL’s l Creation of a new “web space” l Aggregation can be on-the-fly, cached, or both n Composition with “filters” l After processing, “fall through” to TranSend by returning “redispatch” status l Get TranSend benefits for free: now you can view the virtual site on your PalmPilot too n Does this sound like Web clipping? Moral: services can be designed for composition

© 1999, Armando Fox A Meta-Search Engine In TACC n Uses existing services to create a new service n 2.5 hours to write using TACC franework Metasearch Web UI Internet

© 1999, Armando Fox Example: Group Annotation n Multivalent Documents/Digital Libraries project: Marcel Kornacker, Ray Gilstrap, Tom Phelps, Prof. Robert Wilensky n Annotate the Web, without owning it Anno. DB UI Stock Web servers Anno. Svr. Web browser MVD Editor (Java)

© 1999, Armando Fox Superiority of proxy-based approach to enabling “thin” clients successfully demonstrated. Groupware: Top Gun MediaBoard n mb comes to the Pilot n Combines MASH tools with TACC server l Enables future MASH collaboration n Inherits TACC robustness & scalability