© 2003 IPD, Prof. Lockemann Universität Karlsruhe (TH) BTW03 Information System Architectures: From Art to Science Peter C. Lockemann

© 2003 IPD, Prof. Lockemann 2 BTW03 Agenda requirements specification systems architecture S/H system architectural design = programming-in-the- very-large n Requirements = services n Well... what is a service? n Our design hypothesis n Classical DBMS as our testbed n... and newer DBMS n From analysis to construction n Layers and components n The hypothesis under stress: components n Tier-architectures n Conclusion

© 2003 IPD, Prof. Lockemann 3 BTW03 information process 1 resource manager 1 work unit 1 Shared resources and services resource manager 2resource manager 3resource manager 4 work unit 2work unit 3work unit 4 work unit 3work unit 2work unit 1work unit 5 information process 2 service client service provider business process

© 2003 IPD, Prof. Lockemann 4 BTW03 work unit What’s a service? resource manager work unit service client service provider competence requirements service level agreement service = obligation Service design: Determine the competences needed by a broad clientele Settle which obligations (responsibilities) to accept

© 2003 IPD, Prof. Lockemann 5 BTW03 work unit What’s in a service? resource manager work unit service client service provider service = obligation The core: Functionality Qualities of service: Ubiquity Durability Interpretability Robustness Security Performance Scalability

© 2003 IPD, Prof. Lockemann 6 BTW03 Characterization of a service The core: Functionality: Collection of functions Qualities of service: Ubiquity: Unrestricted access in space (data comm.) Durability: Unlimited access in time (data storage) Common Interpretability in space and time: Consistency as best-effort Robustness: Guarantees under all circumstances: failure resilience, conflict resilience, function persistence Security: No effects other than those guaranteed Performance: Service level at cost, e.g., throughput, response time Scalability: Guaranteed service level under growth

© 2003 IPD, Prof. Lockemann 7 BTW03 The design goal: service hierarchies The basics of architectural design: divide-and-conquer functionality service higher-level responsibility qualities lower-level responsibilities functionality service qualitiesfunctionality service qualitiesfunctionality service qualities

© 2003 IPD, Prof. Lockemann 8 BTW03 Our design hypothesis functionality service higher-level responsibility qualities composition: assemble higher- level responsibility decomposition: divide higher-level responsibility lower-level responsibilities functionality service qualitiesfunctionality service qualitiesfunctionality service qualities 1. Functional decomposition ???

© 2003 IPD, Prof. Lockemann 9 BTW03 Our design hypothesis functionality service higher-level responsibility qualities lower-level responsibilities functionality service qualitiesfunctionality service qualitiesfunctionality service qualities 1. Functional decomposition ??? 2. Propagation of service qualities Exclusive control: sole responsibility, no propagation Partial control: shared responsibility, partial propagation Complete delegation: full propagation

© 2003 IPD, Prof. Lockemann 10 BTW03 Our design hypothesis functionality service higher-level responsibility qualities lower-level responsibilities functionality service qualitiesfunctionality service qualitiesfunctionality service qualities 1. Functional decomposition ??? 2. Propagation of service qualities Exclusive control: sole responsibility, no propagation Partial control: shared responsibility, partial propagation Complete delegation: full propagation 3. Priority of service qualities Choose one as the primary quality Refine decomposition Consider others as orthogonal

© 2003 IPD, Prof. Lockemann 11 BTW03 Relational DBMS as our testbed n Functional decomposition u Relational functionality as the service. n Propagation of service qualities u QoS to be considered: durability, consistency, robustness, performance, scalability. n Priority of service qualities u Durability requires non-volatile storage medium: Slow magnetic disk as the physical bottleneck. u Thesis 1: Performance is primary QoS. Thesis 2: Observe performance on all levels  partial control Question: Are the remaining QoS orthogonal?

© 2003 IPD, Prof. Lockemann 12 BTW03 Performance as a shared responsibility resource manager i data model D i Data model more expressive data model less expressive data model data model D i-1 wider usage context narrower usage context Performance Data model mapping access profile Data staging

© 2003 IPD, Prof. Lockemann 13 BTW03 DBMS reference architecture External data model Data rearrangement Query processing Internal data model Data modelData staging Static data structure Access density Management of sets of records Assignment of physical data structs Physical data structures Access patterns Data types: Sets of records Operators: Set operators Data types: Records and sets of records Operators: Record operators Data types: Lists, Trees, Hash tables,... Operators: Lists: seq. navigation Trees: seq. navigation, key search Hash tables: key search

© 2003 IPD, Prof. Lockemann 14 BTW03 DBMS reference architecture External data model Data rearrangement Query processing Internal data model Data modelData staging Static data structure Access density Management of sets of records Assignment of physical data structs Physical data structures Access patterns Data types: Sets of records Operators: Set operators Data types: Records and sets of records Operators: Record operators Data types: Lists, Trees, Hash tables,... Operators: Lists: seq. navigation Trees: seq. navigation, key search Hash tables: key search Blocks and files File management Parameter control Device interface Storage structure Device I/O Data types: Block = fixed number of Bytes File = variable number of blocks Operators: Create/open/close of files Read/write of blocks Pages and segments Segment and buffer management Mapping pages to blocks Timely provision of data in main memory Transparency of peripheral storage Data types: Page = fixed number of Bytes Segment = variable number of pages Operators: Request/release of pages Create/open/close of segments Physical proximity

© 2003 IPD, Prof. Lockemann 15 BTW03 DBMS reference architecture External data model Data rearrangement Query processing Internal data model Data modelData staging Static data structure Access density Management of sets of records Assignment of physical data structs Physical data structures Access patterns Data types: Sets of records Operators: Set operators Data types: Records and sets of records Operators: Record operators Data types: Lists, Trees, Hash tables,... Operators: Lists: seq. navigation Trees: seq. navigation, key search Hash tables: key search Blocks and files File management Parameter control Device interface Storage structure Device I/O Data types: Block = fixed number of Bytes File = variable number of blocks Operators: Create/open/close of files Read/write of blocks Pages and segments Segment and buffer management Mapping pages to blocks Timely provision of data in main memory Transparency of peripheral storage Data types: Page = fixed number of Bytes Segment = variable number of pages Operators: Request/release of pages Create/open/close of segments Physical proximity Record clustering Implementation on pages

© 2003 IPD, Prof. Lockemann 16 BTW03 Relational DBMS as our testbed n Functional decomposition u Relational functionality as the service. n Propagation of service qualities u QoS to be considered: durability, consistency, robustness, performance, scalability. n Priority of service qualities u Durability requires non-volatile storage medium: Slow magnetic disk as the physical bottleneck. u Thesis 1: Performance is primary QoS. Thesis 2: Observe performance on all levels  partial control Question: Are the remaining QoS orthogonal?

© 2003 IPD, Prof. Lockemann 17 BTW03 The remaining qualities logical data model data rearrangement query translation and optimization internal data model management of sets of records assignment of physical data structs physical data structures implementation on pages segment management mapping pages to blocks buffer management device interface file management parameter control metadata model metadata management transaction management scheduling logging recovery archive management backup long-range storage consistency robustness durability

© 2003 IPD, Prof. Lockemann 18 BTW03 Semistructured databases (XML) Relational front-ends Native systems XML structures structural mapping query translation and optimization assignment of physical data structs to subtrees and media fields subtree implementations segment management mapping pages to blocks buffer management device interface file management parameter control media data to block clusters physical data structures index implementation physical data structures different data model less regularity tree navigation as access profile text orientation performance, performance,...  More difficult decomposition!

© 2003 IPD, Prof. Lockemann 19 BTW03 Construction: What if … n Functional decomposition u Relational functionality as the service. n Propagation of service qualities u QoS to be considered: durability, consistency, robustness, performance, scalability. n Priority of service qualities u Durability requires non-volatile storage medium: Slow magnetic disk as the physical bottleneck. u Thesis 1: Performance is primary QoS. Privacy Motivation: VLDB2002 paper on Hippocratic databases by R. Agrawal et al. QoS of purpose

© 2003 IPD, Prof. Lockemann 20 BTW03 Hippocratic databases: A new try sets of records descriptive queries purpose descriptions Functionalities (data model) Performance sets of records query graphs purpose descriptions sets of records with identical purpose record-oriented access data records and main storage blocks block-oriented access logical data model query processing purpose bound data model query graph transformation with propagation of query and database purposes secure internal data model implementation of physical data structures for sets and indexes, operator implementations secure buffers protected cache management, placement of records in segments + pages, extraction of records from pages (query, query purpose) traditional storage engine Database purposes (rules, authorizations) Purpose-restricted visibility Full visibility Firewall The price of privacy is performance!

© 2003 IPD, Prof. Lockemann 21 BTW03 Construction: What if … n Functional decomposition u Relational functionality as the service. n Propagation of service qualities u QoS to be considered: durability, consistency, robustness, performance, scalability. n Priority of service qualities u Durability requires non-volatile storage medium: Slow magnetic disk as the physical bottleneck. u Thesis 1: Performance is primary QoS. Consistency Motivation: Heavy use of constraints to make the databases a closer model of the real world

© 2003 IPD, Prof. Lockemann 22 BTW03 Incorruptible databases constrained logical data model routing compile services deductive processing of constraint sets into inference modules and relational queries queries, constraints traditional storage engine just-in-time services deductive processing of queries using the inference modules relational services application of augmented relational operators to cached database sections relational data model query translation and optimization operator mapping physical data structures implementation on pages

© 2003 IPD, Prof. Lockemann 23 BTW03 From service hierarchy to an architecture functionality service qualities functionality service qualitiesfunctionality service qualitiesfunctionality service qualities close couplingloose coupling component 1 component 2 component architecture layer 1 layer 2 layered architecture

© 2003 IPD, Prof. Lockemann 24 BTW03 Coupling, layers, components Candidate criteria for components : n Adjacent services share no resources  loose coupling n No shared responsibilities  loose coupling n Partial control, but by different technical means  loose coupling n Low traffic density  loose coupling n Broad applicability of a service Otherwise choose layers. Actual decision a compromise between several criteria.

© 2003 IPD, Prof. Lockemann 25 BTW03 Applying the criteria logical data model data rearrangement query translation and optimization internal data model management of sets of records assignment of physical data structs physical data structures implementation on pages segment management mapping pages to blocks buffer management device interface file management parameter control metadata model metadata management transaction management scheduling logging recovery archive management backup long-range storage Query engine ? ? ? OS Storage engine

© 2003 IPD, Prof. Lockemann 26 BTW03 What if components are already given? n Much more limited design space! n Hypothetical design hypothesis: 1. Assign service qualities to each component. 2. If the component can be influenced, develop it according to the earlier design hypothesis. 3. For each quality shared between adjacent components and arranged in order of priority, increase by technical means the strength of the coupling. 4. Re-evaluate and adjust the adjacent components in the light of the coupling technique. local optimum! global optimum!

© 2003 IPD, Prof. Lockemann 27 BTW03 Imposed: 4-tier architecture Application Server... Middleware Client Database Server other Servers... Resources WWW Server... Business logic Presentation Application

© 2003 IPD, Prof. Lockemann 28 BTW03 physical layer data link network transport Unconventional idea: multidimensional architectures physical layer data link network transport file management segment management physical data structs. internal data model logical data model middleware client middleware physical medium

© 2003 IPD, Prof. Lockemann 29 BTW03 Performance, performance … and data staging Application Server... Middleware Client Database Server other Servers... Resources WWW Server... Business logic Presentation Application Strengthen the coupling for better performance  data staging

© 2003 IPD, Prof. Lockemann 30 BTW03 Data staging = caching Application Server Middleware Client Database Server WWW Server Cache Separate component: Non-intrusive but high communication overhead Integration into middleware: middleware service, lesser communication overhead, general-purpose into server (which?): intrusive but tailored to the needs Migration to the edge to middleware: added middleware service, lesser communication overhead, general-purpose to server (which?): non-intrusive tailored to the needs, but overhead

© 2003 IPD, Prof. Lockemann 31 BTW03 What about the other qualities? Application Server Middleware Client Database Server WWW Server Cache Consistency: Data replication  Distributed transactions (weakens autonomy) Cache invalidation/coherency Robustness: Distributed transactions (weakens autonomy) E-commerce qualities, e.g., non-repudiation, irrevocability Privacy/purpose: Chain of firewalls? More interdependence / less orthogonality between QoS, hence optimum more difficult to find? More design iterations?

© 2003 IPD, Prof. Lockemann 32 BTW03 Conclusions n Experienced designers don’t need a design methodology – they do it right by common sense … or do they? n But it might help the novice designer. n Services is customer-orientation! n Service qualities act as the guiding principle. n Choice of priorities determines the result. n … and finally: Systematical exploration of the design space!