DiProNN Resource Management System (DiProNN = Distributed Programmable Network Node) Tomáš Rebok Faculty of Informatics MU, Brno Czech Republic

11/15/2008MEMICS 2008, Znojmo2 Programmable Networks  (current) computer networks passive transport medium  Programmable Networks inner network elements provide user-controlled computations on passing data stream extension of the passive transport infrastructure ─ overlay principle (programmable nodes) active program(s) uploading applications: ─ multimedia stream distribution/transcoding, state space search, data pre-processing, etc.

11/15/2008MEMICS 2008, Znojmo3 Virtualization  (Platform) Virtualization hides the physical characteristics of computing platform and shows another (abstract, emulated) computing platform virtualization of computers or operating systems performed on a given hardware platform by host software (a control program – hypervisor, VMM) ─ creates a simulated computer environment – a virtual machine (VM) ─ the VM is able to run an arbitrary operating system (OS)

11/15/2008MEMICS 2008, Znojmo4  benefits: nodes are able to run several OSs simultaneously ─ providing execution environments for user active programs nodes enable users to upload whole VMs (OSs) ─ containing active programs strict isolation of user processes strict scheduling of resources to VMs unification of user processes on a lower layer (HW layer) than in common non-virtualized systems (OS layer) Programmable Networks & Virtualization

11/15/2008MEMICS 2008, Znojmo5 DiProNN Architecture  distributed (cluster-based) architecture able to process high amount of data capable of parallel processing  VM-based design loadable functionality ─ users may upload whole OS and/or an active program only static and dynamic VMs  transport protocol independent design operates up to the Network/IP layer  virtualization system independent design  novel programming model ([Rebok:MEMICS07]) based on workflow principles complex functionality might be formed by assembling several single-purpose active programs

11/15/2008MEMICS 2008, Znojmo6 DiProNN Architecture

11/15/2008MEMICS 2008, Znojmo7 DiProNN Programming Example: Transcode A/V streams, synchronize and send to the user. Simultaneously, save merged stream in original quality.

11/15/2008MEMICS 2008, Znojmo8 DiProNN Resources  HW resources each Processing unit has (and may provide to users’ programs) several HW resources CPU(s), memory, network, disk  SW resources specialized functionality ─ e.g., classical routing execution environments ─ e.g., Linux, Windows, FreeBSD, Irix built-in active programs ─ e.g., functionality available for user sessions

11/15/2008MEMICS 2008, Znojmo9 DiProNN Resource Management System  maintains both HW and SW resources  consists of two processes resources discovery process resources allocation process

11/15/2008MEMICS 2008, Znojmo10 DiProNN Resource Management System (Resources Discovery)  What HW resources can all the units provide?  What SW resources can all the Processing units provide?  2 steps: collecting resources information on the units providing discovered information to DiProNN Control unit

11/15/2008MEMICS 2008, Znojmo11 DiProNN Resource Management System (Resources Allocation)  users specify resource requests in their DiProNN programs for each VM ─ for each active program in the case it is the only active program running in the VM  if the requested resources are available, they are reserved reservation depends on the virtualization system used

11/15/2008MEMICS 2008, Znojmo12 Conclusions  technical details about the RMS are described in my paper  prototype implementation based on the XEN VMM (virtualization system) comprehensive resource management system (RMS) tests available at  further work exploring scheduling of DiProNN sessions from the global point of view  acknowledgements research intent “Intergrated Approach to Education of PhD Students in the Area of Parallel and Distributed Systems” (No. 102/05/H050) research intent “Optical Network of National Research and Its New Applications” (MŠM )

Thank you for your attention. Tomáš Rebok Questions?