Download presentation
Presentation is loading. Please wait.
1
software modularity group
Using AspectC to Improve the Modularity of Path-Specific Customization in Operating System Code Operating systems have a problem with modularity. Case study that we’ve done using a prototype for a programming language called AspectC, which is based on AspectJ, And we’ve used it to improve one particular type of modularity problem in OS code – path specific customizations. Yvonne Coady, Gregor Kiczales, Michael Feeley, Greg Smolyn University of British Columbia
2
perspective modularity problems are common in OS code
“unintentional interaction” [Lehman, OS/ ] “patterns of interaction” [Vogels, WindowsNT 1999] “fragile and intricate mess” [Engler, Best Paper OSDI 2000] can aspect-oriented programming help? case study applying AspectC to UNIX (FreeBSD v3.3) refactored an existing system focus on path-specific customization modular prefetching for virtual memory, file system layering plus explicit crosscutting structures with expected benefits of improved modularity <theory and practice> dijkstra proposed layers sounds good, but not what really happens – it’s just not that clean… Lehman and belady’s study of OS/360 refered to “unintentional interaction” between modules causing structural decay, Vogels study of WindowsNT refers to “patterns of interaction” between modules inhibiting independent development, Engler summed it all up recently saying that OS code is a “fragile and intricate mess” i’m sure you can imagine this, I’ll be giving one example, well-documented In this talk are going to characterize one source of this problem…
3
problem overview some elements crosscut layers
in particular, some align with specific execution paths path-specific customizations concern scattering dynamic context passing associated layer violations <general problem> That source is crosscutting – by that I mean that some elements of the system just naturally get scattered in the code, and tangled within these layers. But they do have a structure, that of a vertical slice tied to a specific execution path. The reason why this poses problems within a traditional modularity based on layering, is that these kinds of concerns require that dynamic context be passed along the path – revealing the context of a call and tailoring an otherwise general purpose service based on that context. Further, we find that this tailoring can involve management of non-native abstractions from a higher layer. clear layers some concerns follow exec paths scattering, tangling, context passing, layer violations -- in part, our inspiration came from a person in the next cubicle in the systems lab…
4
solution overview we want to modularize path-specific customizations
AspectC constructs localize crosscutting concerns aspect path_spec_cust { } What we want to do is modularize this type of concern by making crosscutting structure explicit. So that is the theme in a nutshell, now for some details… basically a concrete example of the problem, our solution, an analysis and evaluation of our solution in terms of modularity. pointcut path_name(p1); explicit path structure & context before function_f(p2) && path_name(p1) { /* regular C code */ } attach code to points on path
5
OS refresher layers and abstractions prefetching virtual memory layer
abstractions: virtual addresses (VM object, VM pages, page map) focus issue: nonresident page means page fault… file system layer abstractions: files (logical blocks, buffer cache) focus issue: services VM page faults prefetching the crosscutting concern we want to modularize amortize disk costs according to pattern of access normal access pattern prefetch window around address sequential prefetch after address <mini tutorial – role of layers, role of prefetching> the crosscutting concern we want to modularize is…. But first an OS refresher – simplified look at 2 layers and their abstractions VM layer based on Mach 2.0 VM object – represents entities in VM such as files, An object is composed of VM pages handled by a VM pager which is responsible for moving pages between physical memory and a backing store because they don’t all fit into physical memory at one time – so the key mapping it provides is between virtual and physical pages. Our focus issue is what happens when a VM page that is not in physical memory is accessed? It is called a page fault. Beneath VM there is a file system layer – abstractions are files, which are compose of logical blocks and typically pass through a buffer cache, the key mapping in this layer is from logical blocks to disk blocks, and our focus issue here is that the file system services page faults. The crosscutting concern that we are focussed on is prefetching. Prefetching is the OS way of amourtizing the cost of going to disk by predicting what else might be necessary to retreive – and getting it too. This prediction is based on the pattern of access, be it random – in which case there is no prefetching, or sequential in which case data that comes after the current request would be prefetched. two layers: VM and File System, two sets of abstractions
6
2 path-specific customizations
vm_fault VM layer normal access pattern sequential access pattern vm_pager_getpages … vnode_pager_getpages FFS layer ffs_getpages ffs_read <normal path looks like this> So when we open a file, map it into memory, and access it, we cause a page fault – the execution path looks like this – where the ovals represent functions in a layer and the purple line is an execution path. It starts in the VM layer, moves through a function from the pager, and into the file sytem, In this case FFS. This is considered to be Normal access – and prefetching accompanies this disk request. … ufs_bmap DISK bio_wait
7
original prefetching code
poorly modularized scattered (spread around) ~260/2000 lines: ~125 in VM, ~135 in FFS tangled (mixed abstraction levels) dynamic context passed through layers management of non-local data abstractions violates layers more: execution paths, pagers, file systems… ramifications of this implementation hard to unplug hard to comprehend hard to develop independently <if you sat down and looked at this code, this is what you’d see> If you look at the implementation of prefetching – it is poorly modularized… Dynamic context reveals that we are servicing a VM request, And file system includes code to manage VM data abstractions as a result of this poor modularity, prefetching is hard to unplug… develop independently… comprehend…
8
page fault during normal access pattern
initial prefetching plan based on access pattern, location on disk bury plan in allocated pages vm_fault VM layer vm_pager_getpages … vnode_pager_getpages cancel prefetching if: faulted page is now resident request no longer on disk blocks no longer contiguous side-effect allocated pages FFS layer ffs_getpages ffs_read <how normal works> Looking at a little more detail into this implementation… … ufs_bmap bio_wait
9
page fault for normal access
initial prefetching plan based on access pattern, location on disk bury plan in allocated pages vm_fault VM layer vm_pager_getpages … vnode_pager_getpages FFS layer int ffs_getpages( … ) { … for (…) vm_page_free(ap->a_m[i]); … } ffs_getpages ffs_read <how normal works> Looking at a little more detail into this implementation… … ufs_bmap bio_wait
10
(hidden) design structure of prefetching
normal access pattern VM layer: predict & plan +/- window costs allocation FS layer: check & fetch current state de-allocation synchronous <code is a mess, but actually is a design structure here> But – though it is hard to see it when looking at the code – there is a design structure for prefetching – same factors normal/sequential – different details… It depends on the access pattern and accompanies different execution paths as they crosscuts the VM and FS layers. So we organized it into special crosscutting modules, we call aspects.
11
normal access pattern prefetching aspect
/* vm fault path & context passing from top */ int vm_fault(map, .. ) pointcut vm_fault_path( vm_map_t map ): cflow( ); calls( ) vm_fault VM layer vm_pager_getpages … predict and plan (allocate) … vnode_pager_getpages FFS layer vm_map_t map && vm_fault_path( map ) { if ( object->declared_behaviour == NORMAL ) { vm_map_lock( map ); plan_and_alloc_normal( object ); vm_map_unlock( map ); } before( vm_object_t object, ): calls( int vnode_pager_getpages( object, .. )) ffs_getpages Simplification of parameter lists… Gregor Kiczales: this is a simplification… 1 – vm_fault 2 – calls 3 – cflow 4 – name it, and say includes parameter take 20 seconds on naming it <walk through normal mode> aspectc lets us talk about the execution path explicitly, pass context along it… - the path, context from the top - work at a point along the path, local context plus context from the top - sub-path - work at point along it NOTE: this code is removed from the code in the layers, operates in terms of abstraction of layers (function interfaces, not bodies) Aside: We refactored the code – pulled prefetching out of this primary functionality – two new small functions Purple boxes: we are not going inside the functions … an abstraction of the execution path Associating prefetching with normal page fault path – hi/lo level ffs_valid ffs_calc_size ufs_bmap Block layer
12
2 prefetching aspects VM layer normal access pattern
vm_fault VM layer normal access pattern sequential access pattern -> prefetching separated vm_pager_getpages … predict & plan check & fetch … vnode_pager_getpages FFS layer ffs_read ffs_getpages <walk through normal mode> aspectc lets us talk about the execution path explicitly, pass context along it… - the path, context from the top - work at a point along the path, local context plus context from the top - sub-path - work at point along it NOTE: this code is removed from the code in the layers, operates in terms of abstraction of layers (function interfaces, not bodies) Aside: We refactored the code – pulled prefetching out of this primary functionality – two new small functions Purple boxes: we are not going inside the functions … an abstraction of the execution path Associating prefetching with normal page fault path – hi/lo level ffs_valid ffs_calc_size ufs_bmap Block layer
13
aspect localization analysis 1/5
aspect normal_prefetching { pointcut vm_fault_cflow( vm_map_t map ): cflow( calls( int vm_fault( map, .. ))); pointcut ffs_getpages_cflow( vm_obj_t obj, vm_page_t* p, int len, int fpage ): cflow( calls( int ffs_getpages( obj, p, len, fpage ))); before( vm_map_t map, vm_object_t obj, vm_page_t* p, int len, int fpage ): calls( int vnode_pager_getpages( obj, p, len, fpage )) && vm_fault_cflow( map ) { if ( obj->declared_behaviour == NORMAL ) { vm_map_lock( map ); plan_and_alloc_normal( obj, p, len, fpage ); vm_map_unlock( map ); } after( vm_obj_t obj, vm_page_t* p, int* len, int fpage, int valid ): calls( valid check_valid(..) ) && ffs_getpages_cflow( obj, p, len, fpage ) if ( valid ) dealloc_all_prefetch_pages( object, pagelist, length, faulted_page ); after( vm_obj_t obj, vm_page_t* p, int len, int fpage, int error, int* reqno ): calls( error ufs_bmap( struct vnode*, reqno, ..) ) && if ( error || (*reqno == -1) ) dealloc_all_pref_pages( obj, p, len, fpage ); after( vm_obj_t obj, vm_page_t* p, int* len, int fpage, struct args* t_args ): calls( int calc_range( t_args )) && dealloc_noncontig_prefetch_pages( obj, p, len, fpage, t_args ); normal access pattern prefetching code is localized to a single aspect (helper functions not shown) sequential access pattern prefetching is a separate aspect what is the ONE important point about why your code is better than the original code that this slide makes - the code for this aspect is now textually localized why does that matter? (un)pluggability, comprehensibility, reuse? how does this slide shows above? i’m showing all the code as one contiguous block what are the twenty words I most need to say The first thing one might notice about our implementation is it textually localizes prefetching code into special crosscutting modules we call aspects.
14
explicit path-specific structure
analysis 2/5 prefetching normal access pattern predict & plan pointcut vm_fault_path(map m): cflow(calls(vm(m))); before(…): calls(get())&& vm_fault_path(…){} check & fetch after(…): calls(val()) && ffs_path(…){} after(…): calls(bmap()) && after(…): calls(xfer()) && dynamic context explicit: name the path propagate values test if on path <code is a mess, but actually is a design structure here> But – though it is hard to see it when looking at the code – there is a design structure for prefetching – same factors normal/sequential – different details… It depends on the access pattern and accompanies different execution paths as they crosscuts the VM and FS layers. So we organized it into special crosscutting modules, we call aspects.
15
clear aspect / layer interaction
analysis 3/5 prefetching normal access pattern predict & plan pointcut vm_fault_path(map m): cflow(calls(vm(m))); before(…): calls(get())&& vm_fault_path(…){} check & fetch after(…): calls(val()) && ffs_path(…){} after(…): calls(bmap()) && after(…): calls(xfer()) && <code is a mess, but actually is a design structure here> But – though it is hard to see it when looking at the code – there is a design structure for prefetching – same factors normal/sequential – different details… It depends on the access pattern and accompanies different execution paths as they crosscuts the VM and FS layers. So we organized it into special crosscutting modules, we call aspects.
16
code, design structure parallel
analysis 4/5 prefetching normal sequential predict & plan pointcut vm_flt_path(map m): cflow(calls(vm(m))); before(…): calls(…)&& vm_flt_path(m){} before(…): calls(…) && check & fetch after(…): calls(…) && ffs_path(…){} around(…):calls(…){} after(…):calls(…) && vm_flt_path(…)&& what is the ONE important point about why your code is better than the original code that this slide makes - the aspect has an internal structure that clearly aligns with the design structure why does that matter? comprehensibility, code looks like design is good, how does this slide shows above? showing a mapping of code onto design structure what are the twenty words I most need to say these aspects crosscut the VM and FFS layers and have an internal structure that corresponds to the design of prefetching: planning and allocating/checking and fetching
17
clear aspect / aspect interaction
analysis 5/5 prefetching normal sequential predict & plan pointcut vm_flt_path(map m): cflow(calls(vm(m))); before(…): calls(…)&& vm_flt_path(m){…} before(…): calls(…) && check & fetch after(…): calls(…) && ffs_path(…){} around(…):calls(…){} after(…):calls(…) && vm_flt_path(…)&& what is the ONE important point about why your code is better than the original code that this slide makes – clear aspect interaction (independence) why does that matter? comprehensibility, reason about aspects independently, (reason in terms of interface?) how does this slide shows above? we’ve identified a point of of intersection and how we reason about it what are the twenty words I most need to say in terms of this example, the point in the execution path where planning and allocation takes place is the same for both aspects, but they do not communicate w/ each other <independent aspects> run in same context access same value
18
analysis summary scattered, tangled localized, structured
was: ~260 lines out of 2000 lines from 2 layers is : ~260 lines in 3 aspects ~110 VM normal, ~90 VM seq, ~60 FS seq interactions are now clear between aspects and layers among aspects dynamic context explicit crosscutting structure is explicit What have we got… <what we have shown> (note: get these numbers right!) prefetching is not one big aspect - execution path specific – we can structure it accordingly no reduction in amount of code this structure would have helped our friend in the lab aspect & layers: vm and ffs dynamic context: cflow principled layer xcut: vm_page_free in FFS layer among aspects: independent but related in terms of design
19
improved modularity unpluggability comprehensibility
textually localized prefetching modes OS runs with different combinations of aspects comprehensibility aspect interaction with rest of code aspect internal structure independent development clear interface for prefetching aspect what functions it knows about what arguments it sees abstract interface of page fault handling Expected benefits of modularity <compared to original implementation> now plug and play: none, normal, sequential,both - compile time selection in makefile independent dev: we now have clear boundaries we can look at an aspect and see from it’s interface, what functions/arg we have an abstraction of page fault handling, which prefetching accompanies, but not the details comprehensibility aspectC code allows us to chain together prefetching mechanism to specific points in execution paths knowing when it happens gives us insight into how it works and each prefetching aspect adheres to a similar internal structure
20
open issues scalability efficiency tool support
now have 10 prefetching aspects, working on more but, need to see how it scales different aspects, system-wide efficiency system runs, “feels fine”, simple tests but, have not done real performance studies tool support has clear role to play but, following other work in AOSD <where we fit in relation to these things…> THE: layers, US: primary functionality as layers + crosscutting path specific customizations as aspects Synthetix: automatically optimize execution paths with partial eval, US: customize execution path at source level MC: flag crosscutting rule violations at compile time, US: structure crosscutting so easier to see in source HyperJ/CF: separation at source, not necessarily a primary functionality, but multiple dimensions, US: primary funct, dynamic context IC: separate things like dynamic context from core functionality at source, US: then chain an implementation assoc w/ diff functs joined by an exec path Perl/Tcl: access to call stack, US: just want principled access to a few parameters IP: allows parameters to skip intervening functions, source must be written to use it US: similar to cflow, don’t change primary funct
21
related work using programming languages to address software engineering problems in systems code… OS structure THE [Dijkstra], Synthetix [Pu] meta-level compilation [Engler] separation of concerns subject-oriented programming [Ossher], composition filters [Aksit] implicit context [Walker] languages dynamic context (Lisp(s), Perl and Tcl) implicit parameters [Lewis] <where we fit in relation to these things…> THE: layers, US: primary functionality as layers + crosscutting path specific customizations as aspects Synthetix: automatically optimize execution paths with partial eval, US: customize execution path at source level MC: flag crosscutting rule violations at compile time, US: structure crosscutting so easier to see in source HyperJ/CF: separation at source, not necessarily a primary functionality, but multiple dimensions, US: primary funct, dynamic context IC: separate things like dynamic context from core functionality at source, US: then chain an implementation assoc w/ diff functs joined by an exec path Perl/Tcl: access to call stack, US: just want principled access to a few parameters IP: allows parameters to skip intervening functions, source must be written to use it US: similar to cflow, don’t change primary funct
22
summary OS modularity challenge our solution
path-specific customizations needs dynamic context breeds violations our solution explicit support for crosscutting with AspectC improves modularity in the small unpluggability comprehensibility independent development <the end> Prob, sol, benefits.
23
crosscutting in context
Valued Sony Customer: Delete this slide, renumber others crosscutting in context analysis 2/6 aspect normal_prefetching { pointcut vm_fault_cflow( vm_map_t map ): cflow( calls( int vm_fault( map, .. ))); pointcut ffs_getpages_cflow( vm_obj_t obj, vm_page_t* p, int len, int fpage ): cflow( calls( int ffs_getpages( obj, p, len, fpage ))); before( vm_map_t map, vm_object_t obj, vm_page_t* p, int len, int fpage ): calls( int vnode_pager_getpages( obj, p, len, fpage )) && vm_fault_cflow( map ) { if ( obj->declared_behaviour == NORMAL ) { vm_map_lock( map ); plan_and_alloc_normal( obj, p, len, fpage ); vm_map_unlock( map ); } after( vm_obj_t obj, vm_page_t* p, int* len, int fpage, int valid ): calls( valid check_valid(..) ) && ffs_getpages_cflow( obj, p, len, fpage ) if ( valid ) dealloc_all_prefetch_pages( object, pagelist, length, faulted_page ); after( vm_obj_t obj, vm_page_t* p, int len, int fpage, int error, int* reqno ): calls( error ufs_bmap( struct vnode*, reqno, ..) ) && if ( error || (*reqno == -1) ) dealloc_all_pref_pages( obj, p, len, fpage ); after( vm_obj_t obj, vm_page_t* p, int* len, int fpage, struct args* t_args ): calls( int calc_range( t_args )) && dealloc_noncontig_prefetch_pages( obj, p, len, fpage, t_args ); crosscutting still there – but now local to aspect what is the ONE important point about why your code is better than the original code that this slide makes - layer violations are localized, appear in explicit crosscutting context why does that matter? crosscutting structure of prefetching inherently means will be some layer violations, but its better to localize this, context makes it more clear why its happening, (could even type check it? privileged aspects) how does this slide shows above? layer specific colour-coding what are the twenty words I most need to say We have not moved functionality so that there is no more crosscutting – it is still there – but it is explicit and colocated with the appropriate context in which it is used <layer violation in context>
24
MORE… VFS interface … more pagers more file systems SWAP pager DEVICE
VNODE pager VFS interface bio_wait … bio_wait … bio_wait … bio_wait … … <more complicated than this…> There are more pagers and more file systems beneath a virtual file system interface. The low level details of prefetching are different in these different contexts. NFS FFS NTFS EXT2FS
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.