1. 11 September 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser Kaiser+4156@cs.columbia.edu http://york.cs.columbia.edu/classes/cs4156/

2. 11 September 2007Kaiser: COMS W4156 Fall 20072 How does client identify server? How does server make its functionality known and available to prospective clients? How does client make its request? How does server return its response? What design-time support is available? What run-time support is available? Distributed Computing 101 ClientServer

3. 11 September 2007Kaiser: COMS W4156 Fall 20073 Client finds Server Avoid hard-wired solution –Tightly coupled code where any “enhancement” to server may require modifications to client and vice versa –Cannot plug-replace with new server from another vendor (better, faster, cheaper) –System Administration nightmare Better to employ some standard “discovery protocol”

4. 11 September 2007Kaiser: COMS W4156 Fall 20074 Server defines functionality provided Are interfaces used? If so, how expressive? Are interfaces enforced? What happens when the server functionality is “upgraded” (versioning)

5. 11 September 2007Kaiser: COMS W4156 Fall 20075 Client request/Server response mechanism a.k.a. “communication protocol” How complicated? How buggy? Percentage of code (and development effort) devoted to mechanism? ClientServer ClientServer Communication Layer

6. 11 September 2007Kaiser: COMS W4156 Fall 20076 Design-time support Quality Assurance –Errors caught at runtime 10x more expensive to fix than at design time –Reduce errors in first place through intuitive idioms –Type-safety checks incredibly useful (do parameters supplied by clients match those anticipated by server? do responses from server match those expected by client?) Interoperability –Must client and server use same development language? –Third-party code reuse

7. 11 September 2007Kaiser: COMS W4156 Fall 20077 Run-time support Standardized infrastructure crucial –Reduced training, design, coding and testing costs –More reliable and robust –Possibly higher performance Interoperability essential –Countless ‘incompatible’ proprietary standards

8. 11 September 2007Kaiser: COMS W4156 Fall 20078 Motivation for Component Model Frameworks “Plumbing” is difficult and error-prone –How client identifies server –How server makes its functionality known and available to prospective clients –How client makes its request and server returns its response –Design-time and run-time support But nearly the same across many applications Put system programmers’ effort into doing it once per framework rather than once per application Leaves application logic (and business value) to application programmers

9. 11 September 2007Kaiser: COMS W4156 Fall 20079 Quality Factors Component-based development intends to improve quality in various dimensions –Understandability and thus Maintainability –Reliability All software developed according to a given component model can easily interact –Reusability –Productivity –Leverages software engineer training

10. 11 September 2007Kaiser: COMS W4156 Fall 200710 CORBA = Common Object Request Broker Architecture Historically, one of the first organized frameworks for distributed computing (c. 1991) –Extremely influential –Used especially as a middleware in enterprise and business-critical infrastructures Not quite a component model, but on the way there… (later CCM = Corba Component Model)

11. 11 September 2007Kaiser: COMS W4156 Fall 200711 CORBA Standard specification developed and periodically revised by the Object Management Group “Open membership, not-for-profit consortium that produces and maintains computer industry specifications for interoperable enterprise applications” “Membership includes virtually every large company in the computer industry, and hundreds of smaller ones” www.omg.org www.corba.org

12. 11 September 2007Kaiser: COMS W4156 Fall 200712 How does client identify server? How does server make its functionality known and available to prospective clients? How does client make its request? How does server return its response? What design-time support is available? What run-time support is available? Reprise: Distributed Computing 101 ClientServer

13. 11 September 2007Kaiser: COMS W4156 Fall 200713 Big Picture Common Object Request Broker Architecture ClientServer IDL stub IDL skeleton Object Request Broker Request Interface Description Language Response

14. 11 September 2007Kaiser: COMS W4156 Fall 200714 Big Picture Client communicates request to ORB = Object Request Broker –IDL = Interface Definition Language compiler generates stub to hide complexity ORB delivers request to Server –Object-implementation resides on Server –IDL-generated skeleton hides complexity

15. 11 September 2007Kaiser: COMS W4156 Fall 200715 Server objects CORBA applications are composed of objects, individual units of running software that combine functionality and data Typically, there are many instances of an object of a single type – e.g., an e-commerce website would have many shopping cart object instances For some types, there may be only one instance – e.g., when a legacy application, such as an accounting system, is wrapped as a CORBA server object and opened up to clients on the network

16. 11 September 2007Kaiser: COMS W4156 Fall 200716 Object interfaces For each object type, developer defines an interface in OMG’s IDL (Interface Description Language) The interface is the syntax part of the contract that the server object offers to the clients Any client that wants to invoke an operation on the object must use this IDL interface to specify the operation it wants to perform, and to marshal the arguments that it sends When the invocation reaches the target object, the same interface definition is used there to unmarshal the arguments so that the object can perform the requested operation with them

17. 11 September 2007Kaiser: COMS W4156 Fall 200717 Big Picture Common Object Request Broker Architecture Client IDL stub IDL skeleton Object Request Broker Object Object IDL file Server

18. 11 September 2007Kaiser: COMS W4156 Fall 200718 IDL Neutral wrt programming language –OMG has defined mappings to C, C++, Java, COBOL, Smalltalk, Ada, Lisp, Python, PL/1, … –IDL Compilers generate native code Describe objects ‘hosted’ by a server –Decompose application into objects –Separate interface from implementation –Opportunities for reuse

19. 11 September 2007Kaiser: COMS W4156 Fall 200719 IDL ‘HelloWorld’ example module HelloApp { interface Hello { string sayHello(); }; Module is a scoping unit Interface is set of method signatures Base types defined by CORBA include string, int, double, float, boolean, etc…

20. 11 September 2007Kaiser: COMS W4156 Fall 200720 More complicated example module StockObjects { struct Quote { string symbol; long at_time; double price; long volume; }; exception Unknown{}; interface Stock { Quote get_quote() raises (Unknown); void set_quote (in Quote stock_quote); readonly attribute string description; }; Exceptions associated with modules, raised by methods Attributes

21. 11 September 2007Kaiser: COMS W4156 Fall 200721 IDL expressiveness Method Signatures –Declare Parameters as { in, out, inout } –Functions can raise Exceptions –Functions can return a value Declarative Attributes –{readonly} attribute {type} {name} –Equivalent to having _get_att/_set_att(in p) Multiple Inheritance –Interface ISpec:I1,I2 { … }

22. 11 September 2007Kaiser: COMS W4156 Fall 200722 Java example An ORB, IDL compiler and support packages come with JDK (RMI over IIOP) –Compiles into ‘native’ Java hello.idl Interface Hello.java Interface HelloOperations.java HelloHolder.java HelloHelper.java _HelloStub.java _HelloImplBase.java

23. 11 September 2007Kaiser: COMS W4156 Fall 200723 Hello interface package HelloApp; /** * HelloApp/Hello.java * Generated by the IDL-to-Java compiler from Hello.idl */ public interface Hello extends HelloOperations, org.omg.CORBA.Object, org.omg.CORBA.portable.IDLEntity { } package HelloApp; public interface HelloOperations { String sayHello (); }

24. 11 September 2007Kaiser: COMS W4156 Fall 200724 CORBA client-side Steps to contacting Server-side object –Connect to ORB –Contact NameService (standard service provided by CORBA) –Locate (‘resolve’) object by name –Typecast (‘narrow’) to specific interface –Invoke desired method

25. 11 September 2007Kaiser: COMS W4156 Fall 200725 CORBA client // 1. Connect to ORB ORB orb = ORB.init(args, null); // 2. Contact NameService org.omg.CORBA.Object objRef = orb.resolve_initial_references("NameService"); NamingContext ncRef = NamingContextHelper.narrow(objRef); // 3. Locate Object NameComponent nc = new NameComponent("Hello", ""); NameComponent path[] = { nc }; Hello helloRef = HelloHelper.narrow(ncRef.resolve(path));

26. 11 September 2007Kaiser: COMS W4156 Fall 200726 Client-side perspective Client shielded by interface Client accesses ORB services Client communicates with stub ‘proxy’ objects Client IDL stub object Object Request Broker interface

27. 11 September 2007Kaiser: COMS W4156 Fall 200727 CORBA server-side Steps for Server-side object –Connect to ORB –Contact NameService (standard service provided by CORBA) –Register (‘rebind’) object by name HelloServant helloRef = new HelloServant(); orb.connect(helloRef); ncRef.rebind(path, helloRef);

28. 11 September 2007Kaiser: COMS W4156 Fall 200728 Server-side perspective 1.ORB receives call 2.ORB passes request to object implementation through skeleton 3.Response sent back from object to skeleton 4.Sent back to client IDL skeleton Object Request Broker 1. Call 2. Invoke 3. Response 4. To Client Server

29. 11 September 2007Kaiser: COMS W4156 Fall 200729 3. Response 1a. Activate Basic Object Adapter (BOA) 1.ORB receives call a.ORB activates server b.Server calls BOA::impl_is_ready c.BOA instantiates object in Server 2.BOA passes request to object implementation through skeleton 3.Response sent back from object to skeleton 4.Sent back to client IDL skeleton Object Request Broker 1. Call 2. Invoke BOA 1b. Ready 1c. 4. To Client

30. 11 September 2007Kaiser: COMS W4156 Fall 200730 CORBA Summary Object Request Broker (ORB) acts as a “mediator” that abstracts –Object location: server method invocations always local, then ORB redirects –Networking issues: stub/skeleton code automatically generated, usually programmer can ignore –Activation issues: ORB automatically activates and deactivates server objects –Persistent state: Clients should not care whether server objects are persistent or not

31. 11 September 2007Kaiser: COMS W4156 Fall 200731 CORBA Evaluation Strengths 1.Interfaces hide complexities 2.Automatic language interoperability Weaknesses 1.Client must know server’s interface(s) 2.Java RMI does everything CORBA does and more… 3.And today’s component model frameworks do even more…

32. 11 September 2007Kaiser: COMS W4156 Fall 200732 CORBA Limitations [Before version 3.0, c. 2002, which includes “CORBA Component Model”] No common (mandatory) set of object services implemented by all ORBs No standard way of deploying server objects (adding new server objects to an ORB) –Each ORB infrastructure implementation has different approach to IMR (IMplementation Repository)

33. 11 September 2007Kaiser: COMS W4156 Fall 200733 CORBA Limitations No support for common programming idioms –Most server objects implemented as “factories”, creating a new object instance to deal with each client, but new factory code needs to be written for each server –Every programmer has same choices to make, between persistent and transient references, objects identified or not by a primary key, objects maintain persistent state or not, …

34. 11 September 2007Kaiser: COMS W4156 Fall 200734 CORBA Needs “Components” Binary (executable) units that can really be used interchangeably with any ORB –Allows graceful evolution by replacing one component with another –Eases porting to another ORB (better, faster, cheaper) Applications can then be built by assembling components –Components must define what they need and what they offer –Once assembled, deployment must be semi-automatic Need standard development, deployment and runtime environment [Added, more or less, by version 3.0, c. 2002, which includes “CORBA Component Model”]

35. 11 September 2007Kaiser: COMS W4156 Fall 200735 Second IDA Due Next Week! Tuesday 18 September, 10am Assignments posted on course websitecourse website Submit via CourseWorksCourseWorks Individual Development Assignment #2

36. 11 September 2007Kaiser: COMS W4156 Fall 200736 Software Engineering Activities System Engineering Process Selection and Training Requirements –Eliciting –Analysis –Recording Technology Selection and Training Design –Architecture –Components –Modules Coding –Unit Testing –Debugging Integration –Build –Integration Testing –Configuration Management System Testing –Performance Testing & Optimization –Acceptance Testing –Beta Testing Deployment –Delivery –Installation Operations –System Management –Maintenance –Upgrades Support Activities –Project Planning and Tracking –Customer Interaction –Process Improvement –Training –Documentation –Personnel Management

37. 11 September 2007Kaiser: COMS W4156 Fall 200737 Requirements What is the (future) software system supposed to “do”? And “why?” Requirements must be measurable, testable, related to identified business needs or opportunities, and defined to a level of detail sufficient for system design Usage scenarios –User stories –Use Cases –Process specifications –Generic natural language –…

38. 11 September 2007Kaiser: COMS W4156 Fall 200738 User Stories Written by the customers (or end-users) as things that the system needs to do for them About 3 sentences in the customer’s terminology, without technical detail Written on index cards (!) May be augmented by samples (e.g., formatted report) Prioritize (high, medium, low) Generally does not address non-functional requirements (e.g., performance, security)

39. 11 September 2007Kaiser: COMS W4156 Fall 200739 Example Story Card 111 Instructor View The instructor view for a given course includes the course number, name and semester; a button to edit the introduction; a button to designate library reserves; and a button to adjust settings for the course. Otherwise the instructor view is the same as the student view. Priority High

40. 11 September 2007Kaiser: COMS W4156 Fall 200740 Continuing Example 222 Student View The student view for a given course includes the course number, name and semester; general course information; and instructor information. There are buttons to select introduction, discussion, board, class e-mail, dictionary, notepad and help. Priority High

41. 11 September 2007Kaiser: COMS W4156 Fall 200741 Continuing Example 123 Instructor vs. Student View When an instructor selects one of the courses he/she is teaching, the instructor view is shown. The instructor view should include a button to show the student view, and then that special student view should include a button to switch back to the instructor view Priority Medium

42. 11 September 2007Kaiser: COMS W4156 Fall 200742 Continuing Example 321 Login The user is prompted to enter uni and password If the uni and password match database, the user is shown the default screen with his/her list of current courses Priority High

43. 11 September 2007Kaiser: COMS W4156 Fall 200743 Why are user stories on cards? Easy to rip up Tangible Unit of –Discussion –Estimation –Scheduling –Testing –Completion Used for: –Construction of engineering tasks –Creation of acceptance tests –Derivation of time estimates for planning

44. 11 September 2007Kaiser: COMS W4156 Fall 200744 Use Case Modeling Each use case provides one or more scenarios that convey how the system should interact with the end user or another system to achieve a specific business goal Use case model emphasizes the behavior as it appears to external actors, treats the system as a black box Partitions system functionality into interactions (‘use cases’) that are meaningful to users or external systems (‘actors’) Sometimes hundreds of use cases are needed to fully specify a system

45. 11 September 2007Kaiser: COMS W4156 Fall 200745 Use Case Diagram Using, e.g., UML = Unified Modeling Language Visualizes the high-level functions of the system and the system's scope –Including the relationship of "actors" to essential processes and system functionality –As well as the relationships among different use cases By looking at a use case diagram, you can easily tell the functions that the system provides (and doesn’t provide) Generally does not address non-functional requirements (e.g., performance, security)

46. 11 September 2007Kaiser: COMS W4156 Fall 200746 Example Use Case This system lets the band manager view a sales statistics report and the Billboard 200 report for the band's CDs It also lets the record manager view a sales statistics report and the Billboard 200 report for a particular CD The diagram also tells us that our system delivers Billboard reports from an external system called Billboard Reporting Service

47. 11 September 2007Kaiser: COMS W4156 Fall 200747 Continuing Example This use case diagram does not provide a way for a band manager to listen to songs from the different albums on the Billboard 200 — i.e., we see no reference to a use case called Listen to Songs from Billboard 200

48. 11 September 2007Kaiser: COMS W4156 Fall 200748 Another Simple Example

49. 11 September 2007Kaiser: COMS W4156 Fall 200749 Fig. 3-54, UML Notation Guide Not So Simple Use Case Relationships

50. 11 September 2007Kaiser: COMS W4156 Fall 200750 Yet Another Simple Example

51. 11 September 2007Kaiser: COMS W4156 Fall 200751 More Not So Simple Use Case Relationships

52. 11 September 2007Kaiser: COMS W4156 Fall 200752 Use Cases vs. User Stories Both intended to capture functional requirements Neither directly address non-functional requirements Use Cases are more formal User Stories tend to be more fine-grained User Stories don’t easily map relationships

53. 11 September 2007Kaiser: COMS W4156 Fall 200753 One problem with User Stories and User Cases Where do the usage scenarios come from? The customers (or users). But how do the customers know what to put in their usage scenarios?

54. 11 September 2007Kaiser: COMS W4156 Fall 200754 Requirements Elicitation Process of “discovering” the requirements for a system Through communication with customers, system users, system administrators and other stakeholders Through domain analysis Through investigation of previous and similar systems – reuse requirements –Saves time and money –Reduces risk –Reused requirements have a better chance of being understood by all stakeholders –Requirements reuse may lead to additional reuse in other lifecycle activities

55. 11 September 2007Kaiser: COMS W4156 Fall 200755 Requirements Elicitation Use Business Concerns to drive elicitation –If a system is to be useful, it must contribute to the key concerns of the business –If those business concerns are identified and used as drivers of the requirements elicitation process, there will be higher confidence that the system will meet real organization needs Collect requirements from Multiple Viewpoints –If the requirements are collected from a single viewpoint, unlikely to meet other stakeholders requirements –Identified viewpoints can be used to help organize requirements elicitation and the resulting requirements specification

56. 11 September 2007Kaiser: COMS W4156 Fall 200756 Requirements Challenges: “Yes, But” The first time users see the system, the first reaction is either, “Wow this is so cool” or “Yes, but, hmm, now that I see it what about this…?” This reaction is simply human nature Need to get the “buts” out early Anticipate that there will be “buts” and add time and resources to plan for feedback

57. 11 September 2007Kaiser: COMS W4156 Fall 200757 Requirements Challenges: “Undiscovered Ruins” Developers struggle with determining when they are done with requirements elicitation –How can we tell whether all the requirements have been elicited, or have we found at least enough? –Like asking an archeologist “how many undiscovered ruins are there?” First scope the elicitation effort by defining the business problem(s) to be solved by the system Employ techniques that help find some of those ruins and get the stakeholders to buy-into the requirements

58. 11 September 2007Kaiser: COMS W4156 Fall 200758 Requirements Challenges: “User versus Developer” Users do not know what they want, or cannot articulate it Users think they know what they want until developers give them what they said they wanted Developers think they understand user problems better than users do Recognize and appreciate the users as domain experts Provide alternative elicitation techniques earlier: storyboard, role playing, prototypes, etc. Put the developers in the users’ place

59. 11 September 2007Kaiser: COMS W4156 Fall 200759 Requirements Challenge: Living with “the Sins of your Predecessors” Like it or not, your customers and developers remember what happened in the past –“Total Quality Management” programs that promised things would be different –The last project where requirements were vague and/or were delivered short of expectations Need to build trust slowly: do not over-commit to features, schedule or budget Deliver highest priority features early

60. 11 September 2007Kaiser: COMS W4156 Fall 200760 Requirements Elicitation Techniques Interviewing and questionnaires Brainstorming and idea reduction Storyboarding Role Playing Requirements workshops Prototyping

61. 11 September 2007Kaiser: COMS W4156 Fall 200761 Technique: Interviewing Simple direct technique Context-free questions about what system needs to do for the users Convergence on some common needs will initiate a “requirements repository” A questionnaire is no substitute for an interview, but may precede or follow interviews

62. 11 September 2007Kaiser: COMS W4156 Fall 200762 Interview: Context-free questions Goal is to prevent prejudicing the user’s response to the questions Examples: –Who is the customer? –Who is the user? –Who is the user’s customer? –Are their needs different? –Where else can a solution to this problem be found? After context-free questions are asked, suggested solutions can be explored

63. 11 September 2007Kaiser: COMS W4156 Fall 200763 Technique: Brainstorming Brainstorming involves both idea generation and idea reduction The most creative, innovative ideas often result from combining seemingly unrelated ideas Various voting techniques may be used to prioritize the ideas created Although live brainstorming is preferred, web- based brainstorming may be a viable alternative

64. 11 September 2007Kaiser: COMS W4156 Fall 200764 Rules for Brainstorming Do not allow criticism or debate Let your imagination soar Generate as many ideas as possible Mutate and combine ideas Only then do Idea Reduction –Prune ideas that are not worthy of further discussion –Group similar ideas into one super topic Prioritize the remaining ideas

65. 11 September 2007Kaiser: COMS W4156 Fall 200765 Technique: Storyboarding Scripted walkthrough of system activities and/or screen mockups Identify the players, explain what happens to them, and describes how it happens Make the storyboard sketchy and easy to modify

66. 11 September 2007Kaiser: COMS W4156 Fall 200766 Technique: Role Playing Role playing allows developers to experience the user’s world from the user’s perspective Live storyboard or unscripted walkthrough Generate system activities and/or screen mockups as you go along

67. 11 September 2007Kaiser: COMS W4156 Fall 200767 Technique: Requirements Workshop Perhaps the most powerful technique for eliciting requirements Gather all key stakeholders together for a short but intensely focused period Use an outside facilitator experienced in requirements management May combine interviewing, brainstorming, storyboards, role playing

68. 11 September 2007Kaiser: COMS W4156 Fall 200768 Technique: Prototyping Partial implementation of a software system, built to help developers, users and customers better understand system requirements Focus on the “fuzzy” requirements: poorly defined and/or poorly understood

69. 11 September 2007Kaiser: COMS W4156 Fall 200769 Reminder: Second IDA Due Next Week! Tuesday 18 September, 10am Assignments posted on course websitecourse website Submit via CourseWorksCourseWorks Individual Development Assignment #2

70. 11 September 2007Kaiser: COMS W4156 Fall 200770 Upcoming Deadlines Teams announced September 18 th Team project concept due September 25 th Individual development assignment #3 due October 2 nd Revised project concept due October 9 th – first iteration begins

71. 11 September 2007Kaiser: COMS W4156 Fall 200771 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser Kaiser+4156@cs.columbia.edu http://york.cs.columbia.edu/classes/cs4156/