1. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense1 OPRAM: An Online System for Assigning Capstone Course Students to Sponsored Projects Juedong Zhang Acknowledgments: NSF Award RI-111795

2. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense2 Outline  Introduction Background Motivations Contributions  Problem Modeling & Solving Methods  System Implementation & Features  Conclusions & Future Work

3. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense3 Background  CSE Capstone Course An integral part of the undergraduate curriculum Professional development opportunities  Participants Students, sponsors, CSE faculty, & iLab staff  Team formation

4. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense4 Motivations  Team assignment is challenging Large class (CS + CE) Various constraints Capacity limit for each project Students’ preferences Sponsors’ preferences  Constraint-based system for decision support Automatic solver generates solutions upon request Interactive solver maintains consistency for user Visual support guides user to make good assignments

5. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense5 Contributions  Modeled the task as a Constrained Optimization Problem (COP) with three optimization criteria  Designed a search algorithm that can generate the optimal solution in a reasonable amount of time  Developed an interactive solver that maintains consistency while the user explores the search space  Created a web GUI interface that encapsulates both solvers and provides an unified user experience  Co-built an MySQL database for data persistence

6. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense6 Outline  Introduction  Problem Modeling & Solving Methods Problem modeling Problem solving methods Evaluation of search performance  System Implementation & Features  Conclusions & Future Work

7. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense7 Modeling: COP  Given Variables: The students enrolled in the Capstone Course Domains: The projects applied to which a student applies Hard constraints: A project’s capacity limit A student does not choose a project or a sponsor does not choose a student Soft constraints: Students/sponsors preferences between 0…5 Objective function: Three criteria based on preferences  Query: find a value for each variable such that All hard constraints are satisfied and The objective function is optimized

8. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense8 Modeling: Soft Constraints  Students preferences “I am interested in working on this project.”  Sponsors preferences “I think this student is a good fit for my project.” Agreeableness of the statement a preference score

9. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense9 Modeling: Combining Preferences  Product of preferences is more sensitive

10. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense10 Modeling: Objective Function  Avg Maximize the average value over all preferences  Geo Avg Maximize the geometric average value over all preferences  Maxmin Maximize the minimum value over all preferences

11. Constraint Systems Laboratory Problem Solving Method  Backtrack search  Variable Ordering First choose the student with the least number of projects (i.e., least-domain first)  Value Ordering Assign to a student first the project with the highest preference (i.e., most promising value)  Constraint propagation by local consistency  Exhaustive, depth-first search  Branch-and-bound with a heuristic function 11/26/2015Zhang: MS Project Defense11

12. Constraint Systems Laboratory Local Consistency  Node consistency Apply to the zero preference constraint Remove the 0 value from the domain of every variable  Generalized arc consistency (GAC) Apply to the capacity constraint Remove the value (project) from the domain of all future variables when the load of that project is reached 11/26/2015Zhang: MS Project Defense12 GAC is the foundation of our interactive solver

13. Constraint Systems Laboratory Heuristic Function h  Given an assignment of i variables The cost of the partial solution The estimated cost of remaining assignments The total estimated cost  h(.) Choose the value with highest preferences, even if values are inconsistent with partial solution h(.) is admissible, it never underestimates the cost of the real solution 11/26/2015Zhang: MS Project Defense13

14. Constraint Systems Laboratory Search: FC-BnB search algorithm  Forward checking (FC) algorithm [Haralick and Elliott, 1980] A systematic search technique After a variable is instantiated, it looks ahead (executes GAC) When detecting a domain wipeout, it backtracks chronologically  Branch-and-Bound Finds a first solution quickly, the incumbent After a variable is instantiated, compares the quality of current solution to that of the incumbent, updates incumbent Exhaustive search, linear in space 11/26/2015Zhang: MS Project Defense14 FC- BnB is the foundation of our automatic solver

15. Constraint Systems Laboratory {(P 2,5), (P 1,3), (P 3,1)} {(P 3,5), (P 2,3), (P 1,3)} {(P 2,5)} {(P 2,5), (P 1,3)} {(P 3,3), (P 1,1)} VariableDomain Current Variable If Avg optimization criterion is used f ((V 1,5), (V 2,5), (V 3,3), (V 4,5), (V 5,5), (V 6,)) = g ((V 1,5), (V 2,5)) + h ((V 3,3), (V 4,5), (V 5,5), (V 6,3)) = ((5 + 5) + (3 + 5 + 5 + 3)) / 6 = 4.33 Estimated solution quality at V 2. Proceed to V 3 only if it is better than incumbent Future Variables V1V1 V2V2 V3V3 V4V4 V5V5 V6V6 (P 2,5) (P 3,3)(P 1,5)(P 3,5)(P 1,3) The h function ignores hard constraints Assignments made by FC V1V1 V2V2 V3V3 V4V4 V5V5 V6V6 Illustrating Search: Example {(P 1,5), (P 3,3)} Projects removed by FC to comply with the capacity constraints Six students V 1 ~ V 6 Three projects P 1 ~ P 3, each with a capacity of 2 11/26/2015Zhang: MS Project Defense15

16. Constraint Systems Laboratory Evaluating search: CPU  Data Sets Fall 2013: 45 students, 11 projects Spring 2014: 10 students, 3 projects  Impact of the h function 11/26/2015Zhang: MS Project Defense16

17. Constraint Systems Laboratory Evaluating Search: Solution Quality  Impact of the optimization criterion: comparing optimal solutions 11/26/2015Zhang: MS Project Defense17

18. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense18 Outline  Introduction  Problem Modeling & Solving Methods  System Implementation & Features System architecture iLab infrastructure OPRAM architecture OPRAM Functionalities Web GUI The Legend’s color map Interactive solver Automatic solver Solution comparison  Conclusions & Future Work

19. Constraint Systems Laboratory 11/26/2015Zhang: MS Project defense19 iLab Infrastructure A user’s view User Registration Information Exchange Project Assignment Students / Sponsors Progress Management Instructors / iLab Staff iLab Administrator Module we developed Module in progress

20. Constraint Systems Laboratory 11/26/2015Zhang: MS Project defense20 iLab Infrastructure A system view User Registration Information Exchange iLab DB Progress Management iLab Info Project OPRAM Project Assignment iLab X Project Module we developed Module we co-developed Module in progress

21. Constraint Systems Laboratory OPRAM Architecture  Google Web Toolkit 11/26/2015Zhang: MS Project Defense21 Server (Backend) Client (GUI) RPC Google Web Toolkit Application Java EEAJAX Runs in Browser Runs on Server

22. Constraint Systems Laboratory OPRAM Architecture  Remote Procedure Call (RPC) 11/26/2015Zhang: MS Project Defense22

23. Constraint Systems Laboratory OPRAM Architecture  Software design 11/26/2015Zhang: MS Project Defense23 RPC GUI Service Client Impl. Service Async. Service Service Server Impl. Client Server DB Access Auto Solver Interactive Solver Request Reply Interface Implementation Color legend :

24. Constraint Systems Laboratory Web GUI 11/26/2015Zhang: MS Project Defense24

25. Constraint Systems Laboratory Legend’s Color Map 11/26/2015Zhang: MS Project Defense25

26. Constraint Systems Laboratory Interactive Solver: Student View 11/26/2015Zhang: MS Project Defense26 Project not available Total number of students 1-semester or 2-semester student Preference

27. Constraint Systems Laboratory Interactive Solver: Project View 11/26/2015Zhang: MS Project Defense27 Project overview sub-panelIndividual project sub-panel Project not available Total number of projects How many more students can be assigned Average preference Individual project tab Preference Display the list of projects applied by the student when mouseover the preference Project the student currently assigned to is shown in bold

28. Constraint Systems Laboratory Automatic Solver 11/26/2015Zhang: MS Project Defense28 Three options: 10 seconds 1 minute 5 minutes

29. Constraint Systems Laboratory Solution Comparison 11/26/2015Zhang: MS Project Defense29 Only comparable solutions are displayed Three additional metrics for current solution listed

30. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense30 Outline  Introduction  Problem Modeling & Solving Methods  System Implementation & Features  Conclusions & Future Work

31. Constraint Systems Laboratory Conclusions  Designed & implemented a web-base system for Capstone Course project assignment decision support  OPRAM demonstrated significant improvements on time to solution as well as the quality of assignment found  CSE faculty and iLab staff are satisfied with OPRAM’s capabilities and have decided to take advantage of it in the upcoming semester 11/26/2015Zhang: MS Project Defense31

32. Constraint Systems Laboratory Future Work  A formal proof for NP-hardness of this assignment problem  Explore other optimization criteria such as “stable matching”  Explore whether or not ‘a minimum size per project’ is appropriate  Support further interoperability between our two solvers  Conduct longitudinal study of usefulness & flexibility 11/26/2015Zhang: MS Project Defense32

33. Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense33 Thank you I welcome your questions

34. Constraint Systems Laboratory Ordering Heuristics  Dynamic variable ordering heuristic First choose the student with the least number of projects (i.e., least-domain first) Choose the most constrained variable first (fail first principle) to reduce the branching factor  Dynamic value ordering heuristic For a student, first assign the project with the highest preference Choose the most promising value first to find the best solution more quickly 11/26/2015Zhang: MS Project Defense34