Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense1 OPRAM: An Online System for Assigning Capstone Course Students to Sponsored Projects Juedong Zhang Acknowledgments: NSF Award RI

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense2 Outline  Introduction Background Motivations Contributions  Problem Modeling & Solving Methods  System Implementation & Features  Conclusions & Future Work

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense3 Background  CSE Capstone Course An integral part of the undergraduate curriculum Professional development opportunities  Participants Students, sponsors, CSE faculty, & iLab staff  Team formation

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense4 Motivations  Team assignment is challenging Large class (CS + CE) Various constraints Capacity limit for each project Students’ preferences Sponsors’ preferences  Constraint-based system for decision support Automatic solver generates solutions upon request Interactive solver maintains consistency for user Visual support guides user to make good assignments

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense5 Contributions  Modeled the task as a Constrained Optimization Problem (COP) with three optimization criteria  Designed a search algorithm that can generate the optimal solution in a reasonable amount of time  Developed an interactive solver that maintains consistency while the user explores the search space  Created a web GUI interface that encapsulates both solvers and provides an unified user experience  Co-built an MySQL database for data persistence

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense6 Outline  Introduction  Problem Modeling & Solving Methods Problem modeling Problem solving methods Evaluation of search performance  System Implementation & Features  Conclusions & Future Work

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense7 Modeling: COP  Given Variables: The students enrolled in the Capstone Course Domains: The projects applied to which a student applies Hard constraints: A project’s capacity limit A student does not choose a project or a sponsor does not choose a student Soft constraints: Students/sponsors preferences between 0…5 Objective function: Three criteria based on preferences  Query: find a value for each variable such that All hard constraints are satisfied and The objective function is optimized

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense8 Modeling: Soft Constraints  Students preferences “I am interested in working on this project.”  Sponsors preferences “I think this student is a good fit for my project.” Agreeableness of the statement a preference score

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense9 Modeling: Combining Preferences  Product of preferences is more sensitive

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense10 Modeling: Objective Function  Avg Maximize the average value over all preferences  Geo Avg Maximize the geometric average value over all preferences  Maxmin Maximize the minimum value over all preferences

Constraint Systems Laboratory Problem Solving Method  Backtrack search  Variable Ordering First choose the student with the least number of projects (i.e., least-domain first)  Value Ordering Assign to a student first the project with the highest preference (i.e., most promising value)  Constraint propagation by local consistency  Exhaustive, depth-first search  Branch-and-bound with a heuristic function 11/26/2015Zhang: MS Project Defense11

Constraint Systems Laboratory Local Consistency  Node consistency Apply to the zero preference constraint Remove the 0 value from the domain of every variable  Generalized arc consistency (GAC) Apply to the capacity constraint Remove the value (project) from the domain of all future variables when the load of that project is reached 11/26/2015Zhang: MS Project Defense12 GAC is the foundation of our interactive solver

Constraint Systems Laboratory Heuristic Function h  Given an assignment of i variables The cost of the partial solution The estimated cost of remaining assignments The total estimated cost  h(.) Choose the value with highest preferences, even if values are inconsistent with partial solution h(.) is admissible, it never underestimates the cost of the real solution 11/26/2015Zhang: MS Project Defense13

Constraint Systems Laboratory Search: FC-BnB search algorithm  Forward checking (FC) algorithm [Haralick and Elliott, 1980] A systematic search technique After a variable is instantiated, it looks ahead (executes GAC) When detecting a domain wipeout, it backtracks chronologically  Branch-and-Bound Finds a first solution quickly, the incumbent After a variable is instantiated, compares the quality of current solution to that of the incumbent, updates incumbent Exhaustive search, linear in space 11/26/2015Zhang: MS Project Defense14 FC- BnB is the foundation of our automatic solver

Constraint Systems Laboratory {(P 2,5), (P 1,3), (P 3,1)} {(P 3,5), (P 2,3), (P 1,3)} {(P 2,5)} {(P 2,5), (P 1,3)} {(P 3,3), (P 1,1)} VariableDomain Current Variable If Avg optimization criterion is used f ((V 1,5), (V 2,5), (V 3,3), (V 4,5), (V 5,5), (V 6,)) = g ((V 1,5), (V 2,5)) + h ((V 3,3), (V 4,5), (V 5,5), (V 6,3)) = ((5 + 5) + ( )) / 6 = 4.33 Estimated solution quality at V 2. Proceed to V 3 only if it is better than incumbent Future Variables V1V1 V2V2 V3V3 V4V4 V5V5 V6V6 (P 2,5) (P 3,3)(P 1,5)(P 3,5)(P 1,3) The h function ignores hard constraints Assignments made by FC V1V1 V2V2 V3V3 V4V4 V5V5 V6V6 Illustrating Search: Example {(P 1,5), (P 3,3)} Projects removed by FC to comply with the capacity constraints Six students V 1 ~ V 6 Three projects P 1 ~ P 3, each with a capacity of 2 11/26/2015Zhang: MS Project Defense15

Constraint Systems Laboratory Evaluating search: CPU  Data Sets Fall 2013: 45 students, 11 projects Spring 2014: 10 students, 3 projects  Impact of the h function 11/26/2015Zhang: MS Project Defense16

Constraint Systems Laboratory Evaluating Search: Solution Quality  Impact of the optimization criterion: comparing optimal solutions 11/26/2015Zhang: MS Project Defense17

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense18 Outline  Introduction  Problem Modeling & Solving Methods  System Implementation & Features System architecture iLab infrastructure OPRAM architecture OPRAM Functionalities Web GUI The Legend’s color map Interactive solver Automatic solver Solution comparison  Conclusions & Future Work

Constraint Systems Laboratory 11/26/2015Zhang: MS Project defense19 iLab Infrastructure A user’s view User Registration Information Exchange Project Assignment Students / Sponsors Progress Management Instructors / iLab Staff iLab Administrator Module we developed Module in progress

Constraint Systems Laboratory 11/26/2015Zhang: MS Project defense20 iLab Infrastructure A system view User Registration Information Exchange iLab DB Progress Management iLab Info Project OPRAM Project Assignment iLab X Project Module we developed Module we co-developed Module in progress

Constraint Systems Laboratory OPRAM Architecture  Google Web Toolkit 11/26/2015Zhang: MS Project Defense21 Server (Backend) Client (GUI) RPC Google Web Toolkit Application Java EEAJAX Runs in Browser Runs on Server

Constraint Systems Laboratory OPRAM Architecture  Remote Procedure Call (RPC) 11/26/2015Zhang: MS Project Defense22

Constraint Systems Laboratory OPRAM Architecture  Software design 11/26/2015Zhang: MS Project Defense23 RPC GUI Service Client Impl. Service Async. Service Service Server Impl. Client Server DB Access Auto Solver Interactive Solver Request Reply Interface Implementation Color legend :

Constraint Systems Laboratory Web GUI 11/26/2015Zhang: MS Project Defense24

Constraint Systems Laboratory Legend’s Color Map 11/26/2015Zhang: MS Project Defense25

Constraint Systems Laboratory Interactive Solver: Student View 11/26/2015Zhang: MS Project Defense26 Project not available Total number of students 1-semester or 2-semester student Preference

Constraint Systems Laboratory Interactive Solver: Project View 11/26/2015Zhang: MS Project Defense27 Project overview sub-panelIndividual project sub-panel Project not available Total number of projects How many more students can be assigned Average preference Individual project tab Preference Display the list of projects applied by the student when mouseover the preference Project the student currently assigned to is shown in bold

Constraint Systems Laboratory Automatic Solver 11/26/2015Zhang: MS Project Defense28 Three options: 10 seconds 1 minute 5 minutes

Constraint Systems Laboratory Solution Comparison 11/26/2015Zhang: MS Project Defense29 Only comparable solutions are displayed Three additional metrics for current solution listed

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense30 Outline  Introduction  Problem Modeling & Solving Methods  System Implementation & Features  Conclusions & Future Work

Constraint Systems Laboratory Conclusions  Designed & implemented a web-base system for Capstone Course project assignment decision support  OPRAM demonstrated significant improvements on time to solution as well as the quality of assignment found  CSE faculty and iLab staff are satisfied with OPRAM’s capabilities and have decided to take advantage of it in the upcoming semester 11/26/2015Zhang: MS Project Defense31

Constraint Systems Laboratory Future Work  A formal proof for NP-hardness of this assignment problem  Explore other optimization criteria such as “stable matching”  Explore whether or not ‘a minimum size per project’ is appropriate  Support further interoperability between our two solvers  Conduct longitudinal study of usefulness & flexibility 11/26/2015Zhang: MS Project Defense32

Constraint Systems Laboratory 11/26/2015Zhang: MS Project Defense33 Thank you I welcome your questions

Constraint Systems Laboratory Ordering Heuristics  Dynamic variable ordering heuristic First choose the student with the least number of projects (i.e., least-domain first) Choose the most constrained variable first (fail first principle) to reduce the branching factor  Dynamic value ordering heuristic For a student, first assign the project with the highest preference Choose the most promising value first to find the best solution more quickly 11/26/2015Zhang: MS Project Defense34