1. Scheduling with uncertain resources Elicitation of additional data Ulaş Bardak, Eugene Fink, Chris Martens, and Jaime Carbonell Carnegie Mellon University

2. Problem Scheduling a conference under uncertainty Uncertain room properties Uncertain equipment needs Uncertain speaker preferences The automated scheduler needs to collaborate with the human user.

3. Problem The system may not have enough data for producing a good schedule The user may be able to obtain some of the missing data, but not all data The system should identify critical missing data and ask the user only for these data.

4. Missing info: Invited talk: – Projector need Poster session: – Room size – Projector need Assumptions: Invited talk: – Needs a projector Poster session: – Small room is OK – Needs no projector Initial schedule Available rooms: Room num. Size Proj- ector 123123 200 100 80 Yes No Yes Events and constraints: Invited talk, 9–10am: Needs big room Poster session, 9–11am: Needs a room 1 2 3 Initial schedule: Talk Posters

5. Choice of questions Candidate questions: Invited talk: Needs a projector? Poster session: Needs a larger room? Needs a projector? Events and constraints: Invited talk, 9–10am: Needs a large room Poster session, 9–11am: Needs a room Useless info: There are no large rooms w/o a projector × Useless info: There are no unoccupied larger rooms × Potentially useful info √ 1 2 3 Initial schedule: Talk Posters

6. Improved schedule Events and constraints: Invited talk, 9–10am: Needs a large room Poster session, 9–11am: Needs a room System: Does the poster session need a projector? User: A projector may be useful, but not really necessary. Posters 1 New schedule: Talk 2 3 1 2 3 Initial schedule: Talk Posters Info elicitation:

7. Architecture Info elicitorParserOptimizer Process new info Update the schedule Choose questions Top-level control and learning Graphical user interface Administrator

8. Choice of questions For each candidate question, estimate the probabilities of possible answers For each question, compute its expected impact on the schedule quality, and select questions with large expected impacts For each possible answer, compute the respective change of the schedule quality

9. Experiments Scheduling of a large conference 14 available rooms 84 conference sessions 700 uncertain variables Manual Scheduling 0.61 Auto w/o Elicitation 0.68 Auto with Elicitation 0.72 Schedule Quality

10. 0.68 0.72 Schedule Quality 1030204050 Number of Questions 0.50 Experiments 0 actual estimated optimal schedule

11. Extensions Game-tree search for the most important questions Fast heuristics for pruning unimportant questions Learning new strategies for question selection

12. Conclusions We have developed a system that analyzes the importance of missing data, identifies critical uncertainties, and asks the user to obtain related additional data. It usually finds a near-optimal solution after asking 2% to 6% of all potential questions. The developed technique does not rely on specific properties of scheduling tasks, and it is applicable to a variety of problems that involve optimization under uncertainty.