1. Putting the User in Usable Verification Kathi Fisler, WPI Joint work with Shriram Krishnamurthi

2. System  Properties  true or counter-eg Template Patterns More Domains GUI-based Tools Use English Better Scalability What is Usable Verification?

3. 3 3

4. Access-Control Policies request decision Developer Patient During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p SocNetwork User PC-Chair

5. Policy Verification System  Properties  true or counter-eg Assigned reviewers can submit reviews During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p

6. Policy Verification (EnvModel  System)  Properties  true or counter-eg During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p  Assigned reviewers can submit reviews

7. What effect did this edit have? Transfer confidence During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p During the review phase, a reviewer r may submit a review for paper p if r is not conflicted with p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p Artifact  Ground-truth artifact  Difference assigned(r,p), conflicted(r,p),... difference: permit vs deny

8. 8 Configuration checking Upgrade checkingUpgrade exploring Mutation testing Refactoring testing = ? “What if” questions

9. Those I trust more should have more permissions Others must find our settings reasonable The reviewing process needs to run smoothly People triangulate decisions against personal, subjective measures During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p

10. Those I trust more should have more permissions Artifact  Ground-truth artifact  User View  Difference MoreTrusted(chair, reviewer) MoreTrusted(reviewer, author)... MoreTrusted(R1,R2)   (act,res) : Permit(R1,act,res)  Deny(R2,act,res) During the review phase, a reviewer r may submit a review for paper p if r is assigned to review p During the meeting phase, a reviewer r can read the scores for paper p if r has submitted a review for p

11. System  Properties  true or counter-eg Artifact  Ground-truth artifact  Difference Artifact  Ground-truth artifact  User View  Difference (EnvModel  System)  Properties  true or counter-eg