1. Proxy-Based Image Verification
Thomas Kotwal
MIT Laboratory for Computer Science
January 9, 2002

2. System Layout Bob has access to an untrustworthy computer.
He wishes to communicate with his proxy.
He can trust what he sees on the screen, because his camera equipped K21 is monitoring the screen content.

3. Proxy-Based Approach
Add an IR link between the K21 and untrustworthy computer
This allows the K21 to send data (image) to the proxy
Proxy can do large amounts of computation
Proxy knows what the screen should display

4. Security Protocol U.C. displays an image and the K21 takes a picture
Proxy verifies the picture and sends nonce + MAC to K21 via U.C. using IR link
K21 sends picture + nonce + MAC to proxy via U.C. using IR link
U.C. displays an image and the K21 takes a picture

5. Performing a Secure Request
Proxy verifies the picture and sends nonce + MAC to the K21 using the IR link
K21 sends picture + nonce + MAC to the proxy using the IR link
U.C. displays the image and the K21 takes a picture of it
User enters a request
U.C. sends the request to the proxy
Proxy sends an image summarizing the request to the U.C.

6. Image Verification Problem
Is this…
…the same as this?
Obstacles:
Linear and non-linear distortions
Decreased resolution
Noise
Steps to solution
Undo distortions
Compare content

7. Step 1: Undo Image Distortion
Undo lens distortion
Model as radially symmetric distortion
Non-linear process
Undo linear distortions
Corrects for affine (scaling, rotation, translation) and projection distortion
Requires four known points in distorted image
Undo other non-linear distortions
Corrects for curvature of screen, etc.
Least-squares with orthogonal polynomials
May not be necessary

8. a) Radially Symmetric Distortion
(x,y)
(u,v) r
k : constant

9. b) Linear Distortion
(u,v)
(x,y)

10. Step 2: Compare Content Assume content is text only
Perform OCR on processed image
Advantage: proxy knows what the text should say
To save computation time compare each character with what it should be, not every possible character
Constrain font to facilitate OCR routine

11. Next Step: Compare Images
Generalize content to be anything, not just text
Find areas of large difference between original and processed image
Requires very accurate undistortion

12. Summary of Approach
K21 takes a picture and uses its IR link to send the picture to the proxy
Proxy corrects for image distortion and performs OCR
SELL: IBM
300 SHARES
TOTAL:
$300
Proxy sends a confirmation back to the K21

13. Why Proxy-based Approach?
Advantages
Proxy knows what the screen should display
No image processing on the K21
No calibration necessary
Camera does not have to be immobile during session
Translates easily to color
Disadvantages
Requires IR link between K21 and untrustworthy computer
Possibly long verification time
Image verification allows for possibility of error

14. Summary / Status Q: How do you use a computer that you don’t trust?
Our Answer: Use visual authentication
Pixel-Matching solution is running
Proxy-based solution is under development

15. Questions?