Image Magick in the Cloud Scalable Image Processing Service Sanketh Beerabbi University of Central Florida

Introduction Image processing is a very compute intensive task which requires multiple compute units to provide near instantaneous results. It’s more challenging in online applications involving many simultaneous, “impatient” users creating varying workloads. Cloud computing provides the scalability needed for running such applications online.

Service Allows users to upload photos and store them in the cloud. Allows photos to be manipulated by applying Instagram style image filters. Allows users to create profile pics by detecting faces and cropping them out of the original photo.

Implementation Amazon 64 bit Linux AMI AWS PHP SDK with cURL and PEAR packages GD and Image Magick for image manipulation AWS services: EC2 & S3 SimpleDB (SDB) Simple Queue Service (SQS) Elastic Load Balancer (ELB) Cloudfront CDN

What took time Plan and Design (~2 weeks) LAMP, AWS PHP SDK, GD and ImageMagick (~2 weeks) Browser based uploading to S3 and server side operations using S3 API (~2 weeks) Database operations using Simple DB(1 week) Message passing using SQS API (~2 weeks)

Architecture Simple DB AWS CLOUD

Load Balancing & Redundancy

Modules Upload – Stores file names and task status in SDB Creates bucket and uploads image to S3 Sends message to SQS Process – Reads details from SDB and updates task status. Receives and deletes message from SQS Downloads from S3 and processes image . Uploads processed image to S3.

Message Queing A single Amazon SQS queue can be shared by multiple instances simultaneously. An instance that is processing a message can prevent other instances from processing the same message at the same time by temporarily “locking” a message. When the instance is done processing the message, it deletes the message. If the instance fails while processing the message, another instance can get the message after the lockout period. The number of processing servers can be changed dynamically to handle varying load.

What I Learnt PHP, hundreds of new *NIX commands, tools and Linux from the “terminal” point of view. Virtual networking, load balancing and DNS. Batch processing on the cloud using multiple instances Inter-instance communication using message queuing.

DEMO Main Website: http://imagecloud.vf1.us/ AWS Public address: http://ec2-54-204-45-246.compute- 1.amazonaws.com/ Cloudfront CDN: http://d3emazpmic2hkb.cloudfront.net/

Questions?