1. 1 Working with Queues Presented to KC PUG: Kansas City's PHP User Group 2006-08-19 Daniel Holmes

2. 2 Today's Discussion ● Queue Basics ● What they are great for ● How to build and manage your own ● Not very Code Heavy, but if interested let me know.

3. 3 Queue Basics So, what is a queue? – A queue is a list that keeps everything from happening all at once. – Usually handled in the order received (FIFO), but certainly not required

4. 4 Queue Basics Our lives are filled with Queues: – Unread E-mail * – Todo lists * – Grocery Lists * – Waiting for a table at Zio's – Buzzing Coasters – Standing in line for that morning coffee * We sometimes randomly processes these queues. E-mails from our boss, skipping ahead to bread when in the bread isle; etc.

5. 5 What is a Queue in Software Development? – A queue is a list that keeps everything from happening all at once. – Usually handled in the order received (FIFO), but certainly not required – May be as simple as an array (like the lists in your head) – Or stored in a database – And even accessed by many servers at the same time!

6. 6 Why use a queue? ● Great for keeping tabs on what is not yet done. ● Scalability / Distributes work – Smooth out those processing peaks – Possibility of Multiple Processing Servers ● Not everything can be done in the 2 seconds your users are willing to wait.

7. 7 PHP Queue Examples ● Stack Functions – Push() – Pop() ● Queue Functions – shift() – unshift()

8. 8 Persistent Queue Examples ● Ticket Systems – Helpdesk/Bugtracking – Plumber dispatching system ● Background or “evening” Processing – Workflow systems – Reports Generation ● Distributed Processing

9. 9 More advanced Queue Examples ● Event Brokering – Letting systems know of changes in another ● Communication with Other Systems – Credit card validation – Printing – E-mail distribution (notification e-mail, listserves, etc)

10. 10 Queue Similarities ● An item source ● A central, organized list of items ● A unique id for each item ● A way to select the next item to work on ● Updates and status changes Often the who, what and when of each item gets tracked as well

11. 11 Ticketing Systems ● A user fills out a form on a website to state a problem or a claim ● One or two employees then monitors the queues and takes action ● You track – A unique Ticket ID – Who submitted the issue – The details of the issue – When the issue was opened, closed, last updated, user notified, etc.

12. 12 Ticketing Systems ● Source ● Queue ● Worklist ● Work Item ● Work Item Update

13. 13 Background Processing ● A user fills out a form on a website to run a rather lengthy report ● Upon completion, the user receives an email saying their report is ready to be retrieved ● The report is provided from the supplied link and username

14. 14 Background Processing ● You Track – A unique ID – Who (user) submitted the request – The report to run – Parameters to use in the report – Status – Date Generated – Location of the file on the server – Unique hash to identify the file for pickup

15. 15 Background Processing 1. User Submits the request 2. Request is added to the queue 3. Every minute, the reports server runs “genreports.php” Pull the next ticket in READY status, and update status to PENDING. Run the specified report, updates the queue, and notified the user.

16. 16 Evening Processing ● Start with the same reports process. ● Set “genreports.php” to only run once a day at say 3:00am. ● You may want to alter genreports.php to digest the notifications (1 e-mail per user, per day) Proper table locking and “Item Claiming” is essential! Watch those race-conditions!

17. 17 Distributed Processing ● A year goes by, and your application is so successful it has 1,000's of users. ● Picture the same reports scenario, but with lots and lots of users. ● Basically the same method as before, but with multiple reports servers checking the same queue

18. 18 Advanced Queue Examples How might you go about building the following: ● Event Brokering – Letting systems know of changes in another ● Communication with Other Systems – Credit card validation – Printing – E-mail distribution (notification e-mail, listserves, etc)

19. 19 Working with Queues ● We often build queues without giving them the thoughtfulness and design that we should ● Don't be afraid to check out what's already written ● Even look to other languages for ideas and software. ● Your PHP script submits to a Java based workflow queue, etc.

20. 20 Questions? Daniel Holmes daniel.holmes@gmail.com