Flash An efficient and portable Web server

Today’s paper, FLASH Quite old (1999) Reading old papers gives us lessons We can see which solution among all proposed solutions is more widely adopted We can understand how HW environment affects SW solutions, by comparing old days and now days

SPED Web servers in 1999 Logical flow of a typical Web server Single-process event-driven (SPED) Excellent performance for in-memory cached files Poor performance for files on disks File I/O blocks the CPU

MT / MP approach Multi-process (MP) or multi-threaded (MT) approach Spawn multiple SPED instances to serve HTTP requests One can work for a new request when the others are blocked by file I/O

FLASH Asymmetric Multi-Process Event Driven (AMPED) approach Disk I/O is done by separated helper processes (or threads) Helper processes communicate with main process via IPC (e.g. pipe)

AMPED vs MT/MP Memory: AMPED << MT/MP MT/MP approach creates a thread/process for each connection Concurrency: AMPED > MT/MP MT/MP’s maximum concurrency is limited by Kernel’s process limit AMPED’s helper process can be shared by multiple connections

Limitations of AMPED IPC overhead IPC needs assist of Kernel, which introduces unnecessary mode switching overhead Context switching overhead Running multiple processes/threads incurs frequent context switching (Actually, AMPED is better than MT/MP but not enough)

Background of design decisions 16 yeas ago (1999), No AIO  Spawn a processes only for disk I/O CPU was relatively faster than network interface  IPC overhead or context switching overhead were not critical Memory was expensive  AMPED requires much less memory compare to MT/MP Familiar with single-CPU core environment  No multi-core consideration Evaluation setupFLASH (1999)mTCP (2014)Difference # of CPU cores116x16 CPU clock speed333 MHz2.9 GHzx8.7 Memory128 MB32 GBx256 Network BW200 Mbps10 Gbpsx50

Asynchronous File I/O Non-blocking file I/O Application only requests disk read Kernel notifies completion of disk read to application Enables completely non-blocking single-threaded programming but increases programming difficulty First presented in 1997, (As far as I know) Proactor: An Object Behavioral Pattern for Demultiplexing and Dispatching Handlers for Asynchronous Events 4 th annual Pattern Languages of Programming conference

Nowadays Status AIO is introduced (From 2003, Linux kernel 2.6.x)  Single thread can manage everything without blocking  No IPC overhead, no context switching overhead CPU is bottleneck Memory is cheap Multi-core scalability is very important AMPED vs AIO? Most popular Web servers still stick on MP/MT! Apache: MT, Nginx: MP (Both Apache and Nginx support AIO, but disabled by default)