1. High Performance Networking with the SSH Protocol Chris Rapier rapier@psc.edu Vancouver Joint Techs July 19, 2005

2. Pittsburgh Supercomputing Center Defining the Problem Transferring data over insecure networks requires cryptographically secure authentication –Recent history has highlighted the need for this. RCP and FTP are no longer viable options It should be fast, easy to use, and cheap to install/maintain. Pick two. Sometimes only one.

3. Pittsburgh Supercomputing Center First Solution: GridFTP Pros –Cryptographically secure authentication –High transfer rates for bulk data Cons –Difficult to install and maintain –Key distribution requires additional infrastructure –Often out of reach of smaller organizations –Limited distribution

4. Pittsburgh Supercomputing Center Next Solution: Kerberos Pros –Relatively widely used protocol –Familiar interfaces (ftp, web, etc) –Fast transfer rates Cons –Maintenance and configuration beyond some organizations –Insecure data transfers

5. Pittsburgh Supercomputing Center Another Solution: SCP/SSH Pros –Strong cryptographic security of authentication and data –Easy to install and maintain –Ubiquitous Cons –Astoundingly slow –Somewhat less secure than kerberos or globus (no signed certs, expiring tokens)

6. Pittsburgh Supercomputing Center Using SCP/SSH Anyway In spite of faults SCP/SHH ends up being the default for most people. This can be a *bad* thing –Security depends on universal compliance. –People will ‘drift’ to using the easiest solution. SSH is the easiest *security* solution but the speed is frustrating. Some people will, eventually, resort to insecure but fast methods ‘just this once’ Improve SSH performance and the security environment as a whole is enhanced and we’ll have fast, cheap, secure - pick three. So why is SSH so slow?

7. Pittsburgh Supercomputing Center The Real Problem With SSH It is *NOT* the encryption process! –If it was: Faster computers would give faster throughput. Which doesn’t happen. Transfer rates would be constant in local and wide area network. Which they aren’t. In fact transfer rates seem dependent on RTT, the farther away the slower the transfer. Any time rates are strongly linked to RTT it implies a receive buffer problem

8. Pittsburgh Supercomputing Center SSH is RWIN Limited Analysis of the code reveals –SSH Protocol V2 is multiplexed Multiple channels over one TCP connection –Must implement a flow control mechanism per channel Essentially the same as the TCP receive window –This application level RWIN is effectively set to 64KB. So real connection RWIN is MIN(TCP rwin, SSH rwin ) Thus TPUT max = 64KB/RTT

9. Pittsburgh Supercomputing Center Solving the Problem Use getsockopt() to get TCP rwin and dynamically set SSH rwin –Performed several times throughout transfer to handle autotuning kernels Results in 10x to 50x faster throughput depending on cipher used on well tuned system.

10. Pittsburgh Supercomputing Center HPN-SSH v. SSH

11. Pittsburgh Supercomputing Center Advantages Speed is comparable to GridFTP and Kerberized FTP No need for separate key infrastructure or realm administration Provides authentication and data security Can be used with other applications such as rsync, svn, SFTP, ssh port forwarding & more

12. Pittsburgh Supercomputing Center What’s involved? Get the source code from www.openssh.org www.openssh.org Get the patch from www.psc.eduwww.psc.edu tar -zxf openssh-3.9p1.tgz cd openssh-3.9p1 patch < openssh-3.9p1-hpn.diff configure make install or make install-nokeys Time elapsed: < 5 minutes

13. Pittsburgh Supercomputing Center So what does that get you? Speed increase in direction of HPN –No need to have HPN-SSH on both sides Can set TCP rwin on the command line –To maximum buffer size allowed by system configuration None cipher re-enabled –Using mid stream cipher switching authentication is still secure.

14. Pittsburgh Supercomputing Center New SSH Tool - PMVPN The Poor Man’s VPN –Use ld_preload to load a custom library that captures all network open calls. If it matches a rule then tunnel the connection through SSH Requires ssh keys to be installed on remote host Transparent. Secure. Most things ‘just work’ –Caveat: FTP has to run in passive mode.

15. Pittsburgh Supercomputing Center That’s a Neat Trick Use PMVPN to secure ‘coffeehouse connections’ –Designate all outgoing connections to use ssh tunnels back to ‘home’ machine. –Every packet on wireless network is now encrypted without WEP or WAP Always use the same SMTP server –Connections to SMTP are intercepted and tunneled back to your network. Use as a secure proxy. –Web browsing can be effectively anonymized.

16. Pittsburgh Supercomputing Center It’s Not Perfect FTP must be in passive mode –Even then it fails if using a redirect Establishes a new connection each time –So browsing the web means a new SSH handshake for each new http connection Other applications may or may not work as expected - we’re still testing EVEN SO –Its easy to use, works reasonably well, and provides security here and now *without* additional infrastructure.

17. Pittsburgh Supercomputing Center Big Picture Ideas Impact of hpn-ssh work will obscure packet contents –Multiple apps using port 22, no way to determine which is doing what, payload inspection methods can be easily circumvented. Overall security of network environments will improve. –People will *invariably* circumvent security if its inconvenient - especially in academic environments. Make the lowest acceptable level of security equivalent to insecure methods in terms of ease of use and performance. Remember, most people will *not* strictly follow security guidelines unless they are pressured to or its easy. So make it easy.

18. Pittsburgh Supercomputing Center Availability Source: http://www.openssh.org/ http://www.openssh.org/ Patch: http://www.psc.edu/networking/projects/hpn- ssh/ http://www.psc.edu/networking/projects/hpn- ssh/ Other: FreeBSD ports security/hpn-ssh/

19. Pittsburgh Supercomputing Center Thanks! Questions? Business cards with URL available