Debugging with Fiddler Eric Lawrence @ericlaw

Let’s talk about you…

How did I end up here? Did I imagine that I’d grow up to have a network protocol as my license plate?

In a word, no. But after a decade of obsession with astronautics, I realized I was only really interested in technology, and specifically software.

Once upon a time…

Oh no! What happened?

There must be a better way…

A simple idea takes shape… All problems in computer science can be solved by another level of indirection. - David Wheeler

Only two problems Don’t know HTTP Don’t know C#

Fiddler: Evolution Eleven years, ~35k lines of C#, 160+ release builds, one full-length paperback, a cross-country move to Telerik, and two new supported platforms later…

New Website New Documentation New Platforms Enhanced User-Interface

My current side-project

A quick tour around Fiddler…

UI Evolution – The Web Sessions List

Fiddler on Linux (Mint/Ubuntu) Limitations: No script engine. No WebView. No automatic proxy chaining. UI very glitchy on Mac.

Fiddler on Mac OSX It works, but due to UI glitches, you’re usually better off using VirtualBox / Parallels / Fusion Limitations: No script engine. No WebView. No automatic proxy chaining. UI very glitchy on Mac.

Traffic Monitoring

Typical Architecture

Fiddler Windows/Linux Debug Across Devices PC Mac iOS Fiddler Windows/Linux Internet Tablets Phones

Fiddler as a Reverse Proxy http://fiddler2.com/r/?reverseproxy http://fiddler2.com/r/?reverseproxy

Win8/8.1 “Immersive” Apps & IE11 AppContainer blocks “loopback” network connections. For debugging purposes, you can disable that blocking. Ctrl+Click to exempt all AppContainers http://blogs.msdn.com/b/fiddler/archive/2011/12/10/fiddler-windows-8-apps-enable-loopback-network-isolation-exemption.aspx

.NET Applications YourApp.exe.config or machine.config <configuration>   <system.net>     <defaultProxy>       <proxy bypassonlocal="false" usesystemdefault=“false" proxyaddress= "http://127.0.0.1:8888" />     </defaultProxy>   </system.net> </configuration> http://www.fiddler2.com/Fiddler/help/hookup.asp

node.js Different libraries offer different approaches… var http = require('http'); var options = { host: '127.0.0.1', port: 8888, path: 'https://bayden.com/echo.aspx', headers: { Host: "bayden.com“ }, method: 'POST' }; var req = http.request(options, function(res) { console.log('STATUS: ' + res.statusCode + ‘ HEADERS: ' + JSON.stringify(res.headers)); res.setEncoding('utf8'); res.on('data', function (chunk) { console.log('BODY: ' + chunk); }); }); req.write(‘Post Data\n'); req.end(); http://www.fiddler2.com/Fiddler/help/hookup.asp

Protocols

HTTPS Traffic Decryption For security reasons, proxies cannot normally “see” HTTPS requests. To enable traffic decryption, Fiddler performs a “man-in-the-middle” attack. Decrypting CONNECT tunnel to www.fiddler2.com GET /fiddler2/ GET /Fiddler2/Fiddler.css http://fiddler2.com/fiddler/help/httpsdecryption.asp Export to Desktop for Firefox Added "fiddler.network.https.NoDecryptionHosts" list and UI. Cleanup certificate store GET /Fiddler/images/FiddlerLogo.png

HTML5 WebSockets WebSockets enable bi-directional socket communications over a connection established using HTTP or HTTPS. http://websocketstest.com/

FTP Fiddler supports FTP traffic via a built-in FTP gateway. FTP proxy is off-by-default. SPDY / HTTP2 Fiddler recognizes and tags SPDY connections if HTTPS-decryption is disabled.

SPDY / HTTP2 Fiddler cannot support SPDY until .NET’s SslStream supports ALPN. Please vote for my bug on CONNECT: https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=812003 Also, please vote for this other SslStream bug: https://connect.microsoft.com/VisualStudio/feedback/details/811998/system-net-security-sslstream-calls-localcertificateselection-callback-unconditionally-even-if-server-never-sends-certificaterequest-tls-message

Protocol Violations prefs set fiddler.lint.HTTP True

Store & Load Traffic

Output Formats Fiddler Session Archive Copy to the clipboard Visual Studio .WebTest Store as a plaintext file HTML5 AppCache Manifest Extract binary response bodies WCAT Load Test Archive to a database cURL Script HTTP Archive Format (HAR) Meddler Script

Or write your own… Fiddler’s Import/Export architecture http://fiddler2.com/documentation/Extend-Fiddler/ImporterExporterInterfaces

The SAZ file format Session Archive Zip files contain: Request and response bytes Timing and other metadata WebSocket messages HTML index file For security, SAZ files may be encrypted using AES http://fiddler.wikidot.com/saz-files

FiddlerCap – Simple captures http://www.fiddlercap.com FiddlerCap allows non-technical users to easily capture SAZ archives which can be emailed to developers or experts for analysis. FiddlerCap is available from http://www.fiddlercap.com. User-interface localized to: English | Français | Español | Português | 日本語 | русский

Import Formats HTTP Archive Format (HAR) Internet Explorer F12 Developer Tools (NETXML) Telerik Test Studio LoadTest Packet Capture (WireShark, tcpdump, NetMon) …or write your own

PCAP Import

Traffic Analysis

Convert text between popular web encodings. TextWizard Convert text between popular web encodings.

Traffic Comparison Use WinDiff or the differ of your choice to compare Sessions’ requests and responses.

Use the Differ Extension to compare groups of Sessions at once. Traffic Comparison Use the Differ Extension to compare groups of Sessions at once.

Filtering Traffic Selecting Traffic Using QuickExec Using Find Ignore Images & CONNECTs Application Type Filter Process Filter Troubleshooting with Help menu Selecting Traffic Using QuickExec Using Find

Regular Expression Support

SyntaxView Reformatting

ImageView DataURL Support

ImageView Tools Integration http://fiddler2.com/blog/blog/2013/06/04/what-s-new-in-fiddler-2-4-4-5 http://fiddler2.com/blog/blog/2013/01/17/fiddler-imageview-enhancements

Metadata & GeoLocation http://fiddler2.com/blog/blog/2013/01/17/fiddler-imageview-geolocation

HTML5 Media & Font previews

X-Download-Initiator https://fiddler2.com/dl/EnableDownloadInitiator.reg cols add @request.X-Download-Initiator Run this: https://www.fiddler2.com/dl/EnableDownloadInitiator.reg FiddlerObject.UI.lvSessions.AddBoundColumn ("Reason", 50, “@request.X-Download- Initiator"); cols add @request.Accept cols add @request.X-Download-Initiator

Traffic Manipulation

Automated Rewrites Simple built-in Rules The HOSTS command

Use Fiddler Inspectors to modify requests and responses…. Breakpoint Debugging Use Fiddler Inspectors to modify requests and responses….

Flag, modify or remove headers from all requests and responses. Simple Filters Flag, modify or remove headers from all requests and responses.

Request Composer Create hand-built requests, or modify and reissue a request previously captured. Supports: Automatic authentication File Uploads Redirect chasing Sequential URL Crawling CURL commands

Replay previously-captured or generated traffic. AutoResponder Replay previously-captured or generated traffic. http://fiddler2.com/blog/blog/2012/11/15/better-repro-playback-with-fiddler Drag / drop entire folder structure FARX Bulk updates *drop Latency *delay:###ms // AutoResponder now supports *redir: and HTTP/HTTPS URL overrides // AutoResponder now supports *exit // AutoResponder now supports NOT: rules

FiddlerScript

FiddlerScript – Request Modification static function OnBeforeRequest(oS: Session) { if (oS.uriContains(".aspx")) { oS["ui-color"] = "red"; } if (m_DisableCaching) { oS.oRequest.headers.Remove("If-None-Match"); oS.oRequest.headers.Remove("If-Modified-Since"); oS.oRequest["Pragma"] = "no-cache"; }

FiddlerScript – Response Modification static function OnBeforeResponse(oS: Session) { oS.utilDecodeResponse(); oS.utilPrependToResponseBody( "Injected Content!"); }

Powerups

Understanding Extensibility Each component in red is your code… Fiddler.exe Inspector2 Script / Batch file ExecAction.exe Inspector2 IFiddlerExtension IFiddlerExtension Fiddler ScriptEngine Your FiddlerScript FiddlerCore Xceed*.dll Makecert.exe

Understanding UI Extensibility RulesOptions ToolsActions Custom menus Custom columns ContextActions QuickExec handlers Views Request Inspectors Response Inspectors Import & Export Transcoders

Type-specific Inspectors

Expert Perf Analysis with neXpert http://www.fiddler2.com/fiddler2/addons/nexpert.asp http://calendar.perfplanet.com/2012/building-faster-sites-and-services-with-fiddler/ Creates response time predictions and suggests optimizations for your site.

intruder21 Web Fuzzer By yamagata21 MIX 11 4/14/2017 http://yamagata.int21h.jp/tool/intruder21/ By yamagata21

Watcher & x5s Security Auditors MIX 11 4/14/2017 Watcher & x5s Security Auditors http://websecuritytool.codeplex.com/ http://xss.codeplex.com/ http://websecuritytool.codeplex.com/ http://xss.codeplex.com/

WCF Binary Inspector MIX 11 4/14/2017 http://code.msdn.microsoft.com/wcfbinaryinspector

Integration

ExecAction.exe Calls into OnExecAction in script or extensions Alternatively, invoke directly by sending a Windows Message: oCDS.dwData = 61181; // Magic Cookie oCDS.cbData = lstrlen(wzData * sizeof(WCHAR)); oCDS.lpData = wzData; SendMessage( FindWindow(NULL, "Fiddler - HTTP Debugging Proxy"), WM_COPYDATA, NULL, (LPARAM) &oCDS ); http://www.fiddler2.com/fiddler/dev/#Automation

Fiddler application with extensions Your application hosting FiddlerCore Fiddler.exe YourApp.exe Inspector2 ExecAction.exe Inspector2 IFiddlerExtension IFiddlerExtension Fiddler ScriptEngine Your FiddlerScript FiddlerCore FiddlerCore Xceed*.dll Makecert.exe DotNetZip CertMaker.dll

Programming with FiddlerCore // Call Startup to tell FiddlerCore to begin // listening on the specified port, register as // the system proxy and decrypt HTTPS traffic. Fiddler.FiddlerApplication.Startup(8877, true, true); Fiddler.FiddlerApplication.BeforeResponse += delegate(Fiddler.Session oS) { Console.WriteLine("{0}:HTTP/{1} for {2}", oS.id, oS.responseCode, oS.fullUrl); }; // Later, call Shutdown to tell FiddlerCore to stop // listening and unregister as the system proxy Fiddler.FiddlerApplication.Shutdown(); http://fiddler.wikidot.com/fiddlercore-api http://fiddler.wikidot.com/fiddlercore-demo Now available for .NET CLR 4.0, supporting HTTPS, with an arbitrary numbers of listening endpoints. Export captured traffic to the SAZ format or the database of your choice.

Fiddler Futures WebSockets UI SPDY/HTTP2 UI Enhancements You tell me!

Thank you!!! Eric Lawrence @ericlaw //fiddlerbook.com ~300 pages. Paper or DRM-free PDF. //fiddlerbook.com