Capture-Replay Mocks Scandinavian Developer Conference 4 th April 2011 Geoff Bache

Copyright © 2008 Jeppesen Sanderson, Inc.Jeppesen Proprietary and ConfidentialSlide # Overview What is a “mock”? What do “coded mocks” look like? Introducing Capture-Replay mocks CaptureMock, with examples Demo

Copyright © 2008 Jeppesen Sanderson, Inc. What are “mocks”? “Simulated objects that mimic the behaviour of real objects in controlled ways” (Wikipedia) More strictly: “mocks” actively expect and assert behaviour “stubs” passively return hardcoded values “mocks and stubs” are collectively called “fakes” or “test doubles”. Terminology unfortunately not used consistently.

Copyright © 2008 Jeppesen Sanderson, Inc. Examples of when to use mocks/stubs When using the real code in your test would be  Hard to analyse (e.g. images created)  Too slow (e.g. databases)  Indeterministic (e.g. current time)  Hard/expensive to set up (e.g. network errors)  Impossible (because it doesn’t exist yet) As a design strategy (Behaviour-Driven Development)

Copyright © 2008 Jeppesen Sanderson, Inc. Mocking in Java (Mockito) //You can mock concrete classes, not only interfaces LinkedList mockedList = mock(LinkedList.class); //stubbing stub(mockedList.get(0)).toReturn("first"); stub(mockedList.get(1)).toThrow(new RuntimeException()); //following prints "first" System.out.println(mockedList.get(0)); //following throws runtime exception System.out.println(mockedList.get(1)); //following prints "null" because get(999) was not stubbed System.out.println(mockedList.get(999)); //Checks that we called “get(0)” at some point verify(mockedList).get(0);

Copyright © 2008 Jeppesen Sanderson, Inc. Mocking in Python (Mock) # Fix method return values and verify they were called from mock import Mock my_mock = Mock() my_mock.some_method.return_value = "value" assertEqual("value", my_mock.some_method()) my_mock.some_method.assert_called_once_with() # Set up expectation we will raise exceptions... my_mock.other_method.side_effect = SomeException("message") assertRaises(SomeException, my_mock.other_method())

Copyright © 2008 Jeppesen Sanderson, Inc. Stubbing in Python (“monkey patching”) # Example of ”monkey-patching” import time def hardcoded_asctime(): return 'Thu Mar 24 16:12: ' # We monkey-patch the standard library for our test time.asctime = hardcoded_asctime class ProductionClass: def method(self): print ”The time is”, time.asctime() real = ProductionClass() # Will print our faked, deterministic answer real.method()

Copyright © 2008 Jeppesen Sanderson, Inc. Problems with using mocks Involves writing sometimes complex code  Takes time to write  Has to be maintained Easy to create a lot of dependency on implementation details If the “real code” changes its behaviour :  The tests stay green  But the system doesn’t work any more...

Copyright © 2008 Jeppesen Sanderson, Inc. Capture-Replay Mocks Generate mocks by running the real code and capturing how it behaves Results in a normal test that can be run in two ways  using the stored mocks  or regenerating them from the real code No mock-code to write, just say what you want to capture Much easier to keep in synch with real code

Copyright © 2008 Jeppesen Sanderson, Inc. It's not “capture-replay” like EasyMock! # Some mock frameworks just use “capture-replay” as a syntax import mocker # We start in the “capture phase”... my_mock = mocker.mock() my_mock.some_method() mocker.result("value") # and then we “switch to replay” when we're ready to test mocker.replay() #... test code import mock my_mock = mock.Mock() my_mock.some_method.return_value = "value" #... test code Which is basically just a more verbose way to express:

Copyright © 2008 Jeppesen Sanderson, Inc. CaptureMock recording System under test 3rd party module CaptureMock Record traffic

Copyright © 2008 Jeppesen Sanderson, Inc. CaptureMock replay System under test 3rd party module CaptureMock

Copyright © 2008 Jeppesen Sanderson, Inc. CaptureMock : doing it in practice CaptureMock is a tool written in Python that can do this with  Python code (modules and attributes)  Command-line calls (subprocesses)  Plain-text messaging (over sockets) “Python code” part relies on Python’s dynamic features Other parts are language-independent Plain-text messaging assumes a synchronous model each socket sends one message and gets one reply.

Copyright © 2008 Jeppesen Sanderson, Inc. A Python CaptureMock test # test_ .py from capturemock import capturemock class ProductionClass: def method(self): # call some code resulting in being sent def test_ _sending(): real = ProductionClass() real.method() I can then run: $ env CAPTUREMOCK_MODE=1 test_ .py which will actually send the and record the interaction with smtplib to a file called ” _sending.mock”

Copyright © 2008 Jeppesen Sanderson, Inc. A CaptureMock recorded mock (Python) <-PYT:import smtplib <-PYT:smtplib.SMTP() ->RET:Instance('SMTP', 'smtp1') <-PYT:smtp1.connect('machine.site.com') ->RET:(220, 'machine.site.com ESMTP Sendmail; Tue, 9 Feb :32: ') ['tom', 'dick', 'harry'], '''From: To: tom,dick,harry Subject: Hi Guys! I love you all! ''') ->RET:{} <-PYT:smtp1.quit() When I run without CAPTUREMOCK_MODE, no will be sent and if the calls received differ from above, an exception will be thrown.

Copyright © 2008 Jeppesen Sanderson, Inc. A command-line CaptureMock test # update_and_build.sh # We update some code from CVS and then try to build it returned=`cvs update –dP /path/to/my/checkout` # logic using stuff written on stdout etc... make Here I put the following in.capturemockrc to tell it to capture calls to CVS [command line] intercepts = cvs which will perform the update for real and record to ”cvs_calls.mock” $ capturemock --record cvs_calls.mock update_and_build.sh and then run

Copyright © 2008 Jeppesen Sanderson, Inc. A recorded mock from a subprocess <-CMD:cvs update -dP /path/to/my/checkout ->FIL:checkout ->OUT:U subdir/myfile.txt ->ERR:cvs update: Updating. cvs update: Updating subdir Note the ”FIL” line refers to stored copies of the files changed by cvs. They are stored in a directory called ”cvs_calls.files”. To replay this, we run $ capturemock --replay cvs_calls.mock update_and_build.sh maybe on a machine where CVS isn't installed. CaptureMock will pretend to be CVS, reproducing the standard output, standard error, exit code and files written.

Copyright © 2008 Jeppesen Sanderson, Inc. A client-server CaptureMock test Assuming I have a client that sends strings to a server, which replies with how long they are, and a script that runs both together: $ capturemock --record communication.mock run_client_server.sh <-CLI:Here is a string ->SRV:Length was 16 <-CLI:Here is a longer string ->SRV:Length was 23 Something like this will appear in communication.mock: I can then use this data to test either the client or the server in isolation from the other.

Copyright © 2008 Jeppesen Sanderson, Inc. Demo : graphs and matplotlib

Copyright © 2008 Jeppesen Sanderson, Inc. Examples of when to use CaptureMock When using the real code in your test would be  Hard to analyse - yes  Too slow - yes  Indeterministic - yes, with care (record mode may fail)  Hard to set up - possibly (record mode still hard to set up)  Impossible because it doesn’t exist yet - no As a design strategy (Behaviour-Driven Development) - no

Copyright © 2008 Jeppesen Sanderson, Inc. Conclusions Existing mocking approaches – are labour-intensive when non-trivial code is mocked out – don't provide an easy means to verify the mock against real code CaptureMock – provides a means to (re-)generate mock information from the code – and two "modes" to select from each time tests are run