1. Perspectives on the intersection between computer science and psychology
Developing reproducible – and reusable – methods through research software engineering

2. Open Science and Digital Scholarship
Interoperability of data
Data standards
Cyber infrastructure
Research methods
Often there isn’t a SOP
How can we ensure our bespoke method is reproducible?

3. Reusable Research Methods
Challenges of reproducing research
Automation can help
Documentation isn’t enough
Why should we address these challenges?
To allow verification of results
So others can use methods
How can we addresses these challenges?
Research software engineering

4. Dynamic web content
Can we predict if someone will view an update?

5. Video of eye tracking video (‘raw data’) available from

6. Analysis process Step through the video
Manually define areas of interest
Log start and end times
Final analysis: CHAID in SPSS – all data online.

8. Why isn’t this research reproducible?
Incomplete raw data
Proprietary software
‘Idiosyncratic’ scripts
Experimental materials not available

9. IDInteraction

10. Can we automate behavioural coding?
Manual coding of motor behaviour not viable

11. Video processing: The Horror
Install Linux (Ubuntu )
Install mencoder (version )
Plus all 33 of its dependencies
And all of their dependencies, and so on…
Run, for every input file, one after another:
$ mencoder <input> -vf crop=640:360:640:0 -nosound -ovc x264 -o <front-view-output>
$ mencoder <input> -vf crop=640:360:0:0 -nosound -ovc x264 -o <side-view-output>
$ mencoder <input> -vf crop=640:360:640:360 -nosound -ovc x264 -o <rear-view-output>

12. Video processing: IDInteraction
Install docker
Works on Windows, OS X or Linux
Run, once:
This downloads the video processing pipeline and runs it over all of the input data automatically
Come back tomorrow
For new data, just re-run the docker command above and it will only process new inputs
$ docker run -it -v <input-dir> -v <output-dir> idinteraction/video

13. Object tracking: The Horror
Assuming that Linux is already installed
Install OpenCV (version 2.4.8)
Plus all 34 of its dependencies
Plus all of the development libraries and header files
And all of their dependencies, and so on…
Install C++ compiler
Install CMake
Install Git
Install CppMT (C++ version of CMT tracker)
Clone from GitHub
Get the IDInteraction specific version
Configure the build
Compile to produce ‘cmt’ executable
Run, for every output of the video processing stage, one after another:
$ cmt --bbox=<initial bounding box> --output <output dir> <video file>

14. Object tracking: IDInteraction
Assuming that docker is already installed
Run, once:
This downloads the object tracking pipeline and runs it over all of the input data automatically
Come back tomorrow
For new data, just re-run the docker command above and it will only process new inputs
$ docker run -it -v <input-dir> -v <output-dir> idinteraction/tracking

15. Open source software Specify coding start point
Outputs data into files
Batch process videos
Define, adjust and edit the bounding box
Changes merged back
Georg Nebehay

16. Why bother? Because research should be reproducible
So others can use the method
Our collaborators
The community
Caveat:
I live in computer science

17. How do we address the challenges?
Recognize the requirement for technical excellence in research software
Integrate software engineering into the research process
Work out how to make software sustainable and maintainable without any funding