1. The Insight Journal Kitware Inc.

2. Credits Julien Jomier – UNC Caddlab Stephen Aylward – UNC Caddlab Zack Galbreath – Kitware Andy Cedilnik – Kitware Bill Hoffman - Kitware

3. The Insight Journal Technical work must be reproducible Papers should be publicly accessible The Peer-Review process must be open

4. Lets Implement this paper…. Happy Grad Student, unaware of her destiny Supervisor You just need to implement the code and try it…

5. Lets Implement this paper…. Unhappy Grad Student, aware of her destiny Six months later… Parameters are missing Images are not available Implementation details are missing Abundance of pretty equations Abundance of pretty screenshots

6. Time to face the Truth

8. Publications are a measure of scientific productivity They disseminate knowledge They allow others to reproduce our results They are validated by the peer-review process

9. Why do we really want to publish ?

10. Because we need to have publications in our CV

11. We have met the enemy… and he is us !

12. Publish or Perish Who invented this ? and Why ?

13. Publish or Perish Was invented by those who needed to evaluate researchers productivity.

14. Publish or Perish Empowers those who read your CV to grade you by simply counting lines in thePublications section.

15. Publish or Perish The group of best educated people in the world has been alienated with a simple trick

16. The Scientific Method Hypothesis Outcome Experiment Galileo Galilei Hypothesis Rejection Confidence

17. The Scientific Method The Logic of Scientific Discovery by Karl Popper

20. Image Processing Method Algorithm Input Image(s) Parameters Output Image(s)

21. Insight Journal Manager Author Submits Project DSpace Stores Project Automatic Testing Environment Posts Results Dashboard Submits Entry Reviewer Reviews Project Process Flow Sends Code

22. Submission Code Input Data Journal Repository Web Site Results Data Author Build Machines PDF doc

27. Comparison of output images Publicly available Tests for non-trivial images Reviewers and readers can use it as a first view of the paper. Results Verification

29. DSpace Open Source Developed by MIT & HP Persistant URIs Prevents data from becoming obsolete Insight Journal Manager DSpace Stores Project

30. Insight Journal Manager Open Access Peer Review Searchable Insight Journal Manager Author Reviewer Peer-Review is ONLY useful if it enforces reproducibility

31. Testing Environment Security Issues Open source tools Dashboard Insight Journal Manager Automatic Testing Environment Dashboard

32. Enjoy the Insight Journal !

34. Implementation Details

35. Insight Journal Manager http://insight-journal.org Web interface: PHP/Java/HTML Central place for submissions, reviews, browsing/searching No delay between submissions & reviews Public reviews

36. Issues Title, comments. Creation/Paper Due/Notification/Publication dates Tied to a DSpace collection Submission through DSpace + comments

37. Adding reviews Rating Level of Affiliation Expertise Anonymous Review Title Review

38. Browsing reviews No delays Users can post comments Authors can reply to comments PDF can be downloaded

39. Scoring Weighted average of: Sensitivity (STAPLE algorithm) Helpfulness # submissions/max submissions # reviews/max reviews # anonymous reviews

40. Whats next? Moderated submissions SVN repository for code fixing …

41. Safe Testing Virtualization! Lots of options: User-Mode-Linux, coLinux, VMWare, chroot Xen: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/ http://www.cl.cam.ac.uk/Research/SRG/netos/xen/

42. Tools used in the testing process Xen Debian CMake Python Dashboards

43. Testing Details two CPUs and two partitions, so two tests can be run simultaneously VMs can only access their own partition; no network access Mutexes prevent filesystem corruption Submit test results to dashboard Restore pristine filesystem after every test

44. Enjoy the Insight Journal !