1. February 12, 2011 NEMO All-Hands Meeting: Database and Portal Kurt Mueller and Jason Sydes http://nemo.nic.uoregon.edu

2. Agenda FTP site / file database overview NEMO portal – Present – Future

3. Database – FTP site Place to store files – Raw waveforms (input files) – Intermediate analysis results – Parameter files – Final output files

4. Organization Organize files by lab, experiment, file type: LAB DIRECTORY EXPERIMENT DIRECTORY INPUT_DATA DIRECTORY ANALYSIS_1 DIRECTORY ANALYSIS_2 DIRECTORY

5. Example JTC (lab-level folder) JTC-PMN1 (experiment-level folder) JTC-PMN1_Input_Data (subdir) 'JTC-PMN1.raw' (input data file) JTC-PMN1_Decomposition_2011-01-15 (subdir) 'JTC-PMN1_tPCA.m' (metascript file) 'JTC-PMN1_DecompObj-20110115.mat' (MATLAB output file) 'JTC-PMN1_tPCA.raw' (simple binary output data file)

6. Uploading files – ftp client

7. Data in FTP is both managed and unmanaged Mixture of two different areas – Unmanaged – Portal managed Possible area for documentation? What role will FTP server will serve in future? Can browse and download portal data via FTP – Readonly!

8. Data in portal accessible via FTP

9. Graphical ftp clients Mac OS X + Windows + Linux – Filezilla - http://filezilla-project.org/http://filezilla-project.org/ Mac OS X + Windows – Cyberduck – free - http://cyberduck.ch/http://cyberduck.ch/ Mac OS X – Transmit - $34 - http://www.panic.com/transmit/http://www.panic.com/transmit/

10. Goals of portal 1.Online record of experiment metadata 2.Repository of experiment data files 3.Tight coupling to NEMO ontology 1.Provenance of all data 2.NEMO analysis pipeline execution 3.Querying on metadata now in progress

11. Record experiment metadata Experiment Conditions Sessions Publications Subject groups Data files Task Stimuli Responses

12. Quick Sampling of Portal

17. Portal database tables

18. Repository of experiment data files New upload widget provides progress feedback (standard http upload does not)

19. OpenID access to NEMO resources SSO: Single Sign On Login once, get access to – NEMO portal – NEMO wiki – NEMO sourceforge page Learn about it: http://openid.net/http://openid.net/ How to create a new SSO account: http://nemo.nic.uoregon.edu/wiki/New_Nemo_User

20. Role-based access control (RBAC)

22. Database coupled to Ontology Example Table: Stimulus

23. Database coupled to Ontology Example “Ontology”: Stimulus

24. Database coupled to Ontology

25. Database coupled to Ontology Example mapping: Stimulus

26. Database coupled to Ontology Example term: stimulus_intensity

27. Database coupled to Ontology Example tooltip

28. Database coupled to Ontology Quick syncing

29. Open Provenance Model (http://openprovenance.org/)http://openprovenance.org/

30. Open Provenance Model – subset used –

31. Open Provenance Model Example

33. Open Provenance Model Simplified Example

34. Workflow – MATLAB in portal Current workflow: 1.Conduct experiment, producing raw waveforms 2.Process data with NEMO Toolkit scripts, locally 3.Upload raw and processed data to ftp or portal 4.Create an experiment in the portal, associate data with experiment

35. Worfklow – MATLAB in portal Future workflow (in design) 1.Conduct experiment, producing raw waveforms 2.Create an experiment in the portal 3.Upload raw data through portal 4.Execute MATLAB pipeline through portal Automatic storage of output data Automatic provenance tracking Single place for MATLAB scripts to live Run on big iron

36. Questions?

37. Thank you!