1. TSD: a Secure and Scalable Service for Sensitive Data and eBiobanks Gard Thomassen, PhD Head of Research Support Services Group University Center for Information Technology (USIT) University of Oslo

2. Outline Sensitive Data TSD setup, solutions, status and future Lessons learned How to get on board

3. What is sensitive data? Norway : Personal Data Act §2, point 8 –race/ethnic data, political opinion, philosophical and religious beliefs, the fact that a person has been suspected of, charged with, indicted for or convicted a criminal act, health, sex life and trade-union membership

4. Who has sensitive data Almost everyone

5. TSD launch in Computerworld 16/5-14

6. TSD Pilot 2009 - 2012

7. System requirements Security, isolation and access control as given by law Large storage capacity Multi tenant (multiple users) High performance computing (HPC) resource High bandwidth Easy to maintain and operate Easy to use and “practical” (also for audio and video) Some freedom within confined user space Accessible from anywhere through proper mechanisms A variety of software and public data-sources must be available Windows and Linux support (server/host-side) Data collection services Data sharing services

8. Setup, solutions and status

9. System outline Gateway HPC - ColossusVM-server Storage Internet Secure encrypted network to special high volume data production sites 1 (project) 1 (storage area) n 1

10. Using TSD VM U 1 S 1 S1S1 TSD disk VM U 2 S 1 GW User 1 Study 1 Colossus disk Colossus Front end Colossus User 2 Study 1 TSD S 1 DB SPSS Office Stata SAS R Matlab.... Libre Office R Module load...

11. Data import and export using TSD File lock server Virtual file lock server Virtual project- server File lock HD Project HD TSD NFS mount 2 Data copied here by sftp (2-factor authentication) encrypted data if sensitive 1 4 3

12. Data collection using TSD “Nettskjema-minID” “Nettskjema-minID” Nettskjema homepage minID Project VM Project disk File lock Encrypted XML (PGP) TSD

13. Security details OATH TOTP 2-factor authentication –Smart phones or programmable hardware tokens Import/export is under strict control No open connection to the internet All administration happens from the inside Strong separation between projects Hardened FreeBSD gateway and firewall Encrypted backup, one key per project Sys-admins are single users (traceability) Sys-admins have to use same authentication process

14. TSD status > 80 research projects > 350 users Secure storage (> 1 PiB on disk) Secure data analysis Linux or windows hosts (> 250 VMs) Secure import and export Web-based data harvesting HPC cluster (>1500 cores) Postgres DBs Video and sound display

15. Capabilities enabled by TSD Large scale NGS research on human genomes Large scale medical imaging studies Large scale studies with web-based data collection Off-site analysis of sensitive data Secure storage for verification of published research Electronic consent

16. Future of TSD - main topics How to handle video and sound –harvesting –management –metadata –analysis Journal system for Psychologists (Univ of Umeå collaboration) Biobanks VMware and VDI infrastructure Galaxy inside TSD Elixir helpdesk connected to TSD Hosting docker containers Invariant storage of research data National eInfrastructure investement in TSD

17. Lessons learned Design before you implement Do security assessment during all the time Brainstorm and discuss Test, document and implement in paralell You will have to redo things! Have a “Board of Changes” when in prod

18. How to get on board tsd-drift@usit.uio.no

19. Thanks to tsd-core@usit virt-core@usit storage-core@usit postgres-core@usit network-core@usit hpc-core@usit windows-core@usit unix-core@usit IT-security@usit Project group / developers IT-dir Lars Oftedal Hans A. Eide Märtha Felton Administration / associated