Services for Sensitive Research Data Iozzi Maria Francesca, Group Leader & Nihal D. Perera, Senior Engineer Research Support Services Group ”Services for Sensitive Data” University Center for Information Technology (USIT) University of Oslo

Outline  Part I : Introduction  Part II : Getting started  Part III : More and more difficult… Gard Thomassen,TSD 2.0

Services for Sensitive Data - TSD «Services for Sensitive Data» is an e-Infrastructure which provides a set of services to collect, store, compute and analyze sensitive-data, in a highly secured environment. Our services are recognized by :  Norwegian Data protection Authority (DT)  Regional Ethical Committee (REK)  Norwegian Social Science Data Services (NSD) Our services are designed to serve all the universities, high schools and other public research institutions in Norway.

What is sensitive data?

TSD provides:  Large storage capacity  Data collection service (Web-Forms/Nettskjema)  High performance computing (HPC)  PostgreSql databases  A variety of software (for data analysis) Our services are in compliance with the directive on privacy and electronic communication

Type of machines & services a project can get  A project can have Windows 2012 Server VM and/or a Linux Server VM  The VMs comes with a portofolio of software installed.  If requested each project get access to our HPC cluster – Colossus.  Each project gets their own virtual working environement in a dedicated VLAN/subnett.  A project can have many users and one (or more) administrators

TSD in a nutshell! 7 Gateway HPC - ColossusVM-server Storage Internet Secure encrypted network to special high volume data production sites 1 (project) 1 (storage area) n 1 Gard Thomassen,TSD 2.0

Using TSD VM U1 S1 S1 TSD disk VM U2 S1 GW User1 Study1 Colossus disk Colossus Front end Colossus Gard Thomassen,TSD 2.0 User2 Study1 TSD S1 DB

Demo I am a researcher and I would like to start using TSD…. Order storage and computing power on TSD.

Part II How to log in How to change the password Filesystem and directories structure How to import/export file How to set up nettskjema in TSD How to submit jobs on Colossus Cluster I have done a mess! What shall I do?

Login to TSD Two factor authentication login, just like the bank! Username Password One-time-code

Demo Step1: Connect your laptop to the uio-guest network. Open a browser window and order a UiO-guest account (you need your mobil!) Step2: Are you a linux user? Are you a windows user? (you have to select the “Install VMWare Horizon Client”)

Demo Step3: Download on your mobil the app: Google Authenticator (or FreeOTP for Android). Open the App, select “configure” and then “scan barcode” The barcode is on the paper you received! Step4: Use your username, password and OTP code to connect to TSD! Enjoy!

Demo: How to change password? Open a browser in TSD and type:

Filesystem and directories structure Given that your project is pXX (here p77) there are: Directories that visible to all the pXX-users pXX/data/durable (important stuff!) pXX/data/no-backup (not so important stuff!) /cluster/project/pXX (hpc) Private directories ( single user) pXX/home Import and Export folders (all pXX-users) pXX/fx/import pXX/fx/export pXX/fx/15MC56NAUKWPN629/60044 (nettskjema) /shared/ read only to all! (data useful for everyone)

Filesystem and directories structure Panic! Where are my directories???? \\tsd-evs\pxx

Data import & export facility in TSD

“Sluice HD” “Sluice –server ” “Project–server ” Virtual “Sluice – server ” Project HD Data copied here by ssh + scp (2-factor authentication) Encrypted data if sensitive TSD 2.0

How to import/export files in/out TSD? NB: you need to have file-transfer software on your local machine (either sftp or FileZilla or winSCP) to connect to the filelock: tsd- fx01.tsd.usit.no Export: 1) in TSD drop your file in the /tsd/pxx/fx/export folder 2) on your local machine, login to the filelock and pick up the file! Import: 1) on your local machine, login to the filelock and drop the file 2) in TSD pick up the file from /tsd/pxx/fx/import folder Link :User manual :File import/exportUser manual :File import/export Demo live of the file import.

How to import/export files in/out TSD? Note of caution:  No files bigger then 1TB  No 100 files at the time. Better one tar-ball (or zip or 7zip)!  No special characters, space etc. in the file-names (norwegian characters etc)  Remove the file from the filelock once you have copied it!  Encrypt before transferring!

SECURE DATA COLLECTION FACILITY IN TSD

How to set up nettskjema in TSD? You can use nettskjema to run web-based questionnaire. The sensitive answers will be delivered directly to TSD! Create your Nettskjema form ( Get the Form ID Inform us:

How to set up nettskjema in TSD?... we need to create an encryption key for you. (secret and public key pairs). Once we are ready: Your nettskjema answers will appear in here: /tsd/pxx/fx/import/sns/SBHA5SJDKS8KW8/ / The answers are encrypted. Use either Kleopatra or GPA (windows) or gpg (linux) to decrypt them. The keys are stored in: /tsd/pxx/data/durable/pxxGPG/ With Kleopatra or GPA, you need to import the secret key at the first use:

High performance computing in TSD (HPC)

How to use Colossus (HPC)? Your project needs to be configured in order to use Colossus resources (HPC) and must have a linux VM server Connect to the linux server ( Open a terminal and write your sbatch script Copy your data you want to compute on /cluster/projects/pXX Submit you script Copy your output back to /tsd/pXX/data/durable Do not store data on /cluster/project/pXX permanently! $$$$$$$$

How to use Colossus (HPC)? Software on Colossus are synced with the Abel software portfolio Max run on colossus: 30 days! Colossus has hugemem nodes (16GB node) Attention: you can submit a job 30 day 4 hugemem nodes but it will cost several thousand NOK!!! You can increase your memory, but if max-mem > mem- per-cpu then you consume more CPUs (because you are allocating more CPUs) Be smart! Tune properly your job scripts.

I have done a mess! What shall I do? Have you deleted by mistake the entire data of the PhD thesis to be presented in one month? Breath deeply and calm down.... Linux: cd /tsd/pxx/.snapshot/ here you find everything from thelast night! Windows: ask us! We have the snapshot for you.

Demo: Get support from TSD I have a problem in TSD, what shall I do ?

Part III How to Install a software without internet on your linux VM How to check the integrity of a file after import How to decrypt a file in TSD