Catherine Jones Science and Technology Facilities Council SCAPE Training Statsbiblioteket, Aarhus, November 2013 Control Policy formulation The why and how

Format of this session 11:15 – 11:40 Presentation on creating control policies 11:40 – 12:25 Practical Exercise (small groups) 12: :45 Discussion about the practical exercise and the topic of policy in general 2

What is digital preservation policy about? The organisation’s aims and objectives about the long term care of digital objects: Preservation strategies and acceptable actions Decision about the digital objects (formats, significant properties etc) Who the material is being preserved for Resourcing Responsibilities 3

The role of policy in planning and watch 4

SCAPE Policy Levels - recap 5 Guidance High levelGeneral objectives Applies to all parts of the organisation and collections Written in natural language to be read by a human being Preservation Procedure More detailed levelGeneral approaches Written in natural language to be read by a human being Control Specific, measurable objectives Applies to specific collections or formats In two forms: natural language and machine readable form (RDF)

Why two forms of control policies? Natural language policy needed for humans and may (should) already exist – in procedures/collection management policy/implicit understanding etc. Need machine understandable form to use automated tools 6

What is special about SCAPE machine understandable control policies? Related to a specific set of circumstances – the collection of digital objects; the people who will use them and a purpose. Known as a preservation case Need to be specific so that they can be measured or assessed. File format must be TIFF There must be 3 copies of each object Not all control policies may be machine actionable There must be 3 members of staff who have qualification X 7

SCAPE Control Policy model links a particular content set (collection) with a particular user community (specific requirements) with specific measurable objectives which can be tested automatically 8

Some examples 15

What do you need to create machine understandable control policies? 16 Some written policy – either at the Preservation Procedure level, or at the more detailed control level. An understanding of the goals of preservation Knowledge of the collection and who uses it & manages it & any procedures in place. Some appreciation of what topics you are likely to need Planning & Watch activities for

17 Creating Control policy statements Stage 1: Whole policy activities Stage 2: Policy statements within the whole policy Stage 3: Review and rationalise

18 These are activities considering the policy as a whole 1.Identify the content set the policy addresses What type of material is being preserved in this case? 2.Identify the user communities/roles required by the policy Who will be using the material or interacting with the material? 3.Map policy statements to high level concepts. In general what type of activities are the statements referring to? Creating Control policy statements Stage 1: Whole policy activities

For each statement or section in the policy undertake: 1.Clarification of implicit meaning Are there hidden meanings/context that needs to be stated explicitly? 2.Identification of control policy preservation case What issue is the statement addressing? 3.Identification of objectives What are the measureable statements which embody the policy statement? 4.Generate control statements Use of a tool or knowledge of RDF to create machine understandable statements 19 Creating Control policy statements Stage 2: Policy statements within the whole policy

For preservation cases and associated objectives review: 1.Are there any objectives which are in every preservation case? These are candidates for organisation related objectives 2.Do some of the preservation cases overlap/are the same? You need to consider whether fewer but broader preservation cases or multiple specific ones is the most appropriate. This depends on what you intend to use them for, and what overheads there are in maintaining the optimal number 20 Creating Control policy statements Stage 3: Review & Rationalise

Worked Example 21 “3.1.1 All raw data will be curated in well-defined formats for which the means of reading the data will be made available by the Facility” Express some of the implicit information and rewrite to: “All data curated will be in well-defined formats” “Approved well-defined formats will be able to be read” “The reader will be supplied by at least the ISIS Facility” Also need to express what “curated” means Goals/Objectives: 1.File format must be of an approved format for the contentset 2.The file format should have documentation 3.Any instrument specific schema should be documented 4.There should be at least one piece of software which can read the files 5.This file reader should be available from the organisation holding the data 6.This file reader should be able to be used by the designated user community 7.The file format should be able to be validated 8.Fixity checks should be undertaken Using the contentset 2011 LET Calibration and a user community of domain specific researchers i.The file reader MUST be available to the designated user community Using the contentset 2011 LET Calibration and a user community of ISIS data managers i.File format MUST be NeXus ii.The file format MUST have documentation iii.Any instrument specific schema MUST be documented iv.Nexus File reader software available > 1 v.NeXus file reader MUST be located at STFC vi.The file format MUST be able to be validated vii.Fixity checks MUST be able to be undertaken

Conclusion Having explicit policy in natural language is important Expressing policy in machine testable ways is more complex but can bring benefit through use of tools Natural language policy defines statements of acceptable states; machine understandable control level asks measureable questions Implicit information understood by human audience which needs explicitly expressing for computers Written policy is at a fairly abstract level and practicalities may be addressed in implementation plan/job procedure document or one-off project plan 22

Next – a practical exercise You should have: The example scenario Sheets with possible attributes and measures Control Policy worksheets In pairs or small groups try converting the scenario into control policy statements 23

Barriers to overcome Difference in approach/philosophy natural language preservation policy defining acceptable (and not acceptable) states in statements control level policy defining testable/measurable attributes in questions Coverage going top down Written policy is at a fairly abstract level Practicalities may be addressed in implementation plan/job procedure document or one-off project plan Implicit information understood by human audience which needs explicitly expressing for computers 24

25 Flow of control policy creation This work was partially supported by the SCAPE Project. The SCAPE project is co‐funded by the European Union under FP7 ICT‐ (Grant Agreement number ).

Guidance policy This will be at a high level that a Director of an organisation would understand. Topics: Preservation goals & strategies of an organization Designated Community/Stakeholders Digital Objects Metadata Authenticity Rights Standards Organisation Storage 26

Preservation Procedure Preservation Procedure: Natural language human readable policy which may encompass the whole organisation or may be focused on a particular collection or material type depending on the needs of the particular organisation SCAPE outcome in this area will be information and guidance on how to construct this level of policy and what factors need to be taken into consideration when composing it for areas of particular interest in watch and planning. 27 The list of suitable data formats for digital preservation will be based on the following criteria: Openness of the format: Is the format well described and is documentation available? Is the format subject to any patents? Is a licence or permission required to use the format? Distribution of the format: Is the format used widespread? Will many programmes be able to understand the format? Error tolerance of the format: Will a single bit error make the whole file unreadable? Has the format been compressed (lossless or lossy data compression)? Acceptance of the format as a preservation format: How is the format evaluation on corresponding lists of recommended formats? Dependency of the format of external sources of information, for example fonts or pictures with external references. Ability of the format to embed data in other formats, for example embedding of video in a pdf-file. Based on these criteria the owner of the digital collections can add a data format to the list as “Recommended” and “Accepted”.