1. The International Seed Testing Association GMO PT Program
AIEC Fall Meeting October New Orleans, Louisana

2. What is the International Seed Testing Association?
An international, non-profit association of persons and seed testing laboratories from the public and the private sector
Under the Governance of the Governments of member countries and distinct economies
An Association standardizing the determination of seed quality to facilitate international trade
Explain the strategy used by ista.

3. Ordinary Meeting Executive Committee Secretariat
Voting delegates from 72 Member Governments
Executive Committee
11 Members from all Continents of the World nominated by Governments
Secretariat
11 professional employees
18 Technical Committees and Boards
Over 200 experts from all over the world working voluntarily
Experts from the public/private sectors, from member/non-member laboratories

4. ISTA Executive Committee 2013-2016:
President
Vice President
9 Members-at-Large
Joël Léchappé, France
Craig McGill, New Zeeland
Mable Simwanza, Zambia
Steve Jones, Canada
Alexander Malko, Russia
Berta Killerman, Germany
Masatoshi Sato, Japan
Grethe Tarp, Denmark
Margus Friedenthal, Estonia
Rita Zecchinelli, Italy
Cecilia Jones, Uruguay

5. ISTA Secretariat Benjamin Kaufman Secretary General Rasha El-Khadem
Name:
Role:
Benjamin Kaufman
Secretary General
Jonathan Taylor
Publication Specialist
Nadine Ettel
TCOM Coordinator
Rasha El-Khadem
Head of Accreditation Dept.
Branislava Opra
System Auditor
Florina Palada
Patricia Muschick
Head of Administration Dept.
Cannice Gubser
Membership Administration
Agnes Hegedüs
Sales- and Office Administrator
Sejal Patel
Administrator
Olga Stockli

6. ISTA Technical Committees:
ADVANCED TECHNOLOGIES COMMITTEE
BULKING AND SAMPLING COMMITTEE
FLOWER SEED COMMITTEE
FOREST TREE AND SHRUB SEED COMMITTEE
GERMINATION COMMITTEE
GMO COMMITTEE
MOISTURE COMMITTEE
NOMENCLATURE COMMITTEE
PROFICIENCY TEST COMMITTEE
PURITY COMMITTEE
RULES COMMITTEE
SEED HEALTH COMMITTEE
STATISTICS COMMITTEE
SEED STORAGE COMMITTEE
TETRAZOLIUM COMMITTEE
VARIETY COMMITTEE
VIGOUR COMMITTEE
EDITORIAL BOARD (SST)

7. Why is ISTA relevant to seed stakeholders?
The vision of ISTA:
Uniformity in
Seed Quality Evaluation
Worldwide

8. International standardization of reporting results
International standardization of methodologies
International standardization of laboratory performance
International exchange of scientific results

9. International exchange of scientific results: ISTA Publications and Symposia
Editing and issuing the Scientific Journal: Seed Science and Technology
Organize Seed Symposium as part of the tri-annual ISTA Seed Congress

10. International standardization of reporting results
International standardization of methodologies
International standardization of laboratory performance
International exchange of scientific results

11. 2013: New Chapter added to the ISTA Rules!!
Chapter 19-GMO Testing
International Standardization of Methodologies: The International Rules for Seed Testing -> ISTA has been publishing the international rules for seed testing since 1933.
Most recently Rules chapter: 19 - GMO testing
The ISTA Rules can be modified and amended annually Proposals to modify or amend current ISTA International Rules for Seed Testing can be made by: members, non-members, public/private laboratories, persons working in the area of seed science and technology.

12. Chapter 19: GMO Testing Objective: Why develop a new chapter?
To formalize the ISTA approach for GMO testing within the ISTA Rules
To describe a system that will ensure uniformity in GM seed testing
Why develop a new chapter?
ISTA has been working to address challenges related to testing GM seed since 2001!
GMO testing is an increasing activity for seed trade
Laboratories need guidance about ISTA‘s principles and technical aspects of GMO testing.
Answer ISTA‘s strategy decision (2001: to establish a GMO chapter)
Key Considerations:
This chapter remains general on all aspects of GMO seed testing
This chapter is based on already existing ISTA‘s papers
Doesn‘t provide protocols (follows the Performance Based Approach)
Is linked to a Handbook for the detailed technical considerations (in development!)
Is not creating new ISTA position / statement about GMO analysis
Established TF in 2001, incorporated process for “sprecified trait testing” into chapter 8… TF became Technical Committee in 2010, rules development followed from ; Came following 10 year’s of experience in working within the system and recognizing what worked and where gaps were. Next most important step is development of the handbook – which provides technical guidance for labs wanting to establish gmo testing and becom eaccreditted.

13. International standardization of reporting results
International standardization of methodologies
International standardization of laboratory performance
International exchange of scientific results

14. International Standardization of Laboratory Performance: The ISTA Accreditation Program:
Open to : Seed Testing Laboratories from the public and the private sector
official language: English
all relevant documents can be downloaded from the ISTA website at

15. International standardization of reporting results
International standardization of methodologies
International standardization of laboratory performance
International exchange of scientific results

16. International Standardization of Reporting Results: Facilitation of International Seed Trade
ISTA Certificates have international recognition
Indicate the sampling and testing has been carried out in accordance with the ISTA Rules
Sampling and testing methods used have been validated, internationally harmonised and voted on by the ISTA membership.  
The International Seed Analysis Certificates are available only for accredited member laboratories of ISTA. They must only be issued in accordance with the ISTA Rules currently in force.
Blue Certificates are test reports for seed samples. They do not represent a seed lot; therefore, sampling is the responsibility of the customer. We can issue these certificates on any sample, regardless of origin or lot location.
Orange Certificates are test reports that represent whole seed lots. To ensure that the certificate accurately represents the seed lot, the lab is responsible for both sampling and testing.

17. International standardization of reporting results
International standardization of methodologies
International standardization of laboratory performance
International exchange of scientific results

18. The ISTA Accreditation Programme
ISTA Rules
ISTA Accreditation Programme
Proficiency Testing
120 participating laboratories world wide
The laboratory‘s performance is directly linked to its accreditation status
Three test rounds per year
ISTA Membership
Q-Audit
Seed Testing Laboratory
ISTA Certificate
Quality assurance system according to the ISTA Accreditation Standard
ISTA audit every three years
Audit by two ISTA auditors

19. The ISTA Proficiency Test Program
Generally: you are accredited  obligatory, not accredited  voluntary
What they all have in common: measure competency.
We are not talking about comparison tests to evaluate a method.
Voluntary and obligatory - explain
Basic Tests
Seed Health Tests
GMO Tests

20. The ISTA Proficiency Test Program
Goal: measure and monitor laboratory‘s proficiency
Proficiency Test fee is covered in yearly Membership fee
Participation is mandatory for accredited labs, for the tests/crop group/events for which they have been accredited
Voluntary participation possible for all member laboratories, regardless if accredited or not
Non members are welcome to participate (service fee will be charged)
Why voluntary participation?
Gain experience with other species
Compare your laboratory‘s testing performance with the performance of accredited laboratories
Your participation will be rated, but the rating has no influence on your accreditation

21. The ISTA Proficiency Test Program:
Several rounds per year
Labs test ‘blind’ samples in accordance with determined methods*
Submission of results
Review and Statistical evaluation
Evaluation of the laboratory‘s proficiency
Detection of systematic errors and performance trends
Results influence accreditation status (only labs for which participation is mandatory)
Accreditation maintained? þ

22. The ISTA Proficiency Test Program: Ratings
Ratings are assigned: A, B, C and BMP (Below Minimum Performance) for each test in each round
After six rounds of participation calculation of overall rating
Only mandatory test results are included in overall rating
Performance of a laboratory influences their accreditation:
Overall rating of C  warning letter
Overall rating of BMP  suspension
Failure to participate in 2 rounds out of 6  suspension
A & B: good ratings, C and BMP require CA

23. GMO Proficiency Testing: Challenges!
ISTA’s performance based approach for GMO testing means the way development and execution of a PT used for Basic Seed Tests and Seed Health Tests must be modified in order to have a Proficiency Testing program for GMO testing

24. GMO TESTING: Performance Based Approach
The ISTA approach has targeted the uniformity in GMO testing results, not by the uniformity in testing methodology, by using a “Performance Based Approach (PBA)”.
The “Performance Based Approach” requires that laboratories demonstrate that the GMO detection, identification or quantification methods they are using on seed samples for reporting results on the ISTA International Seed Lot Certificate, meets acceptable standards set by ISTA.
This standard includes, among others, sampling, reporting and test accuracy.
This requirement will ensure reliability of the analysis and the final test result reported on the ISTA International Seed Lot Certificate.
The Performance Based Approach gives seed testing laboratories the choice to use different technological approaches, e.g. bioassays, protein based methods and DNA based methods. 24 24

25. What does this mean?
Basic Seed Tests and Seed Health Tests must use ISTA approved protocols… but….
…the Performance Based Approach gives seed testing laboratories the choice to use different technological approaches, e.g. bioassays, protein based methods and DNA based methods.
The GMOPT must be able to accommodate evaluation and rating of results obtained from many test types!

26. GMO PT Program basics: 1 species per round, 1-3 Events
Qualitative and Quantitative evaluations
Spiking levels between 0-4%
10-12 samples of seeds each
Two panels per year offered
Matrix is SEED

27. GMO PT19: Design of Round

28. Announcement and Technical Protocol:
Announcement is made on the ISTA Website and also sent to member labs via e.mail
Technical protocol provides information needed to develop the test plan
Reporting sheets are provided for results to be returned

29. ISTA GMO PT Database
Data from the past 20 rounds are contained. The data is evaluated to improve the program and also by a group within the publications WG of the GMO TCOM for future publication.
The PT process is tracked and executed through the use of a specially developed Access database which is used for registration to sample prep, shipping, participant tracking over time, data recording and analysis, etc.

30. Results Evaluation: Qualitative Determination
The rating for the presence/absence ( yes/no - qualitative) results is based on a % of misclassified samples/total samples.
Misclassification may either be a false positive or a false negative result.
Missing results for individual samples are considered as misclassification.

31. Results Evaluation: Quantitative Determination
The quantitative rating is based on the quantification results for the positive samples and their reference value (samples with zero spiking level are not used in quantification rating).
The reference value is either the # of GM seeds in %, the mass of the GM seeds in % or the median of the results reported by the participants in the unit ‘%DNA copies’.
Which of the reference values is chosen is determined by the panel of experts appointed for each round, the guiding principle being:
Sub-sampling quantification: %number
Results reported in %number: %number
Results reported in %mass: %mass
Results reported in any other unit, such as number DNA copies: median of the results

32. ISTA GMO Proficiency Test – Ratings
The rating system is based on the system established for basic and seed health tests which are accredited by ISTA – A, B, C, BMP
A No problem has been detected in this test
B There are small problems, but no specific look or action is suggested to the participant
C Problems, ISTA indicates there might be things to consider by the laboratory to explain or correct things
BMP Below Minimum of Performance, ISTA indicates that the results were poor and the laboratory need to find explanations and to improve/correct

33. GMO PT19: Results/Evaluation
Sample sets were sent to 46 laboratories; 39 participants submitted results; 8 provided qualitative results only
For this PT round, a change in the procedure has been made in order to better take into account recognized standards regarding the precision of a quantitative result. Therefore, in the definition of the Reference intra-laboratory standard-deviation used in the computation of the z-scores, we have introduced the following change (using the same notations as in the reference document in ISTA News Bulletin No. 130): If (reference lab stdev)/(true level) is lower than 0.25, use 0.25 x (true level) as the Reference intra-laboratory standard deviation.

34. GMO PT: Qualitative and Quantitative Rating Systems
Procedure updated
STI 128 – 2004:
STI 130:-2005:

35. Success of the Performance Based Approach for GMO Testing within ISTA:
13 laboratories are accredited for GMO Testing
Performance Trends:
Qualitative Detection
Quantification
Quantitation results not necessarily due to poor performance of labs, but seems to be due to increased accuracy – a statistical artefact of a low z-score (JLL/KR)

36. GMO PT Program: Participant Stats:
About half the participants are ISTA labs
The majority of participants are from Europe

37. 1 round of the GMO PT program:
Complete process:
1 round of the GMO PT program:
The execution of a single round of the GMO PT Program is complex and involves significant organization, coordination, and administration from all parties involved in it’s delivery. Steps include:
1. Program Planning and Seed procurement
2. Purity testing of sample seed material
3. Announcement of PT Round
4. Initiation of a new PT in the PTDB
5. Registration of participants and invoicing
Adding to the complexity – there are always 3 PT rounds ongoing at any time, all at different stages of completion!
6. Sample preparation and shipment
7. Participant Results
8. Evaluation of Results and Ratings
9. Generation of Final results reports
10. Record keeping

38. How is this work accomplished?
Working Groups
Lead
Members
Work Description and Projects
GMO Proficiency Test: Administration and Sample Preparation
ISTA TCORD
Nadine Ettel
Christoph Haldemann, Jean-Louis Laffont and Cheryl Dollard (Expert Team); Andrea Jonitz (Sample Prep)
Purpose: Execution of the GMO PT ; Lead: Coordinate administration of the PT program from within the ISTA secretariat;
Expert Team: Develop PT Plans, Evaluate Results and Ratings, address technical questions, assist with administration of program as described in the SOP for administration of the GMO PT Program.
Sample prep: Purity check of material and preparation of PT Samples as described in the GMO PT Program Sample Prep SOP
GMO Proficiency Test:
Resource Procurement
Ray Shilito
Elizabeth Bates (Bayer), Bruno Zaccomer (Monsanto), Sofia Ben Tahar (Limagrain),
Purpose: Resource procurement in support of the GMO PT
Program: Secure reference materials (GM and non-GM) from Industry; Coordinate financial support for PT program from Industry/corporate partners; Action items: solicit WG members from other Industry areas: Syngenta, Pioneer, etc
Secretariat contact: ISTA TCOM Coordinator (Nadine Ettel)

39. GMO PT Program: Procedures established
Two SOPS developed describing all steps required to execute a single PT round:
“Administration of GMO PT program”
“Sample preparation for GMO PT program”
Significant challenges:
Sourcing GM and Non-GM Seed;
Funding to cover associated costs
Finding labs willing to prepare the samples

40. One Round of the GMO PT ..Mostly!
K. Studer

41. Sample Preparation A sample preparation laboratory
prepares the blind samples, packages
and ships lots to participants.
Requires Seed Lot Purity Checks: GM and Non-GM Seed
Preparation and blind coding of the spiked samples (10 samples x 50 participants – 500)
Arrangement of phytos
Rounds 1-19: LTZ (Germany), Round 20: Eurofins

42. Expert Team
Group of experts who are involved in statistical analysis and interpretation of the results for each round
They address the activities which cannot involve other committee or WG members in order to ensure secrecy of the program, and to protect confidentiality of participants and results.
Includes: finalizing details and logistics to addressing questions and issues that arise and are unique in each PT.

43. Expert Team: A snapshot…
Development of PT test plans (computing percentages/spiking levels/seed requirements/statistical verification).
Provision of support for Administrative functions carried out in the Secretariat as required (preparation/review of documents/files, and database use, generation of reports, etc).
Preparation of the PT schedule
Coordination of activities and problem solving issues related to seed procurement, including addressing Industry questions regarding seed used for the PT program.
Verification of seed check procedure; analyse data for approval of lots; revision of test plans when necessary.
Provided input for development of new GMODB; beta tested versions during development process.
Review of raw results, ratings computation and determination of final ratings.
Provision of advice and answers to the Secretariat, Sample prep lab and members with questions.
Review of data for trends that signal potential issues which could require modification of the Program.
** “Corporate Memory”
** Addressing issues related to the PT: These are often complex and require technical analysis, research, etc.

44. # spiking levels (incl. 0)
20 rounds and counting… PT
Crop
# of Events
# of participants
# of samples in a set
# seeds per sample
# spiking levels (incl. 0)
PT01
Maize 2 46 30
300 4
PT02 1 50 10
3000 3
PT03 42 12
1500 7
PT04
Soybean 52
PT05 58 5
PT06
Canola
3300
PT07 67
2000 6
PT08 53 14
PT09
Maize* 64
12+2
2000/50 11
PT10
Cotton**
2+1
2500 9
PT11 49
PT12 60 8
PT13 31
PT14 40
PT15
Flax**** 35
PT16
Maize*****
1800
PT17 45
PT18
Maize+ 48
PT19
Soybean+
PT20
Cotton++
in progress
PT09 Included Trait Purity samples- 12 samples for AP, 2 samples for Purity
** PT10 Cotton round used 3 GM varieties - 2 varieties were single Event and 1 variety was a double stack (stack included same Events as those in the single Trait varieties)
*** # of spiking levels calculated to include 0%; double spiked samples (i.e. those which had 2 Events spiked into the same sample) are calculated as 2 spiking levels - this is to get an accurate estaimate of the work involved
**** Seed material (GM and NON) as well as purity testing was done by CFIA - no charge to ISTA
***** non-GM seed had to be replaced, Lot and purity testing was done by Bayer - no Charge to ISTA
+ Seed for PT18 and 19 - most purity testing was not required as leftover seed from PT16 and 17 was able to be used again
++ Seed for PT20 - purity testing to be done by Provider - no charge to ISTA

45. Ordinary Meeting Executive Committee Secretariat
Voting delegates from 72 Member Governments
Executive Committee
11 Members from all Continents of the World nominated by Governments
Secretariat
11 professional employees
18 Technical Committees and Boards
Over 200 experts from all over the world working voluntarily
in the Technical Committees
Experts from the public/private sectors, from member/non-member laboratories

46. Conclusion
ISTA GMO PT program is a unique tool for achieving uniform and reliable seed quality determination for GMO testing
It is possible to evaluate and compare test performance internationally in the dynamic field of GM testing internationally while still achieving uniformity of results reporting that is ISTA’s aim.

47. Thank you for your attention

48. Questions ?