1. TTB M 1334.01 (04/2010) Methods Criteria and Performance Patricia Nedialkova, Ph.D., TTB International Wine Technical Forum May 7, 2015 Prepared for the International Wine Technical Forum and presented for purposes of discussion. This is not an authoritative statement of agency policy or guidance.

2. Fit for Purpose Methods Action Items from 2014 Forum Evaluate CTS proficiency test (PT) data and TTB Certified Chemist data against CODEX fit for purpose method criteria Compare CODEX with OIV criteria Develop plans for a Forum PT Support Principle 9: Labs use methods that are validated for wine and are proficient in their use

3. CODEX Fit for Purpose Method Criteria Define acceptability criteria for regulatory analytical methods – Limit of detection and quantitation (LOD and LOQ) – Range – Precision - % Relative Standard Deviation (%RSD) – Accuracy - % Recovery Internationally recognized Developed by consensus Ensure confidence in regulatory results Avoid trade disputes and economic loss

4. Comparison OIV Reference methods Multi-lab Validation One size fits all approach – impedes new method development – doesn’t guarantee that all labs get the same answer CODEX Any method that meets the performance criteria Performance Criteria Flexibility for labs with different – sample throughput – budget

5. TTB Method Performance Evaluation – Alcohol by Distillation/Density Example

6. TTB Methods, Regulatory Limits & Measurement Uncertainty (MU) All TTB test methods, and the regulatory limits they are used to enforce, are published on our website: http://www.ttb.gov/ssd/methods.shtml MU is reported alongside regulatory analytical results and is taken into account when determining compliance Supports Principles 7 and 11 and ISO 17025 MU requirements

7. Evaluation of Alcohol PT Data Compiled PT data by method – Pat Howe provided 13 years of Collaborative Testing Services (CTS) data – Eric Wilkes provided Interwinery Analysis Group (IWAG) data Noticed differences in method performance – Accuracy – disagreement on true value – Precision – variability of the results

8. Alcohol CTS PT Data A Review of Thirteen Years of CTS Winery Laboratory Collaborative Data Howe, P.A., Sacks, G.L., and S.E. Ebeler Am J Enol Vitic

9. Alcohol IWAG PT Data Provided by Eric Wilkes, Ph.D.

10. Workgroup Distillation Recoveries Insignificant losses when run by trained analysts with good quality controls in place Laboratory% Recovery Bronco100.0 Gallo99.9 TTB Compliance Lab99.9 TTB Beverage Alcohol Lab99.8

11. Alcohol Method Comparison Workgroup members ran their alcohol methods side by side, including: – Distillation/density – NIR – GC – FTIR Showed no significant bias

12. Historical Data from Control Charts All at least an order of magnitude below the CODEX requirement of 6 LabTechnique%RSD BroncoNIR0.07 TTB CLDist/Dens0.17 JacksonNIR0.23 JacksonGC0.23 GalloGC0.26 GalloFTIR0.43 TTB CLGC0.59

13. Benefits of Control Charts Gives real time method performance data, as opposed to the snap shot at validation, that can be used to: – Troubleshoot trends before they become quality failures – Calculate realistic measurement uncertainties for interpreting results’ compliance Provides an independent check on standards and calibrations Meets ISO requirements for demonstrating your method and equipment are under control

14. Quality Tips Ensure minimal distillation losses Optimize sample and water volumes for distillation in your apparatus Use pure standards Peg calibrations of secondary methods back to primary methods Monitor a wine laboratory control sample using statistical control charts Participate in an international PT

15. The APEC PT APEC funded Provided by the Interwinery Analysis Group Analytes include: alcohol, reducing substances, glucose and fructose, total sulphur dioxide, total acid, copper, iron, manganese, and methanol Results to be broken out by method

16. Workgroup Next Steps 1.Invite each regulatory lab to participate on the workgroup Current members include: – Argentina – China’s Shanghai CIQ – Italy’s UIV – USA’s TTB

17. Workgroup Next Steps 2.Assist workgroup members to a)Establish control charts b)Determine measurement uncertainty c)Evaluate their method performance against the CODEX criteria (Supports Principles 7, 9, and 11)

18. Workgroup Next Steps 3.Populate compendia with a)Regulatory levels b)Validated methods used by regulatory labs c)Method performance as implemented at the regulatory lab, including: i.LOD/LOQ for trace methods ii.% Recovery for contaminant methods iii.Range iv.%RSD v.MU (Supports Principles 7, 9, and 11)

19. Workgroup Next Steps 4.Create a table of CODEX criteria at the relevant levels for the common regulatory analyses and methods know to meet those criteria (supports Principle 9) 5.Propose amending this workgroup’s name to be: “Analytical Method Quality” if the above is to be our scope

20. Many thanks to our workgroup members! We welcome your questions and open the floor for discussion