1. LC Quality Assurance Services Provided by LCBO Quality Assurance Leading Sensory Evaluation Services

2. LC Quality Assurance VQA Mandate Appellation of Origin System Quality Standard Control of use of specified terms, descriptions and designations associated with the VQA appellation system VQA Appellation System

3. LC Quality Assurance Sensory Quality Chemical Composition Microbiological Stability Packaging and Labelling Standards Role of Sensory Evaluation Testing Quality of Beverage Alcohol Products

4. LC Quality Assurance Free of technical faults / defects Typicity of varietal character – for varietal wines A wine bearing varietal designation shall be assessed to determine if the varietal designation for the wine exhibits the predominant character of a wine produced from the designated grape variety/varieties (VQA Rules) Typicity of the wine category: Late Harvest, Icewine, Nouveau, Sparkling – Traditional Method, Icewine Dosage, Botrytized Wine (VQA Rules) Role of VQA Sensory Evaluation Testing Scope

5. LC Quality Assurance A wine shall be deemed to have passed the taste test if a majority of the members of the Tasting Panel determine: That the wines attributes fairly reflect the viticultural and oenological quality standards established in O.Reg. 406/00 (Rules) without defects or flaws; and That the wine is representative of quality wines of the stated category (VQA Rules) Role of VQA Sensory Evaluation Testing

6. LC Quality Assurance Grading Panel – 4 panel groups of 5 panellists each 28 Members (20 regular panellists, 8 alternate tasters) Sensory Panels

7. LC Quality Assurance Panel members are LCBO Product Consultants who work in retail stores within metropolitan Toronto area Continuous training/ development opportunities LCBO Product Knowledge I, I, III Certification from the Wine and Spirit Education Trust Master of Wine Certification Panel Member Selection Criteria

8. LC Quality Assurance Panel members are subjected to annual : Sensory evaluation testing Product knowledge testing 25 % of the questions are VQA specific Demonstrated professionalism – Sensory Evaluations Code of Conduct Panel Member Selection Criteria

9. LC Quality Assurance Consistent Performance Balanced Panels Reproducible Results Panel Groups Assembly Criteria: Results from the annual testing Testing performance history Previous experience (as a Grading Panel member, from the industry) Performance monitoring data Sensory sensitivities

10. LC Quality Assurance Training Elements Aroma, flavour recognition Product category recognition Varietal character recognition Regional character recognition Winemaking techniques Technological influences Defect identification Panel Training

11. LC Quality Assurance Blind sample presentation: Samples are presented without identifying markings Uniform sample presentation Sample Presentation Sample information: Varietal composition, vintage year, wine category, method of production (where applicable: sparkling wines, icewine dosage, etc)

12. LC Quality Assurance Sensory Evaluation Grading System Sensory Evaluation Method

13. LC Quality Assurance Superior objectivity – system design eliminates bias Wider system applicability– system design equally applies to all wine categories Efficient and effective calculation of results – time savings, accuracy Data Analysis Tools & Traceability of results – enabled by automated data collection and quantification of the results Objectivity Applicability Efficiency Client Satisfaction Sensory Evaluation Grading System

14. LC Quality Assurance Sensory evaluation grading system elements: New sensory evaluation grading method Sensory Evaluation Grading System Integrating technology: Automated data collection Automated data quantification, analysis and reporting

15. LC Quality Assurance Appearance and Colour Aroma (primary and secondary) and Bouquet Taste Harmony Sensory characteristics applicable to all types of beverage alcohol products. Appearance Colour Harmony Bouquet Taste Aroma Sensory Evaluation Grading System

16. LC Quality Assurance Sensory characteristics and attributes are evaluated using five grading categories. Sensory Evaluation Grading System CategoryExcellentVery GoodGoodSatisfactory Unsatisfactory Quality Level Outstanding /Exceptional Superior/ Very Correct TypicalWeak/Not at full potential Faulty / Defective

17. LC Quality Assurance Characteristics & Attributes Grading Categories Commen ts Excellen t Very Good GoodSatisfactor y Unsatisfacto ry Appearance & Colour x Aroma& Bouquet Correctne ss x Intensityx Qualityx TasteCorrectne ss x Intensityx Finishx Qualityx Harmonyx Grading Form

18. LC Quality Assurance Data collection Data quantification Data analysis Data reporting Innovative Technology

19. LC Quality Assurance Automated Interactive Flexible-Wireless Pen or/and keyboard Handwritten notes Data Collection – Tablet PCs

20. LC Quality Assurance

21. Quality Sensory Evaluation Result System Controls Proficiency Programs Verification Tasting Performance Monitoring Quality Assurance QA of Sensory Evaluation Results

22. LC Quality Assurance QA of Sensory Evaluation Results VQA Tasting protocol (VQA Rules) 2 nd Bottle tasting Grading System Controls: Security of panellist registration Forced completeness of the assessment Ratings cannot be changed, once finalized Sample evaluation cannot be redone, once completed Validation questions in the grading questionnaire Calculation validation based on statistical measures Automated calculation and reporting

23. LC Quality Assurance QA of Sensory Evaluation Results Verification Tasting - an independent sensory evaluation of the products in conditions identical to the tasting panel. The verification process has a quality assurance role and provides a reference value that is then compared with the panel results to identify any discrepancies in the assessment. Verifiers – qualified Quality Assurance tasters. A tasting session may require up 2 Verifiers. Verifiers results are typically not included in the calculation, unless discrepancy situation.

24. LC Quality Assurance QA of Sensory Evaluation Results Panellists performance is monitored for each tasting session Measures: Outliers Frequency, % - measure of rating accuracy Rating Rank – measure of rating bias (high, low, trends) Sensitivity data - missed defects, good detection, hyper-sensitivity Feedback on performance – Feedback Report Provides tools to help identify training opportunities

25. LC Quality Assurance Panellist Feedback Report QA of Sensory Evaluation Results

26. LC Quality Assurance QA of Sensory Evaluation Results Internally designed to measure quality and consistency of the sensory evaluation assessments Designed to measure: System performance Panel Performance Panellist Performance Typical measures: repeatability, reproducibility, bias, defects identification, etc. Panellist Panel System

27. LC Quality Assurance Internal Proficiency Testing

28. LC Quality Assurance Repeatability Internal Proficiency Testing

29. LC Quality Assurance Internal Proficiency Testing Reproducibility No significant difference between reported results, i.e. median values (p-Value: 0.649671) at a 95% confidence level

30. LC Quality Assurance Reproducibility Internal Proficiency Testing No significant difference between panels' (i.e. all 5 panels) reported results (i.e. median value), at a 95% confidence level (p-value = 0.99605)

31. LC Quality Assurance Sensory Evaluation Grading System Summary A superior method for performing sensory evaluations. Eliminates many sources of bias. Customized for use with all beverage alcohol products. Flexibility (data collection, quantification & reporting). Provides significant time savings in data management. Provides tools to analyze panellist results to help identify training opportunities.

32. LC Quality Assurance Leading the Sensory Evaluation of Beverage Alcohol