Dissertation Status By: R. Bassett May 29, 2003 Prepared as 05/30/03 meeting agenda with Fred & Chuck

Statistics - Which way to go? Descriptive Statistics simply describes what the data shows and is used to present quantitative descriptions in a manageable form and may include the use of graphics analysis. Inferential statistics attempts to reach conclusions that extend beyond the immediate data alone. For instance, we use inferential statistics to try to infer from the sample data what the population might think. Inferential statistics is used to make inferences from our data to more general conditions and descriptive statistics is used to simply to describe what's going on in our data. I plan to use Descriptive Statistics in this research study

The Wrapper Objectivity (Science) – based on hard facts – machine-based system – technical grade Subjectivity (Art) – based on experiences, perceptions and some hard facts – Online Grading – market grade Essentially what I went through during the 05/09 presentation, I would like to get feedback from all committee members on this tie-in

Machine Based Experiments Hypothesis: The larger the trained database that the machine was working from the better the machine would perform. Perform 3 experiments (A,B & C) to test this hypothesis

Machine Based Experiments As the number of images in the database increased in these 3 experiments the error rate decreased

Machine Based Experiments The hypothesis was tested further in tests D & E Test D – is the most exhaustive & most rigorous of the 5 tests Compared to experiments A, B & C

Machine Based Experiments The hypothesis was tested further in tests D & E Test E – Also exhaustive, it compared the coins graded by the 3 rd party grading services Compared to experiments A, B & C

Online Grading Experiments Experts Acceptance Testing #1 Hypothesis: The correlation will be lower than seen between the machine grades and those of the services on the twenty selected coins. Compared to experiments A, B & C The substantive reason is that subjective factors are now brought into active consideration. The statistical reason is that with a greater number of human grades, there is bound to be those with idiosyncratic standards.

Online Grading Experiments Advance Grade Knowledge Testing #2 Hypothesis: Online grading experts who believe that they are provided with a reliable machine-based grade will do better at grading than those graders that are not provided an accurate grade. The substantive reason is that the expert graders should be able to use the grade provided as a reasonable starting baseline or starting point. Lowest Highest

Online Grading Experiments The Importance of Subjective Qualities Testing #3 Hypothesis: Identifying which subjective qualities of importance that are used most by the expert graders will yield less than impressive statistical significance. Expert graders often claim that grading is an art and not a science by referring to certain subjective qualities that are difficult to repeat and to measure on a repeatable basis. The vast majority of graders failed to identify the incorporation of subjective qualities into their decision process

Online Grading Experiments Grader Consensus Testing #4 Hypothesis: Expert grading consultants will have widely diverse grading opinions on the images that they are grading. Informal experience that humans grade inconsistently is supported by the Stujoe grading tests, Coin World tests, Kevin Foley tests on human grading.

Online Grading Experiments Duplicate Grade Testing #5 Hypothesis: Given that it was previously hypothesized that humans provide widely divergent grading opinions it is expected that graders will offer differing grading opinions on coins which they have previously graded. The expectation was that there would be a certain amount of change that occurred in the grading of the duplicates but the hope was that the change, or variance, would not be significant. Lower grade coins generally provide more challenges to graders

Online Grading Experiments Perception of Internet based Grading Test #6 Hypothesis: Expert graders would see the value in grading images over the Internet although they would rather grade from the actual coins. The graders that participated in this test are predisposed to the use of technology as they are on the Internet.

Bringing this to conclusion Discussion Points Tasks to do still Feedback on manuscript Defense – no surprises Timetable