1. 2004 2009 India Emerging Markets Conference, May 2009 (1) Leigh Walton Animal Improvement Programs Laboratory Agricultural Research Service, USDA Beltsville, MD 20705-2350, USA Leigh.Walton@ars.usda.gov Milk Recording Data

2. India Emerging Markets Conference, May 2009 (2)Leigh Walton 20042009 Topics  Milk recording data  Receiving  Processing  Validation

3. India Emerging Markets Conference, May 2009 (3)Leigh Walton 20042009 AIPL Data Sources  Dairy Records Processing Centers (DHI)  Herd test-day - format 14  Lactation – format 4  Calf/heifer – format 4  Reproductive – format 5  Health – format 6  Calving ability – format CES

4. India Emerging Markets Conference, May 2009 (4)Leigh Walton 20042009 AIPL Data Sources (cont.)  Breed Associations (PDCA)  Pedigree data – format 1  Type appraisal score – format 7  Recessive codes – Holstein USA  NAAB  AI sire cross reference – format 395  Marketing status Codes A, G, F

5. India Emerging Markets Conference, May 2009 (5)Leigh Walton 20042009 Frequency of Receipt  Breed Associations  Varies – weekly to evaluation cutoff  Dairy Records Processing Centers  Daily  NAAB  Evaluation cutoff

6. India Emerging Markets Conference, May 2009 (6)Leigh Walton 20042009 Data Exchange Method  All data exchanged electronically  Standardized file naming convention  Computer optimized format  Files compressed via PKzip (Winzip)  Staging area - AIPL’s FTP server Authentication required

7. India Emerging Markets Conference, May 2009 (7)Leigh Walton 20042009 Records Validation/Processing  Automation  Scan for data Data type determination  Standard naming convention is key  Job stream built  Job stream handles multiple data types  Job stream(s) processed Data Validation  Stand alone edits  Limits  Industry standards and definitions  Consistency with stored data  900 unique error codes Additional job streams initiated  Seek alternative pedigree sources Calculate 305-day projection – Best Prediction Process error records

8. India Emerging Markets Conference, May 2009 (8)Leigh Walton 20042009 Error records  Errors and conflicts stored in a record and returned to processing center to assist in data correction  Reject – record rejected  Notify – input record accepted but a problem may exist  Change – input record changed to match master

9. India Emerging Markets Conference, May 2009 (9)Leigh Walton 20042009 Error records (cont.)  Stored to assist in answering queries  Sometimes forwarded by processing center to milk-recording supervisor or producer for action  Rejected records also available by query on web site – http://aipl.arsusda.gov

10. India Emerging Markets Conference, May 2009 (10)Leigh Walton 20042009 Error records query

11. 2004 2009 India Emerging Markets Conference, May 2009 (11) Leigh Walton Animal Improvement Programs Laboratory Agricultural Research Service, USDA Beltsville, MD 20705-2350, USA Leigh.Walton@ars.usda.gov Processing of data discrepancies for U.S. dairy cattle and effect on genetic evaluations

12. India Emerging Markets Conference, May 2009 (12)Leigh Walton 20042009 How Data impacts Evaluation Accuracy  Accuracy of a recorded trait  Example: milk weight  Emphasis and adjustment  Example: milking frequency, milkings weighed  Other animals influenced  Example: parents, progeny, contemporaries

13. India Emerging Markets Conference, May 2009 (13)Leigh Walton 20042009 Pedigree and yield edits  Identification (ID) verified for valid breed, country, and number  Canadian ID verified against Canadian Dairy Network data  Some American ID use last digit as internal check

14. India Emerging Markets Conference, May 2009 (14)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Birth date  Parentage checked (not too young and not too old for progeny)  Matched to dam calving date Differences of <1 month allowed Omitted if embryo-transfer animal

15. India Emerging Markets Conference, May 2009 (15)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Birth date (cont.)  Parents not previously in database added with estimated birth date 3 years before reported animal’s birth date Revised as data from older siblings received

16. India Emerging Markets Conference, May 2009 (16)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Alias detection  Same birth date and full siblings but not twins  Within-herd ID (cow control number) useful in identifying additional ID  Bulls registered in >1 country common cause

17. India Emerging Markets Conference, May 2009 (17)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Alias detection (cont.)  Numbers differing by single digit investigated as possible invalid ID  Yield data must not conflict for the data from the 2 IDs to be combined as the same cow

18. India Emerging Markets Conference, May 2009 (18)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Yield  Values outside widest range rejected  Values outside more narrow range stored but changed to a floor or ceiling if used  Cow test date checked against herd test date

19. India Emerging Markets Conference, May 2009 (19)Leigh Walton 20042009 Pedigree and yield edits (cont.)  Calving date  Cannot overlap previous lactation  Missing calving date may cause breeding to be associated with previous calving

20. India Emerging Markets Conference, May 2009 (20)Leigh Walton 20042009 Editing principles  Data either rejected or modified when errors encountered  Effect of rejection  Loss of possibly valuable information  No genetic evaluation for animals of interest  System designed to retain data whenever possible  Data elimination preferred to retention of conflicting data

21. India Emerging Markets Conference, May 2009 (21)Leigh Walton 20042009 Example  Animal’s birth date conflicts with dam’s calving date  Both animals already have data in system  Dam ID removed to resolve conflict and to allow records for both animals to remain in database

22. India Emerging Markets Conference, May 2009 (22)Leigh Walton 20042009 Data Quality Assurance  Test herd summarization  Field by field comparison of DRPC data Tolerances for some fields  Types Data sent to AIPL – format 4, 5, and 14  Consistency between formats DRPC exchanged data – STF formats  WEB access  AIPL does not make judgments Reporting function only

23. India Emerging Markets Conference, May 2009 (23)Leigh Walton 20042009 Conclusions  Evaluation accuracy dependent on accuracy of all contributing data  Invalid records diminish accuracy of evaluations for other animals

24. India Emerging Markets Conference, May 2009 (24)Leigh Walton 20042009 Conclusions (cont.)  Highly complex system for checking data used in national U.S. genetic evaluations of dairy cattle  Conflicting data from various sources  Harmonized based on which data are expected to be most accurate  Deleted when necessary