2004
2004
2009India Emerging Markets Conference, May 2009 (1)
Leigh WaltonLeigh WaltonAnimal Improvement Programs Laboratory Agricultural Research Service, USDABeltsville, MD 20705-2350, [email protected]
Milk Recording Data
India Emerging Markets Conference, May 2009 (2) Leigh Walton200
4200
9
Topics
Milk recording dataReceivingProcessingValidation
India Emerging Markets Conference, May 2009 (3) Leigh Walton200
4200
9
AIPL Data Sources
Dairy Records Processing Centers (DHI)
Herd test-day - format 14Lactation – format 4Calf/heifer – format 4Reproductive – format 5Health – format 6Calving ability – format CES
India Emerging Markets Conference, May 2009 (4) Leigh Walton200
4200
9
AIPL Data Sources (cont.)
Breed Associations (PDCA)Pedigree data – format 1Type appraisal score – format 7Recessive codes – Holstein USA
NAABAI sire cross reference – format 395
Marketing status•Codes A, G, F
India Emerging Markets Conference, May 2009 (5) Leigh Walton200
4200
9
Frequency of Receipt
Breed AssociationsVaries – weekly to evaluation cutoff
Dairy Records Processing Centers
Daily
NAABEvaluation cutoff
India Emerging Markets Conference, May 2009 (6) Leigh Walton200
4200
9
Data Exchange Method
All data exchanged electronically
Standardized file naming convention
Computer optimized formatFiles compressed via PKzip (Winzip)
Staging area - AIPL’s FTP server•Authentication required
India Emerging Markets Conference, May 2009 (7) Leigh Walton200
4200
9
Records Validation/Processing
Automation Scan for data
• Data type determination Standard naming convention is key Job stream built Job stream handles multiple data types
Job stream(s) processed• Data Validation
Stand alone edits Limits Industry standards and definitions
Consistency with stored data 900 unique error codes
• Additional job streams initiated Seek alternative pedigree sources
• Calculate 305-day projection – Best Prediction• Process error records
India Emerging Markets Conference, May 2009 (8) Leigh Walton200
4200
9
Error records Errors and conflicts stored in a
record and returned to processing center to assist in data correction
Reject – record rejected
Notify – input record accepted but a problem may exist
Change – input record changed to match master
India Emerging Markets Conference, May 2009 (9) Leigh Walton200
4200
9
Error records (cont.)
Stored to assist in answering queries
Sometimes forwarded by processing center to milk-recording supervisor or producer for action
Rejected records also available by query on web site – http://aipl.arsusda.gov
India Emerging Markets Conference, May 2009 (10) Leigh Walton200
4200
9
Error records query
2004
2004
2009India Emerging Markets Conference, May 2009 (11)
Leigh WaltonLeigh WaltonAnimal Improvement Programs Laboratory Agricultural Research Service, USDABeltsville, MD 20705-2350, [email protected]
Processing of data discrepancies for U.S.
dairy cattleand effect on
genetic evaluations
India Emerging Markets Conference, May 2009 (12) Leigh Walton200
4200
9
How Data impacts Evaluation Accuracy
Accuracy of a recorded traitExample: milk weight
Emphasis and adjustmentExample: milking frequency, milkings
weighed
Other animals influencedExample: parents, progeny, contemporaries
India Emerging Markets Conference, May 2009 (13) Leigh Walton200
4200
9
Pedigree and yield edits
Identification (ID) verified for valid breed, country, and number
Canadian ID verified against Canadian Dairy Network data
Some American ID use last digit as internal check
India Emerging Markets Conference, May 2009 (14) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Birth date
Parentage checked (not too young and not too old for progeny)
Matched to dam calving date• Differences of <1 month allowed
• Omitted if embryo-transfer animal
India Emerging Markets Conference, May 2009 (15) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Birth date (cont.)Parents not previously in database added with estimated birth date• 3 years before reported animal’s
birth date• Revised as data from older
siblings received
India Emerging Markets Conference, May 2009 (16) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Alias detectionSame birth date and full siblings but not twins
Within-herd ID (cow control number) useful in identifying additional ID
Bulls registered in >1 country common cause
India Emerging Markets Conference, May 2009 (17) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Alias detection (cont.)
Numbers differing by single digit investigated as possible invalid ID
Yield data must not conflict for the data from the 2 IDs to be combined as the same cow
India Emerging Markets Conference, May 2009 (18) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Yield
Values outside widest range rejected
Values outside more narrow range stored but changed to a floor or ceiling if used
Cow test date checked against herd test date
India Emerging Markets Conference, May 2009 (19) Leigh Walton200
4200
9
Pedigree and yield edits (cont.)
Calving date
Cannot overlap previous lactation
Missing calving date may cause breeding to be associated with previous calving
India Emerging Markets Conference, May 2009 (20) Leigh Walton200
4200
9
Editing principles Data either rejected or modified when errors
encountered
Effect of rejection
Loss of possibly valuable information
No genetic evaluation for animals of interest
System designed to retain data whenever possible
Data elimination preferred to retention of conflicting data
India Emerging Markets Conference, May 2009 (21) Leigh Walton200
4200
9
Example
Animal’s birth date conflicts with dam’s calving date
Both animals already have data in system
Dam ID removed to resolve conflict and to allow records for both animals to remain in database
India Emerging Markets Conference, May 2009 (22) Leigh Walton200
4200
9
Data Quality Assurance Test herd summarization
Field by field comparison of DRPC data
•Tolerances for some fieldsTypes
•Data sent to AIPL – format 4, 5, and 14 Consistency between formats
•DRPC exchanged data – STF formats
WEB accessAIPL does not make judgments
•Reporting function only
India Emerging Markets Conference, May 2009 (23) Leigh Walton200
4200
9
Conclusions Evaluation accuracy dependent on
accuracy of all contributing data
Invalid records diminish accuracy of evaluations for other animals
India Emerging Markets Conference, May 2009 (24) Leigh Walton200
4200
9
Conclusions (cont.)
Highly complex system for checking data used in national U.S. genetic evaluations of dairy cattle
Conflicting data from various sourcesHarmonized based on which data are expected to be most accurate
Deleted when necessary