2004 2009 India Emerging Markets Conference, May 2009 (1) Leigh Walton Animal Improvement Programs Laboratory Agricultural Research Service, USDA Beltsville,

2004

2004

2009India Emerging Markets Conference, May 2009 (1)

Leigh WaltonLeigh WaltonAnimal Improvement Programs Laboratory Agricultural Research Service, USDABeltsville, MD 20705-2350, [email protected]

Milk Recording Data

India Emerging Markets Conference, May 2009 (2) Leigh Walton200

4200

9

Topics

Milk recording dataReceivingProcessingValidation


4200

9

AIPL Data Sources

Dairy Records Processing Centers (DHI)

Herd test-day - format 14Lactation – format 4Calf/heifer – format 4Reproductive – format 5Health – format 6Calving ability – format CES


4200

9

AIPL Data Sources (cont.)

Breed Associations (PDCA)Pedigree data – format 1Type appraisal score – format 7Recessive codes – Holstein USA

NAABAI sire cross reference – format 395

Marketing status•Codes A, G, F


4200

9

Frequency of Receipt

Breed AssociationsVaries – weekly to evaluation cutoff

Dairy Records Processing Centers

Daily

NAABEvaluation cutoff


4200

9

Data Exchange Method

All data exchanged electronically

Standardized file naming convention

Computer optimized formatFiles compressed via PKzip (Winzip)

Staging area - AIPL’s FTP server•Authentication required


4200

9

Records Validation/Processing

Automation Scan for data

• Data type determination Standard naming convention is key Job stream built Job stream handles multiple data types

Job stream(s) processed• Data Validation

Stand alone edits Limits Industry standards and definitions

Consistency with stored data 900 unique error codes

• Additional job streams initiated Seek alternative pedigree sources

• Calculate 305-day projection – Best Prediction• Process error records


4200

9

Error records Errors and conflicts stored in a

record and returned to processing center to assist in data correction

Reject – record rejected

Notify – input record accepted but a problem may exist

Change – input record changed to match master


4200

9

Error records (cont.)

Stored to assist in answering queries

Sometimes forwarded by processing center to milk-recording supervisor or producer for action

Rejected records also available by query on web site – http://aipl.arsusda.gov


4200

9

Error records query

2004

2004

2009India Emerging Markets Conference, May 2009 (11)

Leigh WaltonLeigh WaltonAnimal Improvement Programs Laboratory Agricultural Research Service, USDABeltsville, MD 20705-2350, [email protected]

Processing of data discrepancies for U.S.

dairy cattleand effect on

genetic evaluations


4200

9

How Data impacts Evaluation Accuracy

Accuracy of a recorded traitExample: milk weight

Emphasis and adjustmentExample: milking frequency, milkings

weighed

Other animals influencedExample: parents, progeny, contemporaries


4200

9

Pedigree and yield edits

Identification (ID) verified for valid breed, country, and number

Canadian ID verified against Canadian Dairy Network data

Some American ID use last digit as internal check


4200

9

Pedigree and yield edits (cont.)

Birth date

Parentage checked (not too young and not too old for progeny)

Matched to dam calving date• Differences of <1 month allowed

• Omitted if embryo-transfer animal


4200

9


Birth date (cont.)Parents not previously in database added with estimated birth date• 3 years before reported animal’s

birth date• Revised as data from older

siblings received


4200

9


Alias detectionSame birth date and full siblings but not twins

Within-herd ID (cow control number) useful in identifying additional ID

Bulls registered in >1 country common cause


4200

9


Alias detection (cont.)

Numbers differing by single digit investigated as possible invalid ID

Yield data must not conflict for the data from the 2 IDs to be combined as the same cow


4200

9


Yield

Values outside widest range rejected

Values outside more narrow range stored but changed to a floor or ceiling if used

Cow test date checked against herd test date


4200

9


Calving date

Cannot overlap previous lactation

Missing calving date may cause breeding to be associated with previous calving


4200

9

Editing principles Data either rejected or modified when errors

encountered

Effect of rejection

Loss of possibly valuable information

No genetic evaluation for animals of interest

System designed to retain data whenever possible

Data elimination preferred to retention of conflicting data


4200

9

Example

Animal’s birth date conflicts with dam’s calving date

Both animals already have data in system

Dam ID removed to resolve conflict and to allow records for both animals to remain in database


4200

9

Data Quality Assurance Test herd summarization

Field by field comparison of DRPC data

•Tolerances for some fieldsTypes

•Data sent to AIPL – format 4, 5, and 14 Consistency between formats

•DRPC exchanged data – STF formats

WEB accessAIPL does not make judgments

•Reporting function only


4200

9

Conclusions Evaluation accuracy dependent on

accuracy of all contributing data

Invalid records diminish accuracy of evaluations for other animals


4200

9

Conclusions (cont.)

Highly complex system for checking data used in national U.S. genetic evaluations of dairy cattle

Conflicting data from various sourcesHarmonized based on which data are expected to be most accurate

Deleted when necessary

Documents

2004 2009 India Emerging Markets Conference, May 2009 (1) Leigh Walton Animal Improvement Programs Laboratory Agricultural Research Service, USDA Beltsville,