View
260
Download
5
Category
Preview:
Citation preview
Business IntelligenceHow to translate different measurements and scores into
Business Intelligence
Tom Shaw (Capita TI)Paul Mangell (Alpha CRC)Scott Carothers (Kinetic TheTechnologyAgency)
The Goal – Improve our Industry using Business Intelligence
?
What Data? How to capture?
What’s relevant? To whom?
How to interpret?
Adequacy
Fluency
TM Leveraging
User Sentiment
MT Leveraging
Accuracy
Style / Readability
Translator Quality
User Engagement
TER, Bleu, Meteor
Per Word
Per Hour CTR
PE Distance
TAT
Translation Quality – Business Intelligence from a LSP perspective
Tier 3 content
Tier 2 content
Tier 1 content
Support content, non-publishable user driven or reference
Publishable content for user assistance or product support related
Highly visible content, such as Marketing or software localization as
well as legal
Quality Evaluation Triangle
QUALITY
Users
LSPBuyer
The user experience should be equal or better of that achieved in Source language
Customising metrics and QA process in coordination with the
buyer will assure expectations are measured adequately
The buyer’s “internal clients” play a key role in setting up quality
expectations and measuring success
Quality Evaluation - LSP and Buyer
Content type specific QA process and metricsLSP Evaluation Criteria / Model:
Error Typology / Adequacy / Fluency
Territory specific requirements and preferences
Buyer Internal Clients
Quality Evaluation - Users
Users User feedback and content engagement levels
Evaluation Criteria / Model:User Activity / Engagement
Quality Evaluation Models - Thoughts
1) Quality Evaluation Models have to be closely linked to purpose / end use
2) Standard models not always good enough. Customised to specific needs
3) Metrics / Benchmarks should be set with end user in mind
……………………. Who should drive? LSP<>Buyer, Buyer<>Users….
Common Metrics the Buyers Maintain
Most common metrics:
• Which LSP has the assignment
• Delivery Time
• How long did post-editing take?
Most Valuable Metrics for the Buyer
1. Aggregate volume
2. Turnaround time
3. TM Analysis
4. LSP ratings
5. Cost tracking
Enterprise Metrics Dashboard
Checks and balances in collecting and sharing data
• How will the data be collected, analysed, presented?
• Is all data good data?
• Who needs to know which data set?
• What is the cost of collecting, analysing, and presenting data?
• What us the impact of negative or positive data on the enterprise, and the resource/supplier?
• What should be done in the light of the new data?
• Who will take the actions implied/required?
• Who needs to agree that the data strategy is correct? Buy in v apathy/antipathy
Challenges / Considerations / Points to Discuss?• What is a quality level? Who sets / defines it?
• All supply chain to work towards the same goal
• Linking QE results from LSP for different levels with End User Feedback
• External factors affecting the results – e.g. translation quality directly linked to # sales?
• Not all languages can be treated equally – e.g. acceptance of MT in Japan vs Spain
• High quality / perfect really always required?
• Variable quality levels – impact on brand?
• How do high QA scores correlate to user engagement?
• How to customise your metrics based on the communication objectives
• Professional translators offering a ‘less than perfect’ service
Recommended