Service Team Organizational Chart – 1 Year Ago, <20 FTEs

Now with > 200+ FTEs, Hieararchies and Divisions are Inevitable

Exponential Complexities in Operations Division

Case Study : Service team Promotions Problem

Attribute agreement analysis can be utilized to assess the consistency of QA scoring

Based on the latest excercise of QA team, the value of coefficient is far from satisfying at 0.66

There are three possible causes on why this issue occured that we may test one by one

Let’s deep dive on the first cause: Experience of the team

We tested the relatively more experienced QA members and it’s found that the coefficient remains low at 0.68

We then conducted exercise to two different groups of the most experienced members vs. the least experienced members

Let’s take a look at the second cause: Standardized matrix

Some questions deserve attention to ensure better standardized variables matrix

Let’s take a look at the third cause: Scoring logic

We found the calculation can be improved to make it less bias

Steps to improve QA agreement and assessment

STEP 1: FGD to determine categories & attributes

STEP 2: Determine weighting score for more reliable calculation logic

STEP 3: Make new scoring template

STEP 4: Sampling and Making Detail Guideline

STEP 4: Sampling and Making Detail Guideline

New Guideline is developed with clear scoring description and examples

Conclusion

Other People Operation Projects...

Semangat Gan! Makasih