Upload
ram-yonish
View
443
Download
0
Tags:
Embed Size (px)
DESCRIPTION
how to R&D activities - QA mertics by meira diskin from ceragon
Citation preview
R&D MaturityDiskin Meira
1
Ceragon Networks proprietary and confidential
Agenda
2
�Maturity of What?
�R&D Project Maturity
�R&D Conclusion
�Summary
Ceragon Networks proprietary and confidential
Maturity of What ?
-100-90-80-70-60-50-40-30-20-10
0102030405060708090
100
Week
R&D Maturity
Quality
Coverage
R&D Maturity
3
3 2
36
9 4 7 2 2
34
1 14 51 40
1020304050607080
SW IP
I
EM
S
HW
Alg
pri
thm
Syst
em
7
Medium
High
Critical
138 129 140 133 126144 143
1.65 3.5 13.12 19.16 26.71 39.09 46.89
0
100
200
1 3 5 7Week (Calendar)
Open Coverage
58.52% 61.00%
36.79%
48.63%
26.88%35.00%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
6 7
Coverage
Int
SWQA
VoD
(C,H,M)
Week Open Resolved
21-Sep-08 6 3
28-Sep-08 5 4
05-Oct-08 2 1
12-Oct-08 0 4
26-Oct-08 21 15
02-Nov-08 10 9
09-Nov-08 15 15
16-Nov-08 17 13
23-Nov-08 46 32
30-Nov-08 6 25
07-Dec-08 16 31
14-Dec-08 33 14
21-Dec-08 34 30
28-Dec-08 15 35
04-Jan-09 26 13
11-Jan-09 28 27
18-Jan-09 20 16
25-Jan-09 24 35
01-Feb-09 14 18
08-Feb-09 19 22
15-Feb-09 34 26
22-Feb-09 35 34
01-Mar-09 27 24
08-Mar-09 12 14
15-Mar-09 8 10
22-Mar-09 3 3
29-Mar-09 1
Ceragon Networks proprietary and confidential
R&D Project Maturity
-100-90-80-70-60-50-40-30-20-10
0102030405060708090
100
Week
R&D Maturity
Quality
Coverage
R&D Maturity
4
138 129 140 133 126144 143
1.65 3.5 13.12 19.16 26.71 39.09 46.89
0
100
200
1 3 5 7Week (Calendar)
Open Coverage
Ceragon Networks proprietary and confidential
R&D Project Maturity
5
3 2
36
94
72 2
34
1 14 5
1 40
10
20
30
40
50
60
70
80
SW IP
I
EM
S
HW
Alg
pri
thm
Syst
em
7
Medium
High
Critical
58.52%61.00%
36.79%
48.63%
26.88%
35.00%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
6 7
Coverage
Int
SWQA
VoD
Ceragon Networks proprietary and confidential
Open/Resolved per Week
6.0(C,H,M)
Week Open Resolved
01-Apr-07 1 1
08-Apr-07 6 4
15-Apr-07 5 4
22-Apr-07 1 1
29-Apr-07 2 2
13-May-07 2 2
20-May-07 1
27-May-07 1 1
03-Jun-07 3 3
10-Jun-07 1 1
24-Jun-07 1
12-Aug-07 15 1
19-Aug-07 1 5
26-Aug-07 3 1
02-Sep-07 19 3
09-Sep-07 6
16-Sep-07 4 6
23-Sep-07 5 1
30-Sep-07 2 2
07-Oct-07 20 13
14-Oct-07 16 10
21-Oct-07 8 14
28-Oct-07 20 14
04-Nov-07 6 7
11-Nov-07 8 5
18-Nov-07 15 3
25-Nov-07 12
02-Dec-07 17 9
09-Dec-07 39 31
16-Dec-07 28 15
23-Dec-07 14 8
30-Dec-07 10 2
06-Jan-08 43 20
13-Jan-08 21 31
20-Jan-08 29 27
27-Jan-08 26 13
03-Feb-08 19 42
10-Feb-08 29 32
17-Feb-08 25 37
24-Feb-08 17 28
02-Mar-08 47 20
09-Mar-08 41 21
16-Mar-08 29 53
23-Mar-08 43 32
30-Mar-08 36 40
06-Apr-08 44 41
13-Apr-08 29 40
20-Apr-08 29 20
27-Apr-08 37 32
04-May-08 29 22
11-May-08 79 74
18-May-08 66 62
25-May-08 49 49
01-Jun-08 30 29
08-Jun-08 22 12
15-Jun-08 31 41
22-Jun-08 24 24
29-Jun-08 27 19
06-Jul-08 22 26
13-Jul-08 12 24
20-Jul-08 16 9
27-Jul-08 9 8
03-Aug-08 7 6
10-Aug-08 12 10
17-Aug-08 5 6
24-Aug-08 2
\r&d_work_space\Teams\Validation & Verification\Formal Release\Albatross\IP-10 6.0 & 6.1
Statistics\ Working Days IP-10 6.0 & 6.1.xlsx
6
R&D Project Maturity
Ceragon Networks proprietary and confidential
R&D Conclusions
Open bugs – reason
7
System – incomplete system design
Reopening – a bug that was resolved, and reopened during re-validation..
New – new bug in the implementation.
Hidden – hidden bug due to coverage overcast.
Design – incompatible design with regard to system requirements – should
have been detected at DR.
Aanother bug – chain of bugs – fixing one bug leads to the opening of another.
Reject - Bugs
Ceragon Networks proprietary and confidential
R&D Conclusions
8
Bugs detected per GroupsGroup Open bugs Critecal &
High
close/Reso
lve
Reject Duplicate Deep
Freeze
Monitor
x 139 65 85 38 12 33 18
% 46.17% 21.59% 28.24% 27.34% 8.63% 10.96% 5.98%
xx 59 34 43 13 2 9 6
% 19.60% 11.30% 14.29% 22.03% 3.39% 2.99% 1.99%
xxx 85 57 59 15 8 13 11
% 28.23% 18.94% 19.60% 17.65% 9.41% 4.32% 3.65%
xxx 18 5 9 1 6 1
% 5.90% 1.66% 2.99% 0.00% 5.56% 1.99% 0.33%
∑ 301 161 196 66 23 61 36
∑ % 53.48% 61.11% 21.92% 7.64% 20.26% 11.90%
Open bugs
Group Critic
al
high ∑
x 14 24 40
xx 8 58 102
xxx 1 17 65
xxxx 9 50
xxxxx 1 1
xxxxx 43
Open Bugs for Each Group
Ceragon Networks proprietary and confidential
R&D Conclusions
9
Bugs Detected by Tester
Group Open bugs
Monitor Reject Duplicate Change request
Integrationzz 89 3 7 81
sss 26 5 8 5
oooo 9
SWQA llll 151 4 16 42
hjhjh 465 11 29 132
hjhjhj 18 2 2
hjhjh 43 1 3
lolol 14 2 2
dfgdfg 2
VoD Yyyyy 41 1 42
Il 26 2
hghgh 23 3 5
Al 10 1 1
St 26 4 2
Ale 62 1 1 23
Oth 6
Ceragon Networks proprietary and confidential
R&D Conclusions
0
1
2
3
4
5
6
7
8
Alarm
BIS
T
Boot
CLI
E1/T
1
Ethern
et
FE
FPGA
IDC
Init
Led
Local - Remote
Modem
MRMC
Pic
PM
Pro
tectio
n
PS
RFIC
RFU
Syste
m
Tra
ffic
Wayside
Web
WSC
1 1 1 1 1
2
2
1 1 1
6
1
1 1 1
2
2 4 2
3
2 2
3
1 1
1
1
2
1 1
1
2
1
1 1
1
1 1
1
1
3 - Medium 2 - High 1 - Critical
Detected Module vs Severity Graph
0
2
4
6
8
10
12
AC
M
Ala
rm
Boot
E1/T
1
FPG
A
IDC
Local -
Rem
ote
Loopback
Modem
PM
Pro
tectio
n
Syst
em
Tra
ffic
UC
21 1
5
2
5
1 1 1
1
12
1
4
1
3
3
21
2
2
3 - Medium 2 - High 1 - Critical
Detected Module vs Severity Graph
10
Bugs Detected per module
Ceragon Networks proprietary and confidential
R&D Conclusions
11
Working Days
Project Group Plan Dane Not calculate ∑ Done Delta
X
Int 143.5 188 188 44.5
SWQA 100 191 87 278 178
VoD 109 193 193 84
XX
Int 90.5 138 102 240 149.5
SWQA 94 159 81 240 146
VoD 133 212 40 252 119
XXX
Int 53.35 88.45 78.5 166.95 113.6
SWQA 58.25 91.35 12.5 103.85 45.6
VoD 92 108 72 180 88
Group Sub Plan Done
Int
Reg 9 30.5
PVG Modem 15.5 27
ACM adaptive
mode 17 25
PreDistortion 5 11
Miscellaneous 67
Bugs 35
SWQA
Init 2 8
Cfg 3 19
SW
maintenance 2.5 13
Trap 1 4
protection 12.5 16.5
Security 6 11
NTP 1 4.5
CFM 5 30
bugs 66
setup
problem 15
Vod
Consecutive
actions 1 5
Traffic 3 7
Radio/ACM 14 22
CFM 16 55
E1/T1 and
Ethernet Only
carrier
regression 0 11
Analyze
problems 40
Ceragon Networks proprietary and confidential
Summary
12
Ceragon Networks proprietary and confidential
Summary
13
Ceragon Networks proprietary and confidential
Summary
14
Thanks
15