Upload
junior-norris
View
224
Download
2
Tags:
Embed Size (px)
Citation preview
ITU Workshop on “Monitoring and Benchmarking
of QoS and QoE of Multimedia Services in Mobile Networks”
(Buenos Aires, Argentina, 24-25 July 2014)
Overview of ITU-T Study Group 12 Recommendations on QoS and QoE
Joachim Pomy
[email protected], Germany
Buenos Aires – Argentina – 24 – 25 July 20142
SG 12 Recommendations
E.420-E.479, E.800-E.859G.100-series, except G.160-, G.180- and G.190-seriesG.1000-seriesI.350-series (including Y.1501/G.820/I.351), I.371, I.378, I.381P-series, except P.900-seriesY.1220-, Y.1530-, Y.1540-, Y.1560-serieshttp://www.itu.int/ITU-T/recommendations/index_sg.aspx?sg=12
Buenos Aires – Argentina – 24 – 25 July 20143
Work Programme (1/8)
Coordination of QoS/performance studiesCommunications Quality of Service: A framework and definitionsVocabulary for performance and quality of serviceMean Opinion Score (MOS) terminologyTransmission characteristics for wideband digital handset and headset telephones - Inclusion of extended wideband terminalsTransmission characteristics for cordless and moblie digital terminalsTransmission characteristics for wideband digital loudspeaking and hands-free telephony terminals - Inclusion of extended wideband terminalsTechnical requirements and test methods for the universal wired headset or headphone interface of digital mobile terminalsNarrowband hands-free communication in motor vehiclesWideband hands-free communication in motor vehicles
Buenos Aires – Argentina – 24 – 25 July 20144
P.800.1 Relation between some MOS qualifiers
Buenos Aires – Argentina – 24 – 25 July 20145
Work Programme (2/8)
User interface requirements for automotive applicationsSubsystem requirements for automotive speech servicesArtificial earsCalculation of loudness ratings for telephone setsUse of head and torso simulator (HATS) for hands-free and handset terminal testingBone conduction testing setupMultiple test positions setupSetups and testing techniques for terminal performance measurements with background noiseArtificial voicesSpeech processing devices for acoustic enhancementTransmission characteristics and speech quality parameters of hands-free terminals
Buenos Aires – Argentina – 24 – 25 July 20146
Typical development cycle for a car speakerphone hands-free system and applicability of clauses of Rec. P.1100
P.1110(09)_F6-1
A sample B sample C sample D sample
Research end Pre-development end Dev. B sample end Dev. C sample end Dev. D sample end PPAP
SOP
Typically HFT development for a new car
Application of the different clauses of this Recommendation
Part 8 digital interface Part 8 digital interface
Parts 11 and 12 HFT and SRWTEP parameters
Part 13 subjective car-to-car test
Part 10 microphone parametersWhat is covered by the various parts of the test Recommendation?Part 7: Covers test arrangements used within this Recommendation.Part 8: Covers non-mandatory objective measurements using a digital interface.Part 9: Covers test signals used within this Recommendation.Part 10: Covers non-mandatory objective measurements of the microphone or/and microphone system.Part 11: Covers mandatory objective measurements of the hands-free terminal (HFT).Part 12: Covers non-mandatory objective measurements of short-range wireless transmission enabled phone (SRWTEP).Part 13: Covers non-mandatory subjective evaluation of the hands-free terminal (HFT).
Buenos Aires – Argentina – 24 – 25 July 20147
Work Programme (3/8)
Test signals for use in telephonometryObjective test methods for speech communication systems using complex test signalsMethods for determining the ‘Objective Quality Number’ as overall quality value for terminalsApplicability of psycho-acoustic models to HATS based setups for the benefit of terminal testingSetup signals for terminal with background noiseSubjective quality evaluation of telephone services based on spoken dialogue systemsSubjective performance of active signal processing devicesCultural/language/nationality dependence of subjective qualitySubjective evaluation of generic sound activity detectorsMultidimensional scaling quality assessmentSubjective assessment methods for music quality of narrowband and wideband telephony
Buenos Aires – Argentina – 24 – 25 July 20148
Rating scales used in ITU-T P.806 subjective testing methodology
PQ Scales Description Scale Descriptors
Overall Scales
B-LVLDegradation due to the Level of
Background Noisehissing, rushing, roaring
P.MULTI Perceptual Quality Scales
S-FLTSlow-varying Degradation in the
Speech Signalfluttering, babbling,
discontinuous
S-RUF Fast-varying Degradation in the Speech Signal
rough, raspy, harsh
S-HFCDegradation of High-frequency Coloration in the Speech Signal
small, distant, thin
S-LFC Degradation of Low-Frequency Coloration in the Speech Signal
dull, muffl ed, smothered
Description
LOUD
OVRL
Overall Loudness in the Speech Signal + Background Noise
Overall Quality in the Speech Signal + Background Noise
B-VAR Degradation due to the Variability of the Background Noise
bubbling, intermittent, variable
Buenos Aires – Argentina – 24 – 25 July 20149
Work Programme (4/8)
Test methodology for web browsing and browser based applicationsSubjective usability evaluation of speech-based or multimodal interactive servicesQ.7/12 defines test plans for tests of speech coders, and reports the test results and analysisE-model extension for non-handset user interfacesE-model extension for speech processing devicesE-model update for conversational quality featuresMixed-band and beyond-wideband E-modelE-model for monitoringOperational quality estimatorPerceptual approaches for multi-dimensional analysisPerceptual objective noise reduction
Buenos Aires – Argentina – 24 – 25 July 201410
Test setup for Rec. P.1501 Quality of web browsing
Buenos Aires – Argentina – 24 – 25 July 201411
Work Programme (5/8)
Perceptual Objective Prediction of Speech IntelligibilityNo-reference models for quality predictionSubjective method for simulated conversation tests addressing audio and audiovisual call qualityEffect of delays on the telemeeting qualityUse of auditory and visual cues for high-quality telemeetingsOverall telemeeting quality value (long term goal)Quality implications and requirements for telemeeting and conferencing servicesSpatial audio meetings quality evaluationFrequent updates of AppendicesQuality of Experience aspects of Multi ConnectionEnd-to-end QoE aspects of tandemmed speech processing devices
Buenos Aires – Argentina – 24 – 25 July 201412
Rec. P.1301 Decision tree for Subjective quality evaluation
of audio and audiovisual multiparty telemeetings
P.1301(12)_FA
Communicationmode including text / graphics
(e. g., webconferencing) ?
Yes
ITU-T P.1301 Annex B
No
Audio
Audio-visual
VideoCommunicationmode?
ITU-T P.1301 Annex C
Test mode= audio
Audio VideoTestmode?
Test mode= video
Audio-visual
Type ofquality?
Type ofquality?
Type ofquality?
Non-inter-active
Conver-sational
Conver-sational
Non-inter-active
Continue withFlowchart B
Continue withFlowchart C
Continue withFlowchart D
Continue withFlowchart E
Special testmethod required
Continue withFlowchart F
Conver-sational
Non-inter-active
Start
Buenos Aires – Argentina – 24 – 25 July 201413
Work Programme (6/8)
End-to-end performance for managed voice over LTE networks This Recommendation describes the key aspects impacting end-to-end performance of managed voice applications over LTE networks.Definitions and associated measurement methods for user-centric parameters for call handling in cellular mobile voice serviceMobile QoSSupplement XX to ITU-T E.800-series Recommendations (Guidelines on Regulatory Aspects of QoS)End-user multimedia QoS categoriesNetwork model for evaluating multimedia transmission performance over Internet ProtocolOpinion model for video and audio streaming applicationsQoE of Web-browsingQoE requirements for multicast video streaming servicesQoE requirements for telepresence servicesGuidance for the use of P.1201 and P.1202 in operational contexts
Buenos Aires – Argentina – 24 – 25 July 201414
Rec. G.1010 – Model for user-centric QoS categories
T1213060-02
FaxErrortolerant
Conversationalvoice and video
Voice/video messaging
Streaming audioand video
Errorintolerant
Command/control(e.g. Telnet,
interactive games)
Transactions(e.g. E-commerce,WWW browsing,
Email access)
Messaging,Downloads
(e.g. FTP, still image)
Background(e.g. Usenet)
Interactive(delay <<1 s)
Responsive(delay ~2 s)
Timely(delay ~10 s)
Non-critical(delay >>10 s)
Buenos Aires – Argentina – 24 – 25 July 201415
Work Programme (7/8)
QoE-diagnostic information from P.120X.Y type models on technical causes of QoE impairmentsIntegral index of quality for general service monitoring (per user-session); KQI definitionsMedia-session QoE model (temporal pooling, long sequences; UDP- and TCP-based streaming)Parametric non-intrusive assessment of audiovisual progressive download, considering stalling/re-buffering & quality integrationParametric non-intrusive assessment of TCP-based multimedia streaming quality, considering adaptive streamingIn-service non-intrusive measurement device – Voice service measurementsAnalysis and interpretation of INMD voice-service measurementsConformance testing for voice over IP transmission quality assessment modelsConversational modelFramework for invoking diagnostic functionsVoice Service Diagnosis Framework
Buenos Aires – Argentina – 24 – 25 July 201416
Rec. G.1029 - Voice Service Diagnosis Framework
Framework for mapping diagnostic parameters to root-cause parameters (Work Item 2)
Framework for invoking diagnostic functions (Work Item 1)
QoE/QoS Diagnostic parameters
QoE/QoS Root-cause analaysis
application
QoE/QoS Decision
parameters
Decision process
Diagnostic parameter to root-cause mapping
Invocation
QoE/QoS Root-cause
analysis parameters
Remedial action
network
Terminals
Buenos Aires – Argentina – 24 – 25 July 201417
Work Programme (8/8)
Technical cause analysisBuffer models for development of client performance metricsHand-over performance among multiple access networksInternet protocol data communication service – IP packet transfer and availability performance parametersNetwork performance objectives for IP-based servicesMeasurements in IP networks for inter-domain performance assessmentEthernet frame transfer and availability performanceEthernet service activation test methodologyHome network performance parametersProposed Appendices for Y.1566 : QoS mapping and interworking between Ethernet, IP and MPLS
Buenos Aires – Argentina – 24 – 25 July 201418
Cannot find your Topic ?
Work in ITU-T is contribution driven !Next Meeting of Study Group 12
2 – 11 September 2014 in Cartagena de Indias, Colombia (tbc)Deadline for Contributions
20 August 2014http://www.itu.int/net/ITU-T/ddp/Default.aspx?groupid=T13-SG12