Specifying, predicting and testing - Tait Communications · Coverage prediction software for wide area radio networks (such as a Tait LMR network) accurately models the propagation

www.taitradio.com

WHITE PAPER

Specifying, predicting and testing: Three steps to coverage confidence on your digital radio network

WHITE PAPER SPECIFYING, PREDICTING AND TESTING:

www.taitradio.com ©Tait Limited 2017 2

EXECUTIVE SUMMARY

One of the most important properties of a radio network is coverage. Yet because radio waves are invisible, it can be difficult to understand what radio terminals are experiencing – and how confident network operators and radio users can be, that their critical communications will be heard.

Coverage prediction software applications rely on assumptions and specifications to predict coverage reliability, but these predictions have their limits. Beyond those limits, it is necessary to physically verify that the specified level of coverage is met.

In this paper, we will look at reliability specifications, coverage prediction theory, and how physical, location signal measurements can verify it in a robust, repeatable and affordable way.

Find out about:

The limitation of coverage prediction Specifying network coverage reliability Verifying coverage



THE LIMITATIONS OF COVERAGE PREDICTION

Coverage prediction software for wide area radio networks (such as a Tait LMR network) accurately models the propagation of a radio wave, allowing radio engineers to design and plan a radio network with confidence. It takes into account (among other things) the distance from transmitters, variations in topography, the curvature of the earth and changes in atmospheric density at altitude. Together, these variables give a good picture of the overall radio network coverage. But there are limits. No matter what the resolution of the data, it cannot perfectly represent the real world.

Physical variations

Coverage prediction tools represent the world via a finite number of points, each representing an area (typically 100 sqm) of the landscape. There can be many minor influences that are either too small to practically model in software, or that will change over the life of the network.

Examples include: Small terrain variations

A 20 square meter rocky outcrop won’t be represented in the model, but it will have a small effect on coverage.

Vegetation and buildings Trees grow and die; buildings are constructed or demolished, each impacting on real world coverage. It is impractical to model each and every building, tree and shrub in a wide area network – data cannot reflect changes at this scale.



These objects (known as “clutter”) are allowed for in the prediction model; areas are assigned different “clutter classes” (farmland, forest, suburban etc) which apply typical properties.

Slow fading

If you took a large number of measurements in the area shown in the diagram, you would find that actual signal levels vary around the predicted value (in this case -90dBm) according to a normal distribution curve. Most values will lie close to -90dBm (between -95dBm and -85dBm) with decreasing measurements at more extreme values.

So it is impossible to predict with confidence that the signal at a chosen point will be exactly -90dBm. It will be close, but a measured value will depend on exactly where and when the measurement was taken. (This is referred to as “slow fading”, “log-normal fading”, or “shadowing”.) However, you can confidently specify the probability that a measurement taken in that area will fall within a certain range.

68% of measured values fall within this range

predicted signal level

signal level

-95dBm -90dBm

5dBm 5dBm

num

ber o

f lo

catio

ns

-85dBm

predicted signal level -90dBm

range of actual signal

num

ber o

f lo

catio

ns



From the example, if the predicted signal strength is -90dBm, and signal strength in this area has a standard deviation of 5dB, there is a 68% chance that a measured value will be between -95dBm and -85dBm. A more common approach to describe slow fading is that there is an 84% chance that the measured value is greater than -95dBm.

84% of measured values fall within this range

predicted signal level

signal level

-95dBm -90dBm

5dBm

num

ber o

f lo

catio

ns



THE EDGE OF COVERAGE

For radio users, the signal strength figure is simply not important - they just want to know that they will be able to hear and understand other users. Unfortunately, coverage prediction software alone cannot provide that guarantee – computers are good at maths, but not much else. Bridging the gap between the user experience and the hard numbers that a computer can use is Delivered Audio Quality (DAQ).

Delivered Audio Quality

DAQ is a measure of how intelligible the communication is – the proportion of speech that is intelligible, on a scale of seven tiers, from “Unusable” (DAQ 1) to “Perfect” (DAQ 5). (The downside, of course is that DAQ is subjective. Individual users may not agree that a given transmission was a certain DAQ - the boundaries are “fuzzy”.)

Industry standard TSB-88 defines a signal level1 that can be expected for each DAQ level. For digital technologies, an objective measure of quality (over and above signal strength) is Bit Error Rate (BER). So DAQ allows a subjective experience to be mapped to an objective number that can be both predicted (in coverage prediction software) and measured. TSB-88-1D maps DAQ to both BER and Signal to Noise Ratios.

A DMR radio network is expected to provide DAQ3 (“Speech understandable with slight effort”) or better, with received Bit Error Rate 2.6% or less. The subjective value (DAQ 3) is not easily measured, but the objective value (BER = 2.6%) can be measured with instrumentation.

Defining the edge of coverage

Radio waves get progressively weaker as they travel away from the transmitter. So there is a clearly-defined edge of acceptable coverage, between the radio network meeting requirements on one side, and falling short (however narrowly) on the other.

The edge of coverage is defined by: quality – usually DAQ – mapped to an objective signal strength or bit error rate, contour reliability – how much of the coverage boundary must have the

specified quality.

So, a radio network could be specified as “delivering DAQ3.4 with 95% reliability at the boundary”. On one side “Speech is understandable without repetition” (DAQ3.4) along more than 95% of the boundary, and fewer than 95% on the other side.

1 This is a signal to noise ratio, from which a signal level can be calculated.



Area reliability

As a radio user travels inwards from the edge of coverage, the radio signals get increasingly stronger. Reliability will approach 100%2 when they are close to the radio tower.

Area reliability is the average across an area, which is always higher than contour reliability. Any network that is subject to a coverage guarantee (where the network vendor guarantees a certain level of coverage) should have its coverage specified as area reliability.

There are two types of area reliability that are commonly specified:

Service area reliability This is the average reliability over a defined service area. This service area could be political (such as a county), or operational (such as a 500m buffer around an electricity transmission network). If the service area reliability is 98%, there is a 98% chance that users anywhere in that area will have the desired level of service.

Covered area reliability This the average reliability within the predicted coverage boundary. This might fall partially within, and partially outside the service area. For example if the covered area reliability is 98% and all we know is that a user is within the coverage boundary, they have a 98% chance of experiencing the desired level of service.

Both definitions describe the average reliability of the network, but they do not describe how the network behaves at specific points. Some locations within the area may have very low reliability (“black spots”), but these are offset mathematically by locations with very high reliability.

2 It never precisely reaches 100%, but for all practical purposes, 99.9999….% is the same thing.



Bounded Area Percentage Coverage (BAPC)

Coverage is sometimes specified as needing to cover a percentage of the service area at a certain reliability. For example, “The network shall cover 92% of the service area at a minimum reliability of 95%”. This is Bounded Area Percentage Coverage (BAPC). BAPC merely describes what a coverage map will look like – in this example, all the locations that are predicted to have 95% reliability (or better) will make up 92% (or more) of the service area. This may not represent actual coverage.

Unfortunately, BAPC cannot not be practically tested. It would require sampling at every single location in the service area to demonstrate that the required amount of the service area has the required amount of reliability. BAPC must be translated into area reliability before a practical test can be performed.

Where coverage design must rely solely on BAPC as a specification, it must be split into two separate requirements: coverage prediction across the specified percentage of the service area, to be

verified during detailed design, covered area reliability that meets or exceeds a value calculated from the

coverage prediction, to be verified during a Coverage Verification Test (CVT) after the network is installed and commissioned.

Translating contour reliability to area reliability

There is no simple mathematical relationship between contour reliability (reliability at the edge of coverage) and area reliability (average reliability inside the coverage area). This is because a real-world coverage boundary is almost certainly a very complex shape, due to variable terrain, and features and objects on top of the terrain.

Some coverage prediction software (for example, EDX Signal Pro) can perform a probability simulation on a coverage prediction to calculate area reliability – subject to some assumptions. During design, this can show that the proposed radio network is predicted to meet the required area reliability, or to translate specified contour reliability to testable, verifiable, area reliability.

Specifying network coverage reliability

Area reliability can be practically tested and verified, while contour reliabilities cannot. The table gives a quick overview of the different coverage specifications.

Specification Description Testing?

Contour reliability Average reliability at the edge of coverage Cannot be practically tested.

Bounded area percentage coverage

Proportion of a map predicted to be covered at specified reliability.

Cannot be practically tested.

Service area reliability Proportion of all locations within the service area where service can be expected.

Can be tested

Covered area reliability Proportion of all the locations that fall within the predicted coverage boundary where service can be expected.

Can be tested.



VERIFYING COVERAGE

To verify that your installed network meets coverage requirements, your coverage must be specified as either: covered area reliability - the proportion of randomly-selected locations within the

predicted coverage boundary where service can be expected, service area reliability - the proportion of all locations within the service area where

service can be expected.

The level of service you require also needs to be defined. Where possible, the service threshold should be a single, measurable, objective value. Common coverage design thresholds are signal strength (RSSI) and Bit Error Rate (BER), which may have been derived from a specified DAQ requirement.

Coverage Verification Testing (CVT) physically measures area reliability in a robust, repeatable and affordable way. In this situation, reliability refers to the proportion of locations that meet or exceed the coverage design threshold.

Where to sample

Statistical sampling requires each sample to be randomly and independently selected. Obviously, if all samples were taken right next to radio sites, the test would not be valid. Nor would taking all samples in deep valleys at the edge of coverage. Neither example would provide an accurate measure of reliability.

If time and money were no object, every possible location could be tested, and a very precise reliability measure could be achieved. Clearly this is impractical; another approach is needed, to balance precision and affordability. This requires a controlled randomisation approach, that balances random sampling and even distribution, by spreading sufficient samples evenly across the service area.

To distribute samples across the service area in an unbiased way, coverage engineers create a test grid, which divides the service area into evenly-sized test tiles, typically one-to-two kilometres square. A random sample is taken within each test tile. A common misunderstanding is that each test tile is tested; the tiles are simply a device to distribute samples.

So, while not random in the strictest sense, sampling is randomized within a test tile. When designing the coverage verification test, the coverage engineer can adjust the tile size, to ensure that enough samples are taken to meet specified confidence levels, while keeping the sampling as evenly-spread as possible.

As the actual sampling is performed by vehicles on public roads (which conveniently replicates actual mobile radio network use), any tiles on the grid that do not have full (or partial) road access are excluded.



The method

If random signal quality samples are taken, you can estimate the percentage of locations that meet or exceed the coverage design threshold. The associated degree of confidence will depend on the number of samples taken.

For example:

A radio network specifies area reliability of 90%, and coverage design threshold is -100dBm signal strength. We require a typical confidence level of 99%.

Let’s look at some possible outcomes, based on different numbers of samples. The number of samples will depend on the reliability specification and predicted reliability – the smaller the gap between them, the more samples are required.

Samples ≥ -100dBm

< -100dBm

Measured reliability

Confidence* Acceptable?

10 9 1 90% 50% No

20 19 1 95% 77% No

200 191 9 96% 99.5% Yes

900 831 69 92.3% 99.0% Yes

*using estimate of proportions technique Looking at the table, the first two examples fall well short in terms of confidence, due to their very small sample numbers. The third example exceeds both reliability and confidence, suggesting that the system may in fact have more radio sites than necessary to meet the specified criteria.

The final example – with 900 samples and measured reliability around 92% - meets the confidence criteria, and best represents a realistic, well-executed CVT.

What happens if we increase the confidence figure further? Diminishing returns set in quite quickly: 99.9% confidence requires 2150 samples. That is a significantly greater sampling overhead, so the sampling cost can get out of hand quite quickly.

… there is a clearly-defined edge of acceptable coverage, between the radio network meeting requirements on one side, and falling short (however narrowly) on the other.



SUMMARY

Physical, location signal measurements from a well-designed and executed coverage verification test can verify it in a robust, repeatable and affordable way.

Coverage predictions are statistical by nature; the theoretical nature of coverage prediction can provide only part of the story. They can only provide a statistical likelihood that, in any given location, a certain signal level will be equalled or exceeded. You cannot be 100% confident that a specific signal level will occur at a given time and place.

Averaged out across your entire service area, predictions define the mathematical likelihood that a randomly-selected location will have a signal strength equal to, or greater than your specified threshold.

However, if you require a coverage guarantee, your network coverage requirement must be specified as area reliability. Coverage Verification Testing gives a specified confidence (usually 99%) that your network is delivering its specified area reliability, by randomly sampling the network’s coverage across its service area.



GLOSSARY

BAPC (Bounded Area Percentage Coverage)

How much of a service area falls within the predicted coverage boundary. This is not a testable coverage measure, but is useful for comparing coverage predictions.

BER (Bit Error Rate) A measure of how much information is received incorrectly. High values of BER lead to gaps and distortion in the audio. Voice networks usually specify less than 2.6%.

dBm Measure of signal strength –ratio between decibels and one milliwatt of power. Radio coverage values can range from < -110dBm (very low) to > -60dBm (very high).

Clutter Land cover (trees, building etc) in a radio coverage area.

Confidence level Likelihood that proportion of reliability lies within the confidence window. CVTs typically use 99%.

Confidence window The range that the reliability is likely to fall within. In CVTs, the window may be all reliabilities, equal to or greater than specified reliability. So, “We are 99% confident that true reliability is equal to, or greater than the specified reliability of 90%”

Contour reliability Reliability at the edge of coverage.

Covered Area Theoretical geographic area within the coverage boundary. (May be constrained by service area.

Covered Area Reliability Reliability of a radio network, averaged across all locations in the covered area.

CVT (Coverage Verification Test)

Statistical test to determine if a radio network meets coverage specifications.

DAQ (Delivered Audio Quality)

Subjective scale of audio quality as perceived by a radio user.

Estimate of proportions Statistical method that estimates true proportion from a number of samples. Confidence levels and confidence windows are associated with this estimate.

LMR (Land Mobile Radio) Wireless communications used in vehicles (mobiles) or on foot (portables).

Reliability Percentage of locations that have required signal level.

Service area Geographic area of operation, usually a political boundary (city or county limit).

Service area reliability Reliability of a radio network averaged across all locations in the service area.

TIA TSB-88 Industry recommendations for radio coverage prediction, design and verification.

Follow Us

+ Stay updated with our latest contents

About the author:

Stephen Bunting is a System Engineer at Tait Communication, specializing in coverage prediction and verification. Stephen has twelve years’ telecommunications experience.

COPYRIGHT

General terms of use for Tait technical documentation. While Tait has taken every care to ensure that the information and contents are correct and up-to-date at the time of printing, the information may contain technical inaccuracies and/or printing errors. Tait does not guarantee the accuracy or correctness of the information. Tait cannot be held liable or responsible for errors or omissions in the contents of the technical documentation. All information contained in the technical documentation is given without warranties or representations, expressed or implied.

Disclaimer. Tait Limited marketed under the Tait Communications brand. Tait Limited expressly disclaims all warranties, expressed or implied, including but not limited to implied warranties as to the accuracy of the contents of this document. In no event shall Tait Limited be liable for any injury, expenses, profits, loss or damage, direct, incidental, or consequential, or any other pecuniary loss arising out of the use of or reliance on the information described in this document.

Copyright © 2012 Tait Limited.

© Tait Limited 2017.

Documents

Specifying, predicting and testing - Tait Communications · Coverage prediction software for wide area radio networks (such as a Tait LMR network) accurately models the propagation