A Performance Evaluation is a series of defined tests that subsequently provide a detailed analysis of the radar system at distinct levels:

System Level (Level 1) – This analysis is to assess the quality of the information (data), provided by the radar sensor, by measuring the overall performance of the sensor against the performance parameter reference values within the Standard (e.g. EUROCONTROL) against the Operational Coverage Volume.

Subsystem Level (Level 2) – This analysis is the in depth evaluation, of the technical performance of the individual subsystem components of the sensor (e.g. Antenna, Receiver, Extractor, etc)

Detailed Subsystem (Level 3) –This analysis is a specialised series of defined tests to assist in optimising radar systems, deeper level fault finding, or target injection scenarios, etc.

“In order to maintain optimum performance, modern surveillance systems require sophisticated andpermanent system performance monitoring. During the operational life of a system, periodic performance testing is also necessary to assure continued satisfactory operational performance.” 1

“Comprehensive and continuous radar coverage of high quality and reliability is essential for the uninterrupted provision of radar services and the application of specific radar separation standards.” 2

Using our world-renowned, manufacturer-independent Radar Analysis Support System (RASS) we execute the PE process, to verify the performance specifications as defined by ICAO, EUROCONTROL, Quality of Service (QoS) requirements or the radar OEM specifications.

Our highly trained and experienced Radar and RASS personnel are handpicked subject matter experts. Their training and skills enable a smoother, efficient and value for money approach to the analysis of results and the consequential delivery of detailed recommendations within the evaluation report.

radar performance evaluation

The minimum requirement for Level 1 and Level 2 Performance Evaluations (PE)is an annual evaluation; more regularly subject to failures or coverage volume changes, etc. It is an identified requirement prior to the conduct of any Flight Testing. Level 1 can be carried out in real-time via our Surveillance Monitoring System (SMS) thereby mitigating the requirement for SMS customers to conduct onsite Level 1 evaluations. Level 3 evaluations are conducted on an as required basis, subject to the customers’ requirements.

performance evaluation overview

10101010010101010101010101010101001010101010101010101010101000101010111111010101010101010101010101010101010101010101010101010100000110101010101010100001010101010101010101010101010101010101010101010101010100111001010101011010100101000001010011111010101010101010101010101010100101010101010111010101011010101010101010101000010101010101010100101010101010011111111000101010110100001010110100011110101010101010101010101010000101010101010101010101010101010101010100101010101010101010101010010101010101010101010101010001010101111110101010101010101010101010101010101010101010101010101000001101010101010101000010101010101010101010101010101010101010101010101010101001110010101010110101001010000010100111110101010101010101010101010101001010101010101110101010110101010101010101010000101010101010101001010101010100111111110001010101101000010101101000111101010101010101110101010110101010101010101010000101010101010101001010101010100111111110001010101101000010101101000111101010101010101010111010101001010101010101010101010100101010101010101010101010100010101011111101010101010101010101010101010101010101010101010101010000011010101010101010000101010101010101010101010101010101010101010101010101010011100101010101101010010100000101001111101010101010101010101010101010010101010101011101010101101010101010101010100001010101010101010010101010101001111111100010101011010000101011010001111010101010101010101010101000010101010101010101010101010101010101010010101010101010101010101001010101010101010101010101000101010111111010101010101010101010101010101010101010101010101010100000110101010101010100001010101010101010101010101010101010101010101010101010100111001010101011010100101000001010011111010101010101010101010101010100101010101010111010101011010101010101010101000010101010101010100101010101010011111111000101010110100001010110100011110101010101010111010101011010101010101010101000010101010101010100101010101010011111111000101010110100001010110100011110101010101010101011101010100101010101010101010101010010101010101010101010101010001010101111110101010101010101010101010101010101010101010101010101000001101010101010101000010101010101010101010101010101010101010101010101010101001110010101010110101001010000010100111110101010101010101010101010101001010101010101110101010110101010101010101010000101010101010101001010101010100111111110001010101101000010101101000111101010101010101010101010100001010101010101010101010101010101010101001010101010101010101010100101010101010101010101010100010101011111101010101010101010101010101010101010101010101010101010000011010101010101010000101010101010101010101010101010101010101010101010101010011100101010101101010010100000101001111101010101010101010101010101010010101010101011101010101101010101010101010100001010101010101010010101010101001111111100010101011010000101011010001111010101010101011101010101101010101010101010100001010101010101010010101010101001111111100010101011010000101011010001111010101010101010101110101010010101010101010101010101001010101010101010101010101000101010111111010101010101010101010101010101010101010101010101010100000110101010101010100001010101010101010101010101010101010101010101010101010100111001010101011010100101000001010011111010101010

We record data (e.g. MSSR,PSR, ADS-B, etc) using RASS-R (real-time)hardware/software. Then, a complete filtering process is performed, eliminatingany unreliable aircraft replies, ensuring only the highestquality data is used for quantitative analysis. Evaluation of this data enables the early detection ofany degradation to system components well before, andwell beyond, the radar’s Built InTest Equipment (BITE)capability.

System Level

(RASS Level 1)Subsystem Level

(RASS Level 2)The Performance Evaluation Process

DETAILED ANALYSIS

(RASS Level 3)

Employing the “Top-Down” methodology, we will inject and record signals at RF, IF, Video and Data level within the design limits of the sensor. This is performed using RASS-S (site/subsystem) hardware/software; which was specifically designed for this purpose.

The results of the evaluation provide a detailed technical analysis of the individual radar subsystems in conjunction with the overall performance of the radar. The sub-system components comprise the specific processing sections of a sensor that are imperative to providing an accurate radar data output.

Results are also analysed against the functional specifications of the sensor, as set by the OEM. Theevaluation of an entire radar system is completed quicklyand with minimal operational impact. Performance resultsare provided for subsystem components, including but not limited to, MSSR/PSR antenna patterns, rotating mechanisms (pedestal,motor, rotary joints, etc), receiver dynamic range and bandwidth, clutter maps and STC effectiveness, plot extractor and tracker.

This type of evaluation is a customised series of tests designed to assess the radar system at a deeper level than the tests conducted in a RASS Level 1or 2 PE, using more specialised RASS-S tools.These tests can be conducted at all lifecycles of the radar; during Factory Acceptance Testing (FAT), Site Acceptance Testing (SAT) or during the operational lifetime of the radar to identify deeper level fault analysis.

Our personnel are experienced in working within an ATC and/or Defence operational environment. We work around specific operational requirements to conduct the Performance Evaluation on the radar sensor components.

At the outset, we obtain relevant site information about the radar system to further minimise the time spent on site. This information will include, inter alia, the radar type, radar coordinates, any operational impacts which need considered, site contact details, etc. This information ensures that the evaluation is conducted in a timely and efficient manner as possible.

A Test Schedule is developed in consultation with all stakeholders to minimise any operational impact for the onsite measurement period. It is the client’s responsibility to ensure that all site accessrequirements are coordinated and managed.

The results provide a precise view of the radar sensor’s key performance parameters

The results of the evaluation provide a precise view of the radar sensor’s keyperformance parameters at the data level (i.e. at the Operator’s display system); such as Probability of Detection(Pd), Time-stamping Accuracy, Positional Accuracy,including azimuth and range errors, azimuth and rangebias, RMS errors, Jumps and data validity analysis.

A pre-briefing is held with the relevant local staff upon arrival at the site; prior to the start of the PE measurements. During this briefing, the schedule is reviewed including, identifying any particular performance issues local staff have noticed with the radar and to identify key areas of the radar sensor to concentrate on during the Performance Evaluation.

At the conclusion of the measurement period a debriefing with the local staff is held to discuss how the schedule progressed, if any significant problems arose and what the recommended next steps will be.

After the onsite activities have been completed, we will conduct the analysis of the collected data to determine the overall performance of the radar under test and deliver a comprehensive report within agreed timeframes.

TABLE 1 EUROCONTROL requirement for data collection minima.

Minimum Duration High density traffic area (en-route or major TMA) 1 hour*,Medium density traffic areas – 2 hours*,Low density traffic areas – 4 hours** Must include at least 50,000 data samples

Minimum Quantity of Data Probability of Detection – 200 Chains >5 minutes per chainAccuracy Analysis – 150 ChainsSystematic Error Estimation – 200 Chains >5 minutes duration in cover of >2 radars

System Configuration Normal operational configuration for prevailing traffic and environment

Environment, Weather No Anoprop conditions or heavy Angel activity or abnormal conditions (e.g. jamming, interference) should be used to verify PSR/SSR overall performance

Traffic Recordings shall be made in Peak Traffic times if possible

Measurement and Analysis

RASS Level 2 Evaluation (Sensor Baseline) RASS Level 1 Evaluation (Data Analysis)

The following tests in Table 1 are performed in a RASS Level 1 Evaluation. We will consult with you to determine which tests are appropriate for your configuration of radar, to assess the quality of surveillance data. In most cases, all tests can be performed.

The following tests in Table 2 are generally performed in a RASS Level 2 Evaluation. We will consult with you to determine which tests are appropriate for your configuration of radar.

TABLE 2 The following tests below are performed in a RASS Level 1 Evaluation.

Ref# Description

(M)SSR

L1-101 Probability of Detection AnalysisThe Probability of Detection (Pd) is calculated for the defined coverage volume of the radar in order to characterise the ability of the radar to detect targets.

L1-102 Positional Accuracy AnalysisThe position of the targets are analysed for Bias (systematic error), Accuracy (random error) and Jumps in range, azimuth and time (FT aircraft dependent) within the defined coverage volume of the radar. This check is performed in order to evaluate the ability of the radar to report a target position accurately.

L1-103 False Target AnalysisThe data from the radar will be analysed for false target reports according to the defined specifications. These targets could indicate multi path, reflection or antenna anomalies.

L1-104 Data Validity AnalysisThe data from the radar will be examined and assessed for invalid aircraft responses. This analysis will include invalid Mode A and C responses and replies to side lobes or reflected signals.

PSR

L1-201 Probability of Detection AnalysisThe Probability of Detection (Pd) is calculated for the defined coverage volume of the radar in order to characterise the ability of the radar to detect targets.

L1-202 Positional Accuracy AnalysisThe position of the targets are analysed for Bias (systematic error) and Accuracy (random error) in range, azimuth and time (FT aircraft dependent) within the defined coverage volume of the radar. This check is performed in order to evaluate the ability of the radar to report a target position accurately.

L1-203 False Target AnalysisThe data from the radar will be analysed for false target reports according to the defined specifications.

TABLE3 The following tests below are performed in a RASS Level 2 Evaluation.

Ref# Description

(M)SSR

L2-101 Antenna Horizontal Polar Diagram AnalysisThis recording is conducted to evaluate the performance of the antenna, within the operational environment. This test will determine if the Antenna Pattern exhibits abnormal performance and/or is affected by influences external to the radar system. The result is presented in a Horizontal Polar Diagram (HPD) of the Antenna Pattern.

L2-102 Antenna Vertical Polar Diagram Analysis – OptionalThis recording is conducted to evaluate the performance of the antenna, within the operational environment.This test will determine if the Antenna Diagram exhibits abnormal performance and/or is affected by influences external to the radar system. This test will indicate relative beam shape, gain and peak elevation angle of the vertical radiation pattern of the antenna. The result is presented in a Vertical Polar Diagram (VPD) of the Antenna Pattern.

L2-103 Turning Gear AnalysisThis recording is conducted to evaluate the mechanical performance of the rotating equipment; motor and gearbox performance, azimuth encoders and the inclination of the rotating platform in the horizontal plane. This test will determine the accuracy of the ARP and ACP being generated by the azimuth encoders by comparing the recorded azimuth information against actual antenna movement recorded by an independent three-axis gyroscope that is temporarily mounted on the rotating platform.

L2-104 Transmitter Characteristics Analysis – OptionalThis test utilises the available points to check the transmitter pulse characteristics such as the pulse width, spacing, power and shape.

L2-105 Receiver Sensitivity AnalysisThis recording is conducted to evaluate the performance of each Interrogator’s receiver. This Test willindicate if the receiver is functioning within required parameters. The sensitivity calibration curve is further used by other RASS processes to analyse radar system performance. Receiver sensitivity calibration curves are taken on SUM, DELTA and OMEGA channels of each interrogator.

L2-106 Receiver Bandwidth AnalysisThis recording is conducted to evaluate the performance of each Interrogator’s receiver across its operating bandwidth. The test will indicate if the receiver is functioning to specification. Receiver Bandwidth will be measured on the SUM, DELTA and OMEGA channels of each interrogator.

L2-107 Receiver Video AnalysisThis recording is conducted to evaluate the performance of the interrogator, remote field monitor (RFM) and RF environment. Careful analysis of the windowed video recording can help determine the monopulse performance of the LVA antenna (SUM and DELTA receiver channels). The Windowed Video recording is used to visualise received transponder video in great detail, to observe abnormal or unwanted performance parameters such as reflections, noise or FRUIT.

TABLE 3 The following tests below are performed in a RASS Level 2 Evaluation.

Ref# Description

L2-108 Receiver STC Analysis – OptionalThis recording is conducted to evaluate the Sensitivity Time Control (STC) performance of each receiver. STC performance is measured with the radar operating in normal operational requirements. This test will verify that the applied STC is relevant for the operating environment.

L2-109 Mode A/C Decode Functional Analysis – OptionalThis test is conducted to evaluate the radar’s ability to decode and display the emergency codes and altitude sent from a transponder. This test is performed by injecting emergency A codes 7700, 7600 and 7500 with a Remote Field Monitor, recording the decoded signals and visually confirming the correct code responses on the operator display; and then changing the C code to confirm correct slant range correction is applied.

PSR

L2-201 Antenna Horizontal Polar Diagram AnalysisThis recording is conducted to evaluate the performance of the antenna, within the operational environment. This test will determine if the Antenna Pattern exhibits abnormal performance and/or is affected by influences external to the radar system. The result is presented in a Horizontal Polar Diagram (HPD) of the Antenna Pattern.

L2-202 Antenna Vertical Polar Diagram Analysis – Optional This recording is conducted to evaluate the performance of the antenna, within the operational environment. This test will record the level of solar noise, either immediately leading up to sunset or immediately following sunrise, as the sun transits through the vertical beam pattern of the antenna. This test will indicate relative beam shape, gain and peak elevation angle of the low/main beam and high/auxiliary beam vertical radiation patterns of the antenna. The result is presented in a Vertical Polar Diagram (VPD) of the Antenna Pattern.

L2-203 Turning Gear AnalysisThis recording is conducted to evaluate the mechanical performance of the rotating equipment; motor and gearbox performance, azimuth encoders and the inclination of the rotating platform in the horizontal plane. This test will determine the accuracy of the ARP and ACP being generated by the azimuth encoders by comparing the recorded azimuth information against actual antenna movement recorded by an independent three-axis gyroscope that is temporarily mounted on the rotating platform.

L2-204 Transmitter Characteristics Analysis – OptionalThis test utilises the available points to check the transmitter pulse characteristics such as the pulse width, spacing, power and shape.

L2-205 Receiver Sensitivity AnalysisThis recording is conducted to evaluate the performance of each receiver. Measurements will beconducted on agreed frequencies as agreed by the client. This Test will indicate if the receiver is functioning within required parameters. The sensitivity calibration curve is further used by other RASS processes to analyse radar system performance.

L2-206 Receiver Bandwidth AnalysisThis recording is conducted to evaluate the performance of each receiver across its operating bandwidth. Measurements will be conducted on agreed frequencies as agreed by the client. The test will indicate if the receiver is functioning to specification.

TABLE 3 The following tests below are performed in a RASS Level 2 Evaluation.

Ref# Description

L2-207 Receiver Video AnalysisThis recording is conducted to evaluate the effectiveness of the clutter maps for the given environment. This test will indicate abnormal or unwanted performance issues such as clutter.

L2-208 Receiver STC AnalysisThis recording is conducted to evaluate the Sensitivity Time Control (STC) performance of each receiver. STC performance is measured with the radar operating in normal operational requirements. This test will verify that the applied STC is relevant for the operating environment.

Multi-level Data

L2-301 (M)SSR Data Recording AnalysisLarge amounts of operational data (Plot & Track) are recorded from the output of the Interrogator and/or Tracker systems. The data contains many different parameters and examination can reveal details about probability of detection, interrogator performance and tracking system performance. This examination also provides information such as aircraft with an ‘A’ or ‘C’ code equal to zero, holes in coverage, feathering and detection at different elevations

L2-302 PSR Data Recording AnalysisLarge amounts of operational data (Plot & Track) are recorded from the output of the Plot Extractor and/or Tracker systems. The data contains many different parameters and examination can reveal details about probability of detection and tracking system performance. This examination also provides information such as Antenna elevation, False Targets, Noise, Holes in coverage and Excessive Clutter.

L2-303 Combined Data Recording AnalysisLarge amounts of operational data (Plots & Tracks) will be recorded from the output of the Tracker systems. The data contains many different parameters in the data stream which is examined for radar system performance.

Table 2: Level 1 Analysis Description

At the conclusion of the measurement phase we will undertake analysis of the collected test data in order to compile a comprehensive written report.

This is where the value-add proposition of RASS comes to the fore, as the results are also provided in graphical format, effectively producing an analysis that is easily understood, even for non-technical parties.

At the conclusion of the Performance Evaluation the client will receive a:

PDF copy of the report via electronic medium • i.e. email

If requested, a DVD disk/s or Flash Drive • containing:

PDF copy of the report• Complete RASS Performance Evaluation •

directory, containing the data collected

As the majority of Radar OEMs utilize RASS within their own factory environment, the data sets are invaluable in order to obtain the necessary OEM support to assist in fault rectification.

Reference Documents1 ICAO Doc 8071, First Edition - 1998 - Testing of Surveillance Radar Systems

2 SUR.ET1.ST03.1000.STD-01-01 - Eurocontrol – Radar Sensor Performance Analysis – June 1997

The Main body: • An Executive Summary, providing a concise •

plain-language overview of the background, analysis and conclusions

A background and summary of the radar • equipment and maintenance status prior to the commencement of the Performance Evaluation

A summary list of conclusions obtained • from the analysis with relevant comparison to the respective standards identifying any out- of- tolerance results and a trend analysis comparing the results to the previous report (if applicable).

A list of recommendations for system • performance improvements.

An educated hypothesis is also provided as to • the possible cause and location of any anomalies.

Annex A – Level 1 Analysis • A detailed review of the Level 1 Performance •

evaluation results.

Annex B – Level 2 Analysis• A detailed review of the Level 2 Performance •

Evaluation results.

Annex C – Test Equipment• A list of all hardware and software used to •

perform the measurements and conduct the analysis for traceability.

The Report Deliverables

The report includes:

IE Asia-Pacific Pty Ltd (ABN 21 139 467 744)Canberra Technology Park49 Phillip AvenueWatson ACT Australia 2602Tel +61 2 6241 9237Fax +61 2 6108 3556support@ie-asia-pacific.comwww.intersoft-electronics.com

IMS-600-01/Revision 3.0: 10/02/2015