Upload
ince
View
34
Download
0
Embed Size (px)
DESCRIPTION
Optical Data Links in CMS ECAL. CMS ECAL architecture and needs Optical Data Links in HEP Experiments ECAL Data Link system description Data Link components Data Link system Opto-Hybrid assembly, Quality Assurance Schedule. ECAL. Silicon Tracker. HCAL. Barrel. Preshower + Endcap. - PowerPoint PPT Presentation
Citation preview
1Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Data Links in CMS ECAL
• CMS ECAL architecture and needs• Optical Data Links in HEP Experiments• ECAL Data Link system description• Data Link components• Data Link system• Opto-Hybrid assembly, Quality Assurance• Schedule
2Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
The CMS Detector
Silicon Tracker
ECAL
HCAL
Solenoid
Muon ChambersReturn Yoke
Barrel Preshower +Endcap
3Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
CMS ECAL front-end architecture
ECAL has ~77,000 lead tungstate crystals arranged in supercrystals of 25.
Front-end electronics of each supercrystal send the data off-detector via the optical links. The architecture and the needed data rate require ≥600 Mb/s with modularity:
for Data: 1 link / supercrystal
for Trigger: 1 link / supercrystal (barrel)
5 links/supercrystal (endcap)
Total data + trigger: ~9000 links
Supercrystal DataTrigger sums
APD/VPTPreamp
ADC
Front-end board
4Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Links and HEP
Generic architecture of an optical link:
Tx Rxfiber
“Optical Layer”
Ser
Deser
“First Overlay”
MUX DEMUX
“Framing”
e/o, transport, o/e
encoding/decoding, usually performed in same chip as serialization/deserialization
Multiplexing multiple “conversations” in each link. Not usually relevant for HEP since systems are designed to run each link continuously at maximum data rate.
data in
data out
5Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Layer considerations - 1
Tx considerations:
Edge-Emitting: linear response (relevant if you design an analog link)
VCSEL: said to be more rad-resistant due to smaller active area, but practical experience is that fractional change of behavior per unit irradiation is typically similar. VCSEL is cheaper to produce and test on wafer. Was once thought that they would entirely replace Edge-Emitting. But benefit is diluted since packaging dominates the cost of both.
In any case: device chosen must be consistent with the system speed and environmental specifications (often unique to HEP; more on this later).
Tx: In HEP, usually either Edge-Emitting Laser Diode or VCSEL.
Rx: Typically photodiode or photodiode array (typically of 12), followed by digital amplifying ASIC.
Tx Rxfiber
“Optical Layer”
e/o, transport, o/e
Rx considerations:
Speed, wavelength, saturation and sensitivity limits must be consistent with system specifications.
VCSEL diagram
Edge-Emittingdiagram
test on wafer
6Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Layer considerations - 2
Optical fiber considerations:
Two principle kinds: Single Mode (SM) and Multi Mode (MM). Differences in impurities in the silica and differences in core and cladding diameters give different spectral transmission characteristics.
SM: lower attenuation, is the favored choice for telecom (longhaul) distances (10 km and greater). Uses smaller core to allow only one mode to propagate, making connector alignment more critical (and expensive).
MM: favored for “datacom” (LAN) distances (~5 to 10 km). In principle cheaper than SM, but fiber itself may be more expensive since market is smaller (datacoms did not take off).
HEP distances are short (~100m), so SM/MM choice is not critical. Usually free to select the cheapest solution satisfying environmental specifications. Qualification is essential:
Early ECAL qualification of MM fiber of various suppliers: Good fiber loses <50% transmission after 1013 p/cm2. Bad fiber turns black.
P. Denes
7Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
First Overlay considerations
Serializer/deserializer considerations:
Encoding: Two principle protocols are CIMT (“G-Link”, Agilent proprietary) and 8-bit/10-bit. Both share the following characteristics:
• introduce an overhead of two bits per 8 bits of data• are usually employed continuously transmitting, either data or idle pattern• aid in maintaining synchronization of transmitter and receiver• allow some error checking• stabilize 0 and 1 levels by keeping number of bits of each the same
Principle protocol differences: G-Link is considered more robust in terms of maintaining and recovering synchronization. But it is proprietary. At the moment the fastest CIMT deserializer runs up to 1.4 Gb/s. In general there is no guarantee that a company employing proprietary technology will continue developing new products with it, nor any guarantee that it will not discontinue existing ones.
Speed and bits serialized: Typical scheme at LHC experiments is one (two) 16-bit words serialized at LHC clock rate of 40 MHz implying optical link speed of 800 (1600) Mb/s of which 80% is data and 20% is overhead.
Tx Rxfiber
Ser
Deser
“First Overlay”
encoding/decoding, usually performed in same chip as serialization/deserialization
8Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
General considerations
In designing an optical link for a HEP experiment:
Typically start with a shopping list of needs.
From that define specifications and select/develop technology.
End up with a laundry list of “issues”.
The typically O($1M) procurement for a HEP experiment is relatively small for any company large enough to collaborate in the task. Development teams in laser transmitters are smaller than one might think, typically 10-30 people for a large company. => The experiment will incur significant development and qualification effort and costs.
HEP experiments typically have many specifications which are unusual in industry. The closer the transmitter is to the center of the detector, the more likely this is to be true. Every need has its consequence(s):
radiation resistance, low power (heat), temperature cycle, non-magnetic materials
extensive qualification of several products (variation of behavior between different products can be large)
limited spacecustom laser package, with consequences for hermeticity and mounting techniques
long (3-10 yrs) development time
possibility that a product will be discontinued
100% reliability of ~10k links over 10 years
sleepless nights
9Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Data Link System Diagram
12Rx moduleRx module
12
1GOHGOH1212
1
96
DistributeDistributed Patch d Patch PanelPanel
Back-end Back-end Patch Patch PanelPanel
Off DetectorOff DetectorFront EndFront End Pigtail Pigtail
fiberfiber
RuggedizeRuggedized ribbond ribbon
Dense Dense multi-multi-ribbon ribbon cablecable
GOGOLL
Laser Laser diodediode In-Line In-Line
Patch Patch PanelPanel
CMCMSS
PIN photo-PIN photo-diode diode arrayarray
Digital Digital amp. amp. ASICASIC
1212
Edge-Emitting Laser Diode:ST Microelectronics
Digital OpticalReceiver: NGK
GOL ser. ASIC:CERN MIC
• GOL Opto-Hybrid (GOH): ECAL designs, prototypes, qualifies, defines manufacturing specifications, procures the manufacture, tests samples during production.• 12-channel NGK Rx: Off-the-shelf component: ECAL qualifies, procures, tests samples during production.• Fiber, connectors and adaptors: ECAL uses solutions already developed and procured for CMS Tracker.
10Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Components: Transmitter - 1
The transmitter of the data link is the GOH. Its principle components:
Some specifications of the GOH:• Receives LV, clock, control and 16-bit parallel digital input at 40 Mb/s from FE board via connector.• Transmits serialized optical output at 1310 nm, 800 Mb/s, either protocol, via pigtail fiber.• Output signal power ~-6dBm, depending on bias levels chosen. (0 dBm = 1 mW)
GOHGOH
• Designed by CERN Microelectronics group.• Implemented in 0.25μm CMOS technology employing radiation-tolerant layout practices.• Speeds of 0.8 and 1.6 Gb/s. ECAL will run the GOH at 0.8 Gb/s. • Two protocols (G-Link and 8b/10b). ECAL will use 8b/10b for data and G-Link for trigger.
• The Gigabit Optical Link (GOL) serializer and laser driver ASIC
• The laser diode, made by ST Microelectronics, based on a Mitsubishi die
• Custom-made for CMS Tracker (linear response), suitable also for digital use in ECAL (rise time consistent with 800 Mb/s operation).• Die wafer lots are radiation-qualified (gammas and neutrons) before assembly into laser diodes.• Pigtail fiber is attached and calibration data are taken by ST.
11Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Components: Transmitter - 2
Status of the GOH:
• The design is complete, prototypes exist.
• Functions well within the full data link (more on this later).
• Driven by GOH evaluation board (a modified GOL eval board), gives a clean eye diagram at 800 Mb/s.
12Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Components: Fiber and Connectors
Fiber and connectors are adopted from the CMS Tracker system
• All specifications consistent with use in ECAL as well (e.g. temperature, rad-hardness, attenuation, safety).
• ECAL benefits from commercial work, qualification and acceptance test setups already a part of Tracker program.
• ECAL’s contribution is the in-kind to staffing of acceptance tests.
12
12
1
96
MU-sMU adapter (Sumitomo)
MFS adapter (Diamond)
MFS-MPO multi-ribbon cable (Ericsson, Diamond)
sMU-MFS fanout (Ericsson, Sumitomo, Diamond)
13Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Components: Receiver
12-Channel digital NGK Rx
• COTS device; ECAL tasks are to qualify and procure.
• Specifications are consistent for use within the full data link, e.g.:
• wavelength: 1310 nm• speed: up to 1.25 Gb/s• saturation: -5 dBm• sensitivity: -18 dBm
• Functions well within the link (see following slides).
Rx moduleRx module
1212
PIN photo-PIN photo-diode diode arrayarray
Digital Digital amp. amp. ASICASIC
1212
14Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Data Link System Tests - 1
• Early system tests involved:• GOH driven by GOH evaluation board• Rx on Rx evaluation board• Deser (G-Link or 8b/10b) on Deser eval board
• counted Deser word errors (frame errors)…
Rx Err
Rx Out
ScopeClock Generator
Counter
Attenuator
GOH Evaluation Board
Rx Evaluation Board
Deser Evaluation Board
• …in this case, to study effect of crosstalk in the Rx• Conclusions:
• Link functions well, BER < 10-13 should be obtainable• Rx is well within sensitivity spec (-18 dBm)• Crosstalk costs about 2 dB of the optical power budget
~2 dB
Channel under study
Noise data generated on neighboring channels
15Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Data Link System Tests - 2
High = Des “Data Valid”
(Four) Trigger-generating pulses
High = Des “Rx Err”
• Four triggers generate four bursts of data.
• Each trigger involves the absence of one Control clock cycle.
• Valid data seen by deserializer, except occasionally soon after the missing clock cycle(s). It is expected that this will be resolved when QPLL (CERN MIC) is integrated into Front-end and Off-detector boards.
• The good news is that FE + Data Link works!
• First test of the Data Link with the prototype FPGA version of the Front-End board (mid-Feb 03):
16Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Optical Power Budget
Launched from GOH -6 dBm---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------
Laser diode efficiency 1 dB
Fiber and connector interfaces 3 dB
Rx crosstalk 2 dB---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------
Effective signal power at Rx -12 dBm
Rx sensitivity limit -18 dBm*
*Rx sensitivity spec is “-19 dBm typical, -18 dBm minimum”. We have so far observed better than –20 dBm.
=> At least 6 dB of margin in the power budget
17Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
GOH Assembly
• Manufacturing Specifications document for the GOH is complete. We plan to submit the RFQ as soon as possible to have the maximum amount of time to test the first 100 (Engineering Run).
• GOH assembly involves a number of steps:
Wire-bond protective cover attachment
Laser diode attachment and wire-bonding
Fiber clamp attachment
GOL BGA mounting
PCB manufacture, passivecomponent mounting
18Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
GOH QA and Link System Tests - 1
Construction of first 100 GOH (Engineering Run) will begin when all of these are ready:
• RFQ completed and manufacturer selected• Laser diodes are available• Tested GOL’s are available
We plan thorough tests to verify functionality and reliability.
• The tool for these tests will be our Bit Error Rate Test system. Based on PC, CMS “GIII DAQ” cards and extensive Altera programming.
• Principle is the same as for the frame error tests: plot BER vs. optical power, make a quantitative measure of a stress on the system in terms of a “penalty” on the optical power budget.
• Advantages over frame errors are that it is bit-by-bit, independent of protocol.
• BERT system has been working since beginning of the year. Presently runs at 100 Mb/s: BER of <10-13 in ~1 day.
• Present effort is to automate it to perform many measurements with minimal human intervention.
19Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
GOH QA and Link System Tests - 2
• The functionality tests we plan include:
Temperature sensitivity: GOH, Rx, deserializer.
Sensitivity to LV fluctuations/noise.
Fine-tuning of the eye diagram via GOH compensation network components.
Integration of the Link into the full ECAL electronics chain.
• The reliability tests planned for the GOH include:
Vibration
Mechanical shock
Thermal cycling (-20°C to +40°C)
Pass/fail eye diagram test
Irradiation
Accelerated aging
Long-term stability
GOHGOH
• The goal of the tests is to uncover and resolve any design or manufacturing reliability issues that may exist, and to be able to sign off on the full Link system design by the October ESR.
Manufacturer
Data Links group
20Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
ScheduleECAL Data Links Schedule 2003 2004 2005JG / 02 Apr 03 Apr May Jun Jul Aug Sep Q4 03 Q1 04 Q2 04 Q3 04 Q4 04 Q1 05 Q2 05
1. Test Beam GOH Assembly
2. GOH Assembly Procurement
3. Rx Procurement
4. GOH Engineering Run (first 100)Assembly and Testing by ManufacturerTesting by Data Links group
5. Milestone: October ECAL ESR
6. Full ECAL Electronics ChainSystem TestBeam Test
7. GOH Pilot Run (next 500)Assembly and Testing by ManufacturerTesting by Data Links group
8. Production for ECAL Barrel + PreshowerGOH and Fiber ProductionAcceptance testingInstallation
9. Production for ECAL EndcapGOH and Fiber ProductionAcceptance testingInstallation
Schedule concerns: May 03: If laser diodes or tested GOL are late, time for testing of Eng Run is reduced. Sep 03: If GOH design modification is necessary, little time to test it before the Pilot Run.
Qualification is crucialbut is the only “compressible” task.
21Jim Grahl, University of Minnesota
ECALECAL
HEP seminar, U. Minnesota, April 29th 2003
Summary
• Development of an optical link for a HEP experiment is a fascinating project, invariably involves a lot of co-development with industry.
• ECAL Data Link system design and project have made a lot of progress in a short time. Starting with good components helps.