Upload
devlin
View
29
Download
0
Embed Size (px)
DESCRIPTION
Erlang in global radio astronomy. Controlling embedded hardware. Harro Verkouter/Joint Institute for VLBI in Europe. Joint Institute for VLBI in Europe. V ery L ong B aseline I nterferometry. 5m. 100m. Ever larger telescopes. 15cm. Electromagnetic radiation. wavelength. diameter. - PowerPoint PPT Presentation
Citation preview
Controlling embedded hardware
Erlang in global radio astronomy
Harro Verkouter/Joint Institute for VLBI in Europe
Joint Institute for VLBI in Europe
VeryLongBaselineInterferometry
VeryLongBaselineInterferometry
Ever larger telescopes
15cm
5m 100m
Electromagnetic radiation
(m) D (m) /D (rad)
smallest optical 10-7 10-1 10-6
biggest radio 10-1 100 10-3
wavelength diameter pixelsize
A neat trick
100m
A neat trick
100m
20m
correlator
10,000km
A different ball game!
(m) D (m) /D (rad)
largest optical (IR) 10-6 10 10-6
VLBI (radio) 10-1 107 10-8
Science
Science
Recording the data
Record up to 4096 Mbps (512 Mbyte/s)
Number of combinations to be computed as function of n
~n2
16 telescopesat
1024 Mbps
UniBoard
QuickTime™ and a decompressor
are needed to see this picture.
4 frontnode
4 backnode
1Gbit/s ethernet
16x 10Gbps
16x 10Gbps
>20 Gbps/link
Erlang?!
• typical languages used– 80% C++– 15% Python– PHP, Fortran, perl
• database– MySQL
• needed to convince management– pilot projects!
Decoding data files
• VLBI has framed data format(s)
QuickTime™ and a decompressor
are needed to see this picture.
• Look for ‘syncword’ – different flavours, same syncword, different framesize
• Other format uses different syncword
(not quite) surprising little code
• 212 lines of code– first dabblings ...
• about a week’s work• recognizes + decodes 3 formats• decodes the BCD timestamps• supports distributed system
– reading file on one machine– decoding on other machine
• management duely impressed– got permission to do a larger pilot!
Soft real time monitoring website
• webcams at stations– webpage with http links = #FAIL– unreliable links
• really high latency (China, Chile)• don’t want broken images
– low bandwidth• cache in Europe: multiple users eat all bandwidth
• parse telescope logfiles– immediate antenna calibration info– generate timeseries plots as observation progresses
• stored in MySQL
Soft real time monitoring website
• webcam URLs + status stored in MySQL– supervisor starts webcam monitors
• one gen_server for each webcam– loads info from MySQL– uses http:request/1– strict checking on return value
• only HTTP 200 status replies• content_length header set?• explode if problem
– use a port to ImageMagick to scale image– write binary JPEG data into mnesia
Soft real time monitoring website
• monitor all participating telescopes• telescope log file (ASCII text)
– retrieved via python + ssh – read into Erlang using a port– low data rate after start up
• parses lines– update MySQL
• generate plots using gnuplot – for each telescope– query MySQL every so many minutes– using a port– write .png binary data to mnesia
Soft real time monitoring website
• patch it all together using webtool• cache complete webpage in mnesia
– too old? generate new one
• serve .png and .jpg data from mnesia– image URL points back to webtool
• extremely simple to cache + serve data– brilliant idea: mapping URL to erlang fn call!
• in total used ~3000 lines of code– webcam, logfile monitoring; plotting; serving web pages– excellent test case for OTP and distributed system
• got the green light for the real thing!
UniBoardEthernetSwitch
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
FPGA
Nios2CPU
1Gbit PHY
Control Computerwindows/unix
EthernetSwitch
VHDLHardwareSoftware
FPGA
1Gbit PHY
VHDLHardwareSoftware
Polyphase Filterbank
registers
Delay Module
registers
Nios2CPU
registers
registers
VHDLHardwareSoftware
Control Computerwindows/unix
Nios2CPU
Client
Server
UDP/IPv4 over ethernet
Simple binary protocolRead N 32bit words starting from START ADDRESS
(Over)Write N 32bit words from the packet to START ADDRESS
Read/Modify/Write N 32bit words from START ADDRESS + packet to START ADDRESS
32 bits 32 bits 32 bits 0x01 N START ADDRESS
32 bits 32 bits 32 bits N*32 bits 0x02 N START ADDRESS N 32 bit words of DATA
32 bits 32 bits 32 bits N*32 bits OPCODE N START ADDRESS N 32bit words of MASK
OPCODE 0x03 0x04 0x05
BITWISE OPERATION AND OR XOR
QuickTime™ and a decompressor
are needed to see this picture.
FPGA client library
Registermap:
• list of register definitions
• type of the register (details follow)
• name of the register (symbolic)
• address of the register in the memory map
one controller one fpga
FPGA client library
fpga:bit( <name>, <bit #>, <address> )
fpga:bitrange( <name>, <startbit>, <nbit>, <address> )
fpga:word( <name>, <address> )
fpga:range( <name>, <# words>, <address> )
Available registerdefinition types for the controller
FPGA client library
fpga:read( <name> )
fpga:write( <name>, <value> )
fpga:or_( <name>, <value> )
fpga:and_( <name>, <value> )
fpga:xor_( <name>, <value> )
Available register commands
Example
-module(firfilter). %% this module is called firfilter, driving one of those-behaviour(personality).-export([registers/0, start_filter/2]). %% this module implements an FPGA personality
%% Called automatically by the FPGA control frameworkregisters() -> {ok, [
fpga:bitrange(num_tap, 5, 3, 16#24),fpga:bit(pps_enable, 28, 16#24),fpga:word(control_status, 16#abcc),
].
%% define a high-level command for this personalitystart_filter(FPGA, NumTap) ->
%% disable the PPSfpga:execute(FPGA, fpga:write(pps_enable, 0)),%% read the control status registercase fpga:execute(FPGA, fpga:read(control_status)) of
1 -> fpga:execute(FPGA, fpga:xor(control_status, 42));_ -> ok
end,%% however, you can easily execute a number of commands in one go:fpga:execute(FPGA, [fpga:write(num_tap, NumTap),
fpga:write(pps_enable, 1)]).
Safety
2> fpga:execute(FPGA, fpga:write(pps_enable, 2)).
OH NOES! error:"Value '2' does not fit in one bit (pps_enable)"
FPGAControl terminating [{{badmatch, {reply, {error, {"Value '2' does not fit in one bit (pps_enable)", [{fpga,err,2, [{fil
Full correlator control system
• besides single fpga control– parallel control of eight fpga’s
• parsing + validating configuration files– using yecc
• decoding + sending over UDP of binary data files– data as well as polynomial coefficients
• capture network data to file• configure and control the hardware• MySQL read/write• two weeks VHDL coding = < 1 day Erlang coding
data reader
data reader
data reader
data writerUniBoardUniBoard
command/control
Full correlator control system
The good
• cross platform– we have Linux, Mac OSX and Solaris
• distributed– typically many machines involved
• non-variability of ‘variables’• conciseness / amount of work per LOC• connects to MySQL
– not easy to connect other languages to mnesia
• hard crash on errors– about the most valuable property: can’t ignore!
• binary pattern matching– every language should be jealous of this
• re-use of development/testing code– it’s so easy to re-use code!
The bad
• our VHDL engineers struggle– language simple, but– concepts like recursion + distribution are ‘hard’
• conciseness / amount of work per LOC– takes effort to read code written by someone who’s better at
it than you
• deploying a developing distributed system– we haven’t found a simple way yet– dynamic remote code loading could be awsome
• it works for individual modules• useless if you have dynamic dependencies ...
– erlang:apply/3
Thank you for your attention!