THE PON-SCOPE GRID INFRASTRUCTURE
P.I. – Giuseppe Marrucci
Astrophysics – Longo
Electromagnetism - Franceschetti
High energy Physics - Merola
Computer sciences - Russo
Mathematics - Murli
Materials and environment - Barone
Medicine and Genetics - Salvatore
Social and Human sciences - Zollo
GARR
PI2S2
GARR Altri Enti e realtàGARR
PI2S2PI2S2
GARR Altri Enti e realtà
The four PON’s
THE PON-SCOPE GRID INFRASTRUCTURE
The Metropolitan Grid VST-Cen
The core of the infrastructure
Scala: 1:100
Liv.1 ed G
CentroCentroPON S.CO.P.EPON S.CO.P.E..
DDipartimento di ipartimento di SScienze cienze FFisicheisicheINFNINFNEdificio GEdificio G
Cabina elettricaG.E. 1 MW
Tier2Tier2––ATLASATLASPrototipo Prototipo SCoPESCoPE
CAMPUS GRIDMSA
DMADiChi
DSF INFN
C.S.I.
GARR
Fibra ottica
PON PON S.CO.P.ES.CO.P.E..
CAMPUS GRIDMSA
DMADiChi
DSF INFN
C.S.I.
GARR
Fibra ottica
PON PON S.CO.P.ES.CO.P.E..
LAN INFN
1 Gbps
1 Gbps1 Gbps
1 Gbps
2 x1Gbps
100/200 Mbps
10 Gbps
1 Gbps
10 Gbps
1 Gbps
N x1 Gbps
2.5 GbpsCampus Grid
GARR
MAN UNINA
Siti Campus Grid
Tier 2Pon UniNA
LAN INFN
1 Gbps
1 Gbps1 Gbps
1 Gbps
2 x1Gbps
100/200 Mbps
10 Gbps
1 Gbps
10 Gbps
1 Gbps
N x1 Gbps
2.5 GbpsCampus Grid
GARR
MAN UNINA
Siti Campus Grid
Tier 2Pon UniNA
250 CPU quadri-processoriXeon Quad-core EM64T Clovertown E5320, 1.86 GHz, 2x4MB cache, supportati da 8
GB di RAM, 1066 FSB, con 2 dischi SAS 36GB, controller RAID SAS 3 Gb/s. Storage: 100 HDD FC / 300GB (30TB) + 100 HDD SATA 2 da 500GB (50TB),
expandable up to 120TB & 240 HD4 cpu dual core 16GB RAMRetenon blocking with 240 porte 10/20 gigabit infiniband full redundant, 12 PS “banda
aggregata” 12 Gigabit, 2 gateway fiber channel-infiniband with 2 gates FC4 - 8 gigabit. 230 nodi in infiniband; 20 blade forfibre channel connectivity.
PON-SCOPEPreexistence
Campus GRID AstrogridSM LHC Tier 2 (INFN)
For a total 0f 1 k-CPU’s
Astrophysics in GRID-SCOPE• Astroparticles
• Simulations of CR and induced showers – F. Guarino• Simulation of NEMO - G. Barbarino• Simulations of Gravitational Waves from various astrophysical sources
VIRGO – L. Milano• Cosmology
• Simulations of cosmic string signatures on CMB – G. Longo• Primordial Nucleosynthesis – G. Miele
• VO related activity• Pipeline for survey data (VST-Tube + 2dphot) – de Carvalho, Grado, La
Barbera, Longo• Integration of ASTROGRID with GRID-SCOPE (VO-Tech broker)• Extractor (image segmentation) – O. Laurino• Data Mining • Supervised and unsupervised data mining tools
Photometric redshiftsQSO and AGN search &
classification
Cluster identification & characterization
G. Longo (PI) M. Brescia (INAF - PM)
S. Cavuoti (applications) A. Corazza (models and algorithms)R. D’Abrusco (applications) G. d’Angelo (documentation, GRID)N. Deniskina (GRID – VO interface) M. Garofalo (applications)O. Laurino (System, Applications) A. Nocella (UML software engineering)G. Riccio (Applications) B. Skordovski (models)
External Members
C. Donalek (Caltech-CRAC) G. Djorgovski (Caltech-CRAC)
The VO-Neural Teamhttp://people.na.infn.it/~astroneural/
THE PROBLEM
Grid Launcher (N.V. Deniskina) allows to launch GRID-applications using the ASTROGRID Workbench and to transfer the results from the GRID-UI to the data storage of Astrogrid.
• VO is an environment open to a wide community & the GRID IS NOT (access through personal certificates)
• Time consuming tasks cannot be run from VO users unless the security problem is solved (or “fooled”….)
GRID-Launcher (N.V. Deniskina)
Workbench of the user
GRID SCOPE
AstroGRIDMySpace
(data storage)
UI
RB
SE
WN CE
data
result
result
The workflow of the job is following:
1. “Grid_launcher“ a) takes the user input from Workbench of Astrogrid; b) collects all files, tabs and programs needed; c) wraps them in an archive and sends it to the Scope-GRID UI. (The Authentication on Scope is done by public keys exchange).
2. The Scope UI receives data and JDL program from "GRID_launcher", unpacks them and translates them to Grid job format.
3. Once GRID job jdl file is ready, "GRID_launcher" starts it in Grid (from a AstroGrid node); periodically checks the status; and then (when job is finished) retrieves the results.
5. "GRID_launcher" receives the data archive, unpacks it and puts the results into the “MySpace” data storage of AstroGRID.
GRID launcher has been implemented and tested on :
•VONeural_MLP: supervised clustering
•VONeural_SVM: supervised clustering
•Sextractor: for survey data processing
•SWARP - is a program to resample and co-add FITS images using any arbitrary astrometric projection defined in the WCS standard.
Tests on scientific cases (done and in progress)
•Photometric redshifts of SDSS galaxies (D’Abrusco et al. ApJ, 2007) uses VONeural_MLP
•Classification of AGN in UKIDS+SDSS data (D’abrusco et al. 2008, MNRAS in press) Using VONeural_SVM
•Search for LSB in SDSS data using NEXT (in progress, Laurino)
SDSS-DR4/5 – GG
training validation Test set 60%, 20%, 20%
MLP, 1(5), 1(18)
0.01<Z<0.25 0.25<Z<0.50 99.6 % accuracy
MLP, 1(5), 1(23) MLP, 1(5), 1(24)
rob = 0.206 rob = 0.234
Interpolationof systematic errors
Interpolationof systematic errors
Phot Z for SDSS General Galaxy sampleat least 30 experiments (10-12 h/each)training on 350.000 objects 12 features
results for 32.000.000 objects
σz = 0.02
Redshifts for 30 million galaxies
Photometric redshifts for 30 million SDSS galaxies
Looking for AGN candidates in SDSS+UKIDS
3-D PCA
PPS
SDSS UKIDSS
preprocessing
clustering
labeling
BoK
results
PPS/SVM
NEC
dendrogram
Cluster optimization
1 experiment/one node ca. 11 days
Looking for AGN candidates in SDSS+UKIDS
3-D PCA
Applicazione 2 con SVMApplicazione 2 con SVMMiglior Risultato: 81.5%Miglior Risultato: 81.5%
PON-SCOPE GRID Infrastructure (110 nodes PON NA-CA-CT)
lg2(gamma)
lg2(C)
What types of programs we plan to launch to GRID?
There are typical astronomical tasks that need long-time calculations:
1) all types of numerical simulations.,2) image reduction (+, -, statistic, calibration).,3) search of astronomical solution (astrometry calibration).,4) photometry calibration.,5) determination of luminosity function of the galaxies., 6) photometric redshift.,7) source selection (clustering, determination of the type of the object) (mushroom as an example). , These tasks can be as AG programs (or other VO programs ) inside of AG as user's programs.
These tasks can be the sequence of programs (not one program).