Upload
isaura
View
49
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Emulsion Database Design Status Report. Extends Opera Internal Note # 38: “ Database Architecture for the European Emulsion Scanning System”. Updated DB Schema. Distributed DB Implementation. DB Client Technologies. DB Client Libraries. Conclusions. Cristiano Bozza European Emulsion Group - PowerPoint PPT Presentation
Citation preview
Emulsion Database DesignEmulsion Database DesignStatus ReportStatus Report
Cristiano BozzaCristiano BozzaEuropean Emulsion GroupEuropean Emulsion Group
LNGS, May 2003LNGS, May 2003
Updated DB Schema
Distributed DB Implementation
DB Client Technologies
DB Client Libraries
Conclusions
Extends Opera Internal Note # 38:“Database Architecture for the European Emulsion Scanning System”
Updated DB SchemaUpdated DB Schema
Event Reconstruction
FK_ZONES_ALIGNEDZONES
FK_ALIGNEDBASE_MIPBASE
FK_VOLTKS_ALIGNBASE
FK_USERS_PROGSETTINGS
FK_PROGSETTING_BATCHES
FK_PROCOPERA_RECON
FK_RECON_ALIGNEDZONE
FK_ALIGN_ZONESTRACKS
FK_PROCOPS_VTXLOCS
FK_BRICKS_PROCESSES
FK_OPERATIONS_PROCOPERATIONS
FK_MACHINES_PROCOPERATIONS
FK_PROCESSES_PROCOPS
FK_USERS_PRIVILEGES
FK_SITES_PRIVILEGES
FK_BATCHES_MACHINES
FK_BATCHES_REQUESTER
FK_MACHINES_SITE
FK_MARKS_PLATE
FK_MARKS_TEMPLATEMARK
FK_PLATES_BRICK
FK_TEMPLATEMARKSETS
FK_SITES_USERS
FK_ZONES_MIPEMU
FK_BATCHES_ZONES
FK_ZONES_MIPBASE
FK_BASE_UPTRACK
FK_BASE_DOWNTRACK
FK_PREDICTIONS_PREDTRACKS
FK_PHYSIDS_PREDTRACKS
FK_PROCSTART_PREDTKS
FK_PROCSTARTUPS_PROCS
FK_PROCSTART_SELECTION
FK_VTXLOC_PLATES_SELS
FK_SELECTIONS_OPS
FK_VTXLOC_SEL_VSEL
FK_VTXLOC_USERS_VSELS
FK_VTXLOC_VSEL_SELTKS
FK_VTXLOC_MIPBASE_SELTKS
FK_VTXLOC_SELRSN_SELTKS
FK_RECON_USERS
FK_PROGSETTING_RECON
FK_RECON_ALIGNEDTRACKS
FK_RECON_VOLTKS
FK_BATCHES_PLATES
FK_RECON_VERTICES
FK_VOLTKS_DOWNVTX
FK_VOLTKS_UPVTX
FK_PHYSIDS_VOLTKS
FK_VERTEXTYPES_VERTICES
TB_BATCHES
IDID_PLATEID_MACHINEID_PROGRAMSETTINGSID_REQUESTERTEMPLATEMARKSREQUESTTIMESTARTTIMEENDTIME
<pk><fk4><fk2><fk1><fk3>
TB_BRICKS
IDMINXMAXXMINYMAXYMINZMAXZ
<pk>
TB_MACHINES
IDNAMEADDRESSISSCANNINGSERVERISBATCHSERVERISDATAPROCESSINGSERVERISWEBSERVERISDATABASESERVERISLOCALMASTERCONTROLLERID_SITE
<ak2><pk><ak1>
<fk>
TB_MARKS
IDID_PLATEID_TEMPLATEMARKPOSXPOSY
<ak><pk,fk1><fk2>
TB_OPERATIONS
IDNAMEISSCANNINGISTRANSPORT
<ak><pk>
TB_PLATES
IDID_BRICKPLATECALIBRATIONZISCSEXPOSURESTARTEXPOSUREFINISHMAPXXMAPXYMAPYXMAPYYMAPDXMAPDY
<ak><pk,fk><pk>
<pk>
TB_SITES
IDNAMELATITUDELONGITUDELOCALTIMEFUSE
<ak><pk>
TB_TEMPLATEMARKSETS
IDID_BRICKID_MARKPOSXPOSYMARKROWMARKCOLSHAPE
<ak><pk,fk><pk>
<pk><pk>
TB_ZONES
IDID_BATCHMINXMAXXMINYMAXYRAWDATAPATHSTARTTIMEENDTIMEBASETHICKNESS
<pk><fk>
TB_MIPEMULSIONTRACKS
IDID_ZONESIDEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk>
TB_MIPBASETRACKS
IDID_ZONEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMAID_DOWNTRACKID_UPTRACK
<pk><fk1>
<fk3><fk2>
TB_PREDICTIONS
IDRunEventXYZ
<ak><pk><pk>
TB_PREDICTEDTRACKS
IDID_PREDICTIONID_TRACKID_PHYSICSPOSXPOSYPOSZSLOPEXSLOPEYCHI2
<ak><pk,fk1><pk><fk2>
TB_PHYSICSIDS
IDDESCRIPTION
<ak><pk>
TB_PROCSTARTUPSELECTIONS
ID_PROCESSID_PREDICTEDTRACKID_SELECTION
<pk,fk2><pk,fk1><pk,fk3>
TB_VTXLOC_SELECTIONS
IDID_PLATEID_PROCESSOPERATIONID_SELECTORID_SELECTIONTEXT
<ak><pk,fk2><pk,fk1><fk4><fk3>
TB_VTXLOC_SELTRACKS
IDID_VTXLOC_SELID_BASETRACKID_MOTHERTRACKID_REASON
<ak><pk,fk1><pk,fk2>
<fk3>
TB_VTXLOC_SELECTION_REASONS
IDDESCRIPTION
<ak><pk>
TB_ALIGNEDBASETRACKS
IDID_MIPBASETRACKID_RECONSTRUCTIONID_ALIGNEDZONEID_VOLUMETRACKPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk1><fk4><fk3><fk2>
TB_RECONSTRUCTIONS
IDID_RECONSTRUCTORISCOMPLETEID_PROGRAMSETTINGSID_PROCESSOPERATION
<pk><fk2>
<fk3><fk1>
TB_PROGRAMSETTINGS
IDMODULENAMEMODULEIDENTITYSETTINGSID_AUTHOR
<ak><pk><pk>
<fk>
TB_VOLUMETRACKS
IDID_RECONSTRUCTIONSIGMADOWNZUPZDOWNPOSXDOWNPOSYDOWNPOSZDOWNSLOPEXDOWNSLOPEYUPPOSXUPPOSYUPPOSZUPSLOPEXUPSLOPEYMOMENTUMENERGYID_PARTICLERECONSTRUCTIONNOTESID_DOWNVERTEXID_UPVERTEX
<pk><fk1>
<fk4>
<fk2><fk3>
TB_VERTICES
IDID_RECONSTRUCTIONPOSXPOSYPOSZAVERAGEDISTANCEID_VERTEXTYPE
<pk><fk1>
<fk2>
TB_VERTEXTYPES
IDDESCRIPTION
<ak><pk>
Bricks and Plates
Predictions and Final Results with Physics Info
System Management
Vertex Location
Scanning Data
Scanning Predictions
FK_SELECTIONS_USERS
OPERA European Emulsion Scanning DB
TB_SELECTIONS
IDID_OPERATIONID_SELECTORSELECTIONTEXTSELECTIONDATEDESCRIPTION
<ak><pk,fk1><pk,fk2>
<pk><pk>
TB_USERS
IDUSERNAMEPWDNAMESURNAMEINSTITUTIONID_SITEEMAILADDRESSPHONE
<ak1><ak2>
<pk><pk>
<fk>
TB_PROCESSES
IDID_BRICKSTARTTIMEFINISHTIME
<pk><fk>
TB_PRIVILEGES
ID_USERID_SITEREQUESTSCANREQUESTWEBANALYSISREQUESTDATAPROCESSINGREQUESTDATADOWNLOADREQUESTPROCESSSTARTUP
<pk,fk1><pk,fk2>
TB_PROC_OPERATIONS
IDID_PROCESSID_OPERATIONID_MACHINESTARTTIMEFINISHTIME
<ak><pk,fk3><pk,fk1><fk2>
TB_ALIGNEDZONES
IDID_ZONEID_RECONSTRUCTIONPLATEMAPXXMAPXYMAPYXMAPYYMAPDXMAPDYMAPDZSLOPEDSXSLOPEDSYSLOPEMSXSLOPEMSYREFXREFYREFZDOWNZUPZ
<ak><pk,fk1><pk,fk2>
Updated DB SchemaUpdated DB Schema
Event Reconstruction
FK_ZONES_ALIGNEDZONES
FK_ALIGNEDBASE_MIPBASE
FK_VOLTKS_ALIGNBASE
FK_USERS_PROGSETTINGS
FK_PROGSETTING_BATCHES
FK_PROCOPERA_RECON
FK_RECON_ALIGNEDZONE
FK_ALIGN_ZONESTRACKS
FK_PROCOPS_VTXLOCS
FK_BRICKS_PROCESSES
FK_OPERATIONS_PROCOPERATIONS
FK_MACHINES_PROCOPERATIONS
FK_PROCESSES_PROCOPS
FK_USERS_PRIVILEGES
FK_SITES_PRIVILEGES
FK_BATCHES_MACHINES
FK_BATCHES_REQUESTER
FK_MACHINES_SITE
FK_MARKS_PLATE
FK_MARKS_TEMPLATEMARK
FK_PLATES_BRICK
FK_TEMPLATEMARKSETS
FK_SITES_USERS
FK_ZONES_MIPEMU
FK_BATCHES_ZONES
FK_ZONES_MIPBASE
FK_BASE_UPTRACK
FK_BASE_DOWNTRACK
FK_PREDICTIONS_PREDTRACKS
FK_PHYSIDS_PREDTRACKS
FK_PROCSTART_PREDTKS
FK_PROCSTARTUPS_PROCS
FK_PROCSTART_SELECTION
FK_VTXLOC_PLATES_SELS
FK_SELECTIONS_OPS
FK_VTXLOC_SEL_VSEL
FK_VTXLOC_USERS_VSELS
FK_VTXLOC_VSEL_SELTKS
FK_VTXLOC_MIPBASE_SELTKS
FK_VTXLOC_SELRSN_SELTKS
FK_RECON_USERS
FK_PROGSETTING_RECON
FK_RECON_ALIGNEDTRACKS
FK_RECON_VOLTKS
FK_BATCHES_PLATES
FK_RECON_VERTICES
FK_VOLTKS_DOWNVTX
FK_VOLTKS_UPVTX
FK_PHYSIDS_VOLTKS
FK_VERTEXTYPES_VERTICES
TB_BATCHES
IDID_PLATEID_MACHINEID_PROGRAMSETTINGSID_REQUESTERTEMPLATEMARKSREQUESTTIMESTARTTIMEENDTIME
<pk><fk4><fk2><fk1><fk3>
TB_BRICKS
IDMINXMAXXMINYMAXYMINZMAXZ
<pk>
TB_MACHINES
IDNAMEADDRESSISSCANNINGSERVERISBATCHSERVERISDATAPROCESSINGSERVERISWEBSERVERISDATABASESERVERISLOCALMASTERCONTROLLERID_SITE
<ak2><pk><ak1>
<fk>
TB_MARKS
IDID_PLATEID_TEMPLATEMARKPOSXPOSY
<ak><pk,fk1><fk2>
TB_OPERATIONS
IDNAMEISSCANNINGISTRANSPORT
<ak><pk>
TB_PLATES
IDID_BRICKPLATECALIBRATIONZISCSEXPOSURESTARTEXPOSUREFINISHMAPXXMAPXYMAPYXMAPYYMAPDXMAPDY
<ak><pk,fk><pk>
<pk>
TB_SITES
IDNAMELATITUDELONGITUDELOCALTIMEFUSE
<ak><pk>
TB_TEMPLATEMARKSETS
IDID_BRICKID_MARKPOSXPOSYMARKROWMARKCOLSHAPE
<ak><pk,fk><pk>
<pk><pk>
TB_ZONES
IDID_BATCHMINXMAXXMINYMAXYRAWDATAPATHSTARTTIMEENDTIMEBASETHICKNESS
<pk><fk>
TB_MIPEMULSIONTRACKS
IDID_ZONESIDEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk>
TB_MIPBASETRACKS
IDID_ZONEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMAID_DOWNTRACKID_UPTRACK
<pk><fk1>
<fk3><fk2>
TB_PREDICTIONS
IDRunEventXYZ
<ak><pk><pk>
TB_PREDICTEDTRACKS
IDID_PREDICTIONID_TRACKID_PHYSICSPOSXPOSYPOSZSLOPEXSLOPEYCHI2
<ak><pk,fk1><pk><fk2>
TB_PHYSICSIDS
IDDESCRIPTION
<ak><pk>
TB_PROCSTARTUPSELECTIONS
ID_PROCESSID_PREDICTEDTRACKID_SELECTION
<pk,fk2><pk,fk1><pk,fk3>
TB_VTXLOC_SELECTIONS
IDID_PLATEID_PROCESSOPERATIONID_SELECTORID_SELECTIONTEXT
<ak><pk,fk2><pk,fk1><fk4><fk3>
TB_VTXLOC_SELTRACKS
IDID_VTXLOC_SELID_BASETRACKID_MOTHERTRACKID_REASON
<ak><pk,fk1><pk,fk2>
<fk3>
TB_VTXLOC_SELECTION_REASONS
IDDESCRIPTION
<ak><pk>
TB_ALIGNEDBASETRACKS
IDID_MIPBASETRACKID_RECONSTRUCTIONID_ALIGNEDZONEID_VOLUMETRACKPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk1><fk4><fk3><fk2>
TB_RECONSTRUCTIONS
IDID_RECONSTRUCTORISCOMPLETEID_PROGRAMSETTINGSID_PROCESSOPERATION
<pk><fk2>
<fk3><fk1>
TB_PROGRAMSETTINGS
IDMODULENAMEMODULEIDENTITYSETTINGSID_AUTHOR
<ak><pk><pk>
<fk>
TB_VOLUMETRACKS
IDID_RECONSTRUCTIONSIGMADOWNZUPZDOWNPOSXDOWNPOSYDOWNPOSZDOWNSLOPEXDOWNSLOPEYUPPOSXUPPOSYUPPOSZUPSLOPEXUPSLOPEYMOMENTUMENERGYID_PARTICLERECONSTRUCTIONNOTESID_DOWNVERTEXID_UPVERTEX
<pk><fk1>
<fk4>
<fk2><fk3>
TB_VERTICES
IDID_RECONSTRUCTIONPOSXPOSYPOSZAVERAGEDISTANCEID_VERTEXTYPE
<pk><fk1>
<fk2>
TB_VERTEXTYPES
IDDESCRIPTION
<ak><pk>
Bricks and Plates
Predictions and Final Results with Physics Info
System Management
Vertex Location
Scanning Data
Scanning Predictions
FK_SELECTIONS_USERS
OPERA European Emulsion Scanning DB
TB_SELECTIONS
IDID_OPERATIONID_SELECTORSELECTIONTEXTSELECTIONDATEDESCRIPTION
<ak><pk,fk1><pk,fk2>
<pk><pk>
TB_USERS
IDUSERNAMEPWDNAMESURNAMEINSTITUTIONID_SITEEMAILADDRESSPHONE
<ak1><ak2>
<pk><pk>
<fk>
TB_PROCESSES
IDID_BRICKSTARTTIMEFINISHTIME
<pk><fk>
TB_PRIVILEGES
ID_USERID_SITEREQUESTSCANREQUESTWEBANALYSISREQUESTDATAPROCESSINGREQUESTDATADOWNLOADREQUESTPROCESSSTARTUP
<pk,fk1><pk,fk2>
TB_PROC_OPERATIONS
IDID_PROCESSID_OPERATIONID_MACHINESTARTTIMEFINISHTIME
<ak><pk,fk3><pk,fk1><fk2>
TB_ALIGNEDZONES
IDID_ZONEID_RECONSTRUCTIONPLATEMAPXXMAPXYMAPYXMAPYYMAPDXMAPDYMAPDZSLOPEDSXSLOPEDSYSLOPEMSXSLOPEMSYREFXREFYREFZDOWNZUPZ
<ak><pk,fk1><pk,fk2>
Updated DB SchemaUpdated DB Schema
Event Reconstruction
FK_ZONES_ALIGNEDZONES
FK_ALIGNEDBASE_MIPBASE
FK_VOLTKS_ALIGNBASE
FK_USERS_PROGSETTINGS
FK_PROGSETTING_BATCHES
FK_PROCOPERA_RECON
FK_RECON_ALIGNEDZONE
FK_ALIGN_ZONESTRACKS
FK_PROCOPS_VTXLOCS
FK_BRICKS_PROCESSES
FK_OPERATIONS_PROCOPERATIONS
FK_MACHINES_PROCOPERATIONS
FK_PROCESSES_PROCOPS
FK_USERS_PRIVILEGES
FK_SITES_PRIVILEGES
FK_BATCHES_MACHINES
FK_BATCHES_REQUESTER
FK_MACHINES_SITE
FK_MARKS_PLATE
FK_MARKS_TEMPLATEMARK
FK_PLATES_BRICK
FK_TEMPLATEMARKSETS
FK_SITES_USERS
FK_ZONES_MIPEMU
FK_BATCHES_ZONES
FK_ZONES_MIPBASE
FK_BASE_UPTRACK
FK_BASE_DOWNTRACK
FK_PREDICTIONS_PREDTRACKS
FK_PHYSIDS_PREDTRACKS
FK_PROCSTART_PREDTKS
FK_PROCSTARTUPS_PROCS
FK_PROCSTART_SELECTION
FK_VTXLOC_PLATES_SELS
FK_SELECTIONS_OPS
FK_VTXLOC_SEL_VSEL
FK_VTXLOC_USERS_VSELS
FK_VTXLOC_VSEL_SELTKS
FK_VTXLOC_MIPBASE_SELTKS
FK_VTXLOC_SELRSN_SELTKS
FK_RECON_USERS
FK_PROGSETTING_RECON
FK_RECON_ALIGNEDTRACKS
FK_RECON_VOLTKS
FK_BATCHES_PLATES
FK_RECON_VERTICES
FK_VOLTKS_DOWNVTX
FK_VOLTKS_UPVTX
FK_PHYSIDS_VOLTKS
FK_VERTEXTYPES_VERTICES
TB_BATCHES
IDID_PLATEID_MACHINEID_PROGRAMSETTINGSID_REQUESTERTEMPLATEMARKSREQUESTTIMESTARTTIMEENDTIME
<pk><fk4><fk2><fk1><fk3>
TB_BRICKS
IDMINXMAXXMINYMAXYMINZMAXZ
<pk>
TB_MACHINES
IDNAMEADDRESSISSCANNINGSERVERISBATCHSERVERISDATAPROCESSINGSERVERISWEBSERVERISDATABASESERVERISLOCALMASTERCONTROLLERID_SITE
<ak2><pk><ak1>
<fk>
TB_MARKS
IDID_PLATEID_TEMPLATEMARKPOSXPOSY
<ak><pk,fk1><fk2>
TB_OPERATIONS
IDNAMEISSCANNINGISTRANSPORT
<ak><pk>
TB_PLATES
IDID_BRICKPLATECALIBRATIONZISCSEXPOSURESTARTEXPOSUREFINISHMAPXXMAPXYMAPYXMAPYYMAPDXMAPDY
<ak><pk,fk><pk>
<pk>
TB_SITES
IDNAMELATITUDELONGITUDELOCALTIMEFUSE
<ak><pk>
TB_TEMPLATEMARKSETS
IDID_BRICKID_MARKPOSXPOSYMARKROWMARKCOLSHAPE
<ak><pk,fk><pk>
<pk><pk>
TB_ZONES
IDID_BATCHMINXMAXXMINYMAXYRAWDATAPATHSTARTTIMEENDTIMEBASETHICKNESS
<pk><fk>
TB_MIPEMULSIONTRACKS
IDID_ZONESIDEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk>
TB_MIPBASETRACKS
IDID_ZONEPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMAID_DOWNTRACKID_UPTRACK
<pk><fk1>
<fk3><fk2>
TB_PREDICTIONS
IDRunEventXYZ
<ak><pk><pk>
TB_PREDICTEDTRACKS
IDID_PREDICTIONID_TRACKID_PHYSICSPOSXPOSYPOSZSLOPEXSLOPEYCHI2
<ak><pk,fk1><pk><fk2>
TB_PHYSICSIDS
IDDESCRIPTION
<ak><pk>
TB_PROCSTARTUPSELECTIONS
ID_PROCESSID_PREDICTEDTRACKID_SELECTION
<pk,fk2><pk,fk1><pk,fk3>
TB_VTXLOC_SELECTIONS
IDID_PLATEID_PROCESSOPERATIONID_SELECTORID_SELECTIONTEXT
<ak><pk,fk2><pk,fk1><fk4><fk3>
TB_VTXLOC_SELTRACKS
IDID_VTXLOC_SELID_BASETRACKID_MOTHERTRACKID_REASON
<ak><pk,fk1><pk,fk2>
<fk3>
TB_VTXLOC_SELECTION_REASONS
IDDESCRIPTION
<ak><pk>
TB_ALIGNEDBASETRACKS
IDID_MIPBASETRACKID_RECONSTRUCTIONID_ALIGNEDZONEID_VOLUMETRACKPOSXPOSYGRAINSAREASUMSLOPEXSLOPEYSIGMA
<pk><fk1><fk4><fk3><fk2>
TB_RECONSTRUCTIONS
IDID_RECONSTRUCTORISCOMPLETEID_PROGRAMSETTINGSID_PROCESSOPERATION
<pk><fk2>
<fk3><fk1>
TB_PROGRAMSETTINGS
IDMODULENAMEMODULEIDENTITYSETTINGSID_AUTHOR
<ak><pk><pk>
<fk>
TB_VOLUMETRACKS
IDID_RECONSTRUCTIONSIGMADOWNZUPZDOWNPOSXDOWNPOSYDOWNPOSZDOWNSLOPEXDOWNSLOPEYUPPOSXUPPOSYUPPOSZUPSLOPEXUPSLOPEYMOMENTUMENERGYID_PARTICLERECONSTRUCTIONNOTESID_DOWNVERTEXID_UPVERTEX
<pk><fk1>
<fk4>
<fk2><fk3>
TB_VERTICES
IDID_RECONSTRUCTIONPOSXPOSYPOSZAVERAGEDISTANCEID_VERTEXTYPE
<pk><fk1>
<fk2>
TB_VERTEXTYPES
IDDESCRIPTION
<ak><pk>
Bricks and Plates
Predictions and Final Results with Physics Info
System Management
Vertex Location
Scanning Data
Scanning Predictions
FK_SELECTIONS_USERS
OPERA European Emulsion Scanning DB
TB_SELECTIONS
IDID_OPERATIONID_SELECTORSELECTIONTEXTSELECTIONDATEDESCRIPTION
<ak><pk,fk1><pk,fk2>
<pk><pk>
TB_USERS
IDUSERNAMEPWDNAMESURNAMEINSTITUTIONID_SITEEMAILADDRESSPHONE
<ak1><ak2>
<pk><pk>
<fk>
TB_PROCESSES
IDID_BRICKSTARTTIMEFINISHTIME
<pk><fk>
TB_PRIVILEGES
ID_USERID_SITEREQUESTSCANREQUESTWEBANALYSISREQUESTDATAPROCESSINGREQUESTDATADOWNLOADREQUESTPROCESSSTARTUP
<pk,fk1><pk,fk2>
TB_PROC_OPERATIONS
IDID_PROCESSID_OPERATIONID_MACHINESTARTTIMEFINISHTIME
<ak><pk,fk3><pk,fk1><fk2>
TB_ALIGNEDZONES
IDID_ZONEID_RECONSTRUCTIONPLATEMAPXXMAPXYMAPYXMAPYYMAPDXMAPDYMAPDZSLOPEDSXSLOPEDSYSLOPEMSXSLOPEMSYREFXREFYREFZDOWNZUPZ
<ak><pk,fk1><pk,fk2>
Updated DB SchemaUpdated DB Schema
Overall structure unaltered, only fine tuning of data types and table columnsOverall structure unaltered, only fine tuning of data types and table columns
To assess performance, tuning must be done in real-life conditionsTo assess performance, tuning must be done in real-life conditions
Not only benchmarks, but real use of DB for test exposure data Not only benchmarks, but real use of DB for test exposure data
Updated DB SchemaUpdated DB Schema
DB size estimateDB size estimate
Assumptions for minimum size:Assumptions for minimum size:
Few data from CS (< 100 tracks per sheet), so they are negligible Few data from CS (< 100 tracks per sheet), so they are negligible
Intercalibration through 4 track maps per sheet, 100 tracks / map Intercalibration through 4 track maps per sheet, 100 tracks / map
All sheets in a brick are intercalibrated (conservative) All sheets in a brick are intercalibrated (conservative)
1 candidate is followed on 28 sheets on average (conservative) 1 candidate is followed on 28 sheets on average (conservative)
1 “interesting points” per event (vtx, decays, e.m. showers, ...)1 “interesting points” per event (vtx, decays, e.m. showers, ...)
(depends on Physics as well as instrumental needs)(depends on Physics as well as instrumental needs)
25000 events (OPERA half data set + tests, technical runs...)25000 events (OPERA half data set + tests, technical runs...)
3 pass scanning (Vtx location, selection, precision measurement)3 pass scanning (Vtx location, selection, precision measurement)
100 cosmics / background / fake microtracks in each zone100 cosmics / background / fake microtracks in each zone
Updated DB SchemaUpdated DB Schema
DB size estimateDB size estimate
Assumptions for maximum size:Assumptions for maximum size:
Few data from CS (< 100 tracks per sheet), so they are negligible Few data from CS (< 100 tracks per sheet), so they are negligible
Intercalibration through 4 track maps per sheet, 100 tracks / map Intercalibration through 4 track maps per sheet, 100 tracks / map
All sheets in a brick are intercalibrated (conservative) All sheets in a brick are intercalibrated (conservative)
10 candidates are followed on 28 sheets on average (conservative) 10 candidates are followed on 28 sheets on average (conservative)
3 “interesting points” per event (vtx, decays, e.m. showers, ...)3 “interesting points” per event (vtx, decays, e.m. showers, ...)
(depends on Physics as well as instrumental needs)(depends on Physics as well as instrumental needs)
50000 events (OPERA full data set + tests, technical runs...)50000 events (OPERA full data set + tests, technical runs...)
3 pass scanning (Vtx location, selection, precision measurement)3 pass scanning (Vtx location, selection, precision measurement)
100 cosmics / background / fake microtracks in each zone100 cosmics / background / fake microtracks in each zone
Updated DB SchemaUpdated DB Schema
DB size estimateDB size estimate
Results:Results:
(depends on Physics as well as instrumental needs)(depends on Physics as well as instrumental needs)
Assuming 100 bytes / track + aligned reconstructions, we haveAssuming 100 bytes / track + aligned reconstructions, we have
0.110.11÷2.3 TB÷2.3 TB
Including safety factors for possible underestimations, the Including safety factors for possible underestimations, the DB size should stay withinDB size should stay within
DBSize < 2.5DBSize < 2.5 TB TB
To estimate the network bandwidth needed, we assume To estimate the network bandwidth needed, we assume that the full dataset is read 10 times during 5 years’ data that the full dataset is read 10 times during 5 years’ data takingtaking
0.0550.055÷1÷1.2 Mbit/s ON AVERAGE.2 Mbit/s ON AVERAGE
Distributed DB Implementation Distributed DB Implementation
Core DB
Workstation DB Group
Workstation DB Group
Scanning DB
Scanning DB
Distributed DB Implementation Distributed DB Implementation
Core DB Group:Core DB Group:
Centralized locationCentralized location
Full OPERA emulsion data setFull OPERA emulsion data set
Multimaster replication (each machine has one copy)Multimaster replication (each machine has one copy)
Scanning DB:Scanning DB:
One machine per each scanning site (+ optional backup)One machine per each scanning site (+ optional backup)
Copy of the subset of locally produced scanning dataCopy of the subset of locally produced scanning data
Materialized views (each machine stores only locally produced data)Materialized views (each machine stores only locally produced data)
The full dataset is still accessible in a transparent wayThe full dataset is still accessible in a transparent way
Minimum network trafficMinimum network traffic
Distributed DB Implementation Distributed DB Implementation
Core DB Group Simulation:Core DB Group Simulation:
Two Dell servers with high speed (1 Gbit) Ethernet connectionTwo Dell servers with high speed (1 Gbit) Ethernet connection
Scanning DB Simulation:Scanning DB Simulation:
One Dell server machine with a materialized view of Salerno dataOne Dell server machine with a materialized view of Salerno dataNormal 100 Mbit/s LAN connectionNormal 100 Mbit/s LAN connection
Pilot DB farmPilot DB farm
DB Client Technologies DB Client Technologies
Several client connection technologies are being exploredSeveral client connection technologies are being explored
Goal: highest possible data availability for Oracle dataGoal: highest possible data availability for Oracle data
Windows:Windows:
ODBC, OLE DB, Oracle ODP.NET, ADO, ADO.NETODBC, OLE DB, Oracle ODP.NET, ADO, ADO.NET
Linux:Linux:
Oracle OCI C++ libraries, Perl, Tcl/Tk, Python, GNOME-DB, J2EEOracle OCI C++ libraries, Perl, Tcl/Tk, Python, GNOME-DB, J2EE
Mono (.NET) Oracle providerMono (.NET) Oracle provider
DB Client Technologies DB Client Technologies
Oracle and Windows:Oracle and Windows:
Working with DB is common practice in the Windows community.Working with DB is common practice in the Windows community.
Oracle provides both the DB server and several client tools / libraries.Oracle provides both the DB server and several client tools / libraries.
Oracle 9i Database Server consists of 3 installation disks (2.4 Oracle 9i Database Server consists of 3 installation disks (2.4 GB).GB).We have made several performance tests to choose the best We have made several performance tests to choose the best access method. Since our code will be under .NET, the access method. Since our code will be under .NET, the ADO.NET layer was common to all tests.ADO.NET layer was common to all tests.
ODBC: general DB library.ODBC: general DB library.
OLE DB: 50% faster than ODBC.OLE DB: 50% faster than ODBC.
Oracle Data Provider (ODP.NET): slightly slower (Oracle Data Provider (ODP.NET): slightly slower (~~10%?) than OLE DB.10%?) than OLE DB.
OCI C++: low level API, much harder than other methods. OCI C++: low level API, much harder than other methods.
OK!OK!
DB Client Technologies DB Client Technologies
Oracle and Linux:Oracle and Linux:
Oracle is fully committed to supporting the Linux operating system. Oracle is fully committed to supporting the Linux operating system. Indeed, Oracle was the first commercial database available on Linux.Indeed, Oracle was the first commercial database available on Linux.
All key Oracle products including Oracle 9i Database, Application Server, All key Oracle products including Oracle 9i Database, Application Server, Collaboration Suite, Developer Suite and E-Business Suite.Collaboration Suite, Developer Suite and E-Business Suite.
Oracle 9i Database consists of 3 cpio archive files ( Oracle 9i Database consists of 3 cpio archive files ( ~ 1.4 Gb). ~ 1.4 Gb). We installed it on a Red Hat 7.3 Linux DistributionWe installed it on a Red Hat 7.3 Linux Distribution..
DB Client Technologies DB Client Technologies
Oracle and Linux: OCI C++Oracle and Linux: OCI C++
Oracle Call Interface (OCI) is the Oracle software allowing access to theOracle Call Interface (OCI) is the Oracle software allowing access to thedatabase from an external application.database from an external application.
In principle, you can use this C-based API to build a Database application In principle, you can use this C-based API to build a Database application from scratch.from scratch.
Oracle and Linux: PerlOracle and Linux: Perl
Perl is probably the most famous open source language.Perl is probably the most famous open source language.
It is an interpreted scripting language easy to learn and extremely quick.It is an interpreted scripting language easy to learn and extremely quick.
It is widely used in Internet and Database applications.It is widely used in Internet and Database applications.
DB Client Technologies DB Client Technologies
Oracle and Linux: PerlOracle and Linux: Perl
Oracle DB
Oracle OCIPerl 5 Script Perl DBI DBD:: Oracle
Perl Database applications are based on the Perl Database applications are based on the DBI DBI module, an module, an object oriented architecture module. This module requires a object oriented architecture module. This module requires a specific database dependent driver (specific database dependent driver (DBD::OracleDBD::Oracle) to connect ) to connect to the databaseto the database
DB Client Technologies DB Client Technologies
Oracle and Linux: PerlOracle and Linux: Perl
A sample Perl script to access data from OracleA sample Perl script to access data from Oracle
use strict ;use DBI ;# Connection to the DBmy $dbh = DBI -> connect(dbi:Oracle:operadb,operausr,operapwd);# Database SQL Querymy $sql = qq { SELECT Grains,SlopeX,SlopeY FROM MIPBaseTracks};# Execute SQL Querymy $sth = $dbh ->prepare ($sql);$sth -> execute ();# Loop on the tableWhile (my ($run,$event,$ntracks)= $sth->fetchrow_array){
# Here you can do whatever you want with the data}
DB Client Technologies DB Client Technologies
Oracle and Linux: Tcl/TkOracle and Linux: Tcl/Tk
Tcl is an excellent scripting language created in 1987 by John Tcl is an excellent scripting language created in 1987 by John Ousterhout. Ousterhout. In 1988 he started to develop a graphic tool called Tk. In 1988 he started to develop a graphic tool called Tk. From that moment on, Tcl/Tk is one of the most favourite From that moment on, Tcl/Tk is one of the most favourite language in the Open Source community.language in the Open Source community.
Oratcl is the module allowing the connection to an Oracle database.Oratcl is the module allowing the connection to an Oracle database.
The connection to the database can be tested interactively The connection to the database can be tested interactively using the using the TTcl shellcl shell..
DB Client Technologies DB Client Technologies
Oracle and Linux: Tcl/TkOracle and Linux: Tcl/Tk
A sample Tcl/Tk script to access data from OracleA sample Tcl/Tk script to access data from Oracle
tclsh%package require Oratcl%set handle [oralogon operausr/operapwd@operadb]%set cursor [oraopen $handle]%orasql $cursor {select Grains,SlopeX,SlopeY from MIPBaseTracks}%orafetch $cursor25 0.400 0.00332 0.231 0.005……………%oraclose $cursor%oralogoff $handle%exit
DB Client Technologies DB Client Technologies
Oracle and Linux: Perl/TkOracle and Linux: Perl/Tk
TTk is wrapped inside Perl. So you can combine Perl k is wrapped inside Perl. So you can combine Perl quickness with quickness with TTk graphick graphicss capabilitiescapabilities..
Oraexplain is the Perl/Oraexplain is the Perl/TTk module used to connect a program to k module used to connect a program to an Oracle databasean Oracle database..
Oraexplain scripts are more complex because they involve Oraexplain scripts are more complex because they involve graphical elements like windows, buttographical elements like windows, buttonns and frames. You can s and frames. You can use them to develop login windows or more complex graphicuse them to develop login windows or more complex graphicss applicationsapplications..
DB Client Technologies DB Client Technologies
Oracle and Linux: PythonOracle and Linux: Python
Python is a GUI open source and object-Python is a GUI open source and object-oriented scripting language created by oriented scripting language created by Guido van Rossum. It is used for any kind of Guido van Rossum. It is used for any kind of application: GUI, application: GUI, XMLXML, e-mail, …., e-mail, ….
DCOracle is the module used to connect a Python script to an DCOracle is the module used to connect a Python script to an Oracle databaseOracle database..
With Python you can define classes and re-use them in With Python you can define classes and re-use them in oother ther scriptsscripts..
DB Client Technologies DB Client Technologies
Oracle and Linux: GNOME-DBOracle and Linux: GNOME-DB
The GNOME-DB project aims to provide a free The GNOME-DB project aims to provide a free unified data access architecture to the GNOME unified data access architecture to the GNOME project. GNOME-DB is useful for any application project. GNOME-DB is useful for any application that accesses persistent data (not only databases), that accesses persistent data (not only databases), since it now contains a pretty good data since it now contains a pretty good data management API.management API.
We are currently testing the GNOME-DB libraries.We are currently testing the GNOME-DB libraries.
Oracle and Windows/Linux: MonoOracle and Windows/Linux: Mono
ADO.NET Data Provider for Oracle databases ADO.NET Data Provider for Oracle databases works on Windows and Linux with Oracle 8iworks on Windows and Linux with Oracle 8i and and higher versions.higher versions.
We are currently testing the Mono ADO.NET libraries.We are currently testing the Mono ADO.NET libraries.
Tests in progress: Web applications with Apache, JSP, PHP, Java, JDBCTests in progress: Web applications with Apache, JSP, PHP, Java, JDBC
DB Client Libraries DB Client Libraries
We are developing a full set of .NET / Mono classes specific to Opera DBWe are developing a full set of .NET / Mono classes specific to Opera DBOperaDb
OperaDb::Connection
OperaDb::Transaction
OperaDb::Scanning
OperaDb::Scanning::Batch
OperaDb::Scanning::LinkedZone
OperaDb::Scanning::MIPBaseTrack
OperaDb::Scanning::MIPIndexedEmulsionTrack
OperaDb::TotalScan
OperaDb::TotalScan::Layer
OperaDb::TotalScan::Segment
OperaDb::TotalScan::Track
OperaDb::TotalScan::Volume
OperaDb::ComputingInfrastructure
OperaDb::ComputingInfrastructure::Machine
OperaDb::ComputingInfrastructure::ProgramSettings
OperaDb::ComputingInfrastructure::User
OperaDb::ComputingInfrastructure::UserPermissions
OperaDb::ComputingInfrastructure::Site
Namespaces in blackNamespaces in black
Classes in dark redClasses in dark red
Who does not want to use SQL can Who does not want to use SQL can program the DB in C++, C#, VB, program the DB in C++, C#, VB, FORTRAN, and so on, using these FORTRAN, and so on, using these
classesclasses
More classes to come...More classes to come...
DB Client Libraries DB Client Libraries
Code samplesCode samples
Sample #1: How to store a scanning zone into Opera DBSample #1: How to store a scanning zone into Opera DBC++: idZone = LinkedZone::Save(pMyZone, idBatch, rawDataPath, startTime, endTime, dbConn, dbTrans);
C#: idZone = LinkedZone.Save(MyZone, idBatch, rawDataPath, startTime, endTime, dbConn, dbTrans);
Sample #2: How to retrieve a volume reconstruction from Opera DBSample #2: How to retrieve a volume reconstruction from Opera DBC++: Volume *pVol = new Volume(dbConn, dbTrans, idVolume, true);
C#: Volume Vol = new Volume(dbConn, dbTrans, idVolume, true);
Sample #3: How to convert a scanning zone from Opera DB to Sample #3: How to convert a scanning zone from Opera DB to ROOT file ROOT file C++:
SySal:: Root::Opera::LinkedZone::Save(new LinkedZone(dbConn, dbTrans, idZone), filePath);
C#: SySal.Root.Opera.LinkedZone.Save(new LinkedZone(dbConn, dbTrans, idZone), filePath);
Conclusions Conclusions
The DB architecture that has been proposed some months ago is workingThe DB architecture that has been proposed some months ago is working
Oracle 9iDS looks a very good choice Oracle 9iDS looks a very good choice (easy to implement, maintain, and develop; widely supported)(easy to implement, maintain, and develop; widely supported)
Even people that are not familiar with SQL can easily work with Even people that are not familiar with SQL can easily work with the proposed structure of emulsion DB using interface libraries the proposed structure of emulsion DB using interface libraries
All interesting OS are supported by Oracle and by our interface All interesting OS are supported by Oracle and by our interface librarieslibraries
Conversion to Root data is trivial (1 line of code)Conversion to Root data is trivial (1 line of code)
...everything fine up to now!...everything fine up to now!
Interface libraries for OperaDB are almost (95%) Interface libraries for OperaDB are almost (95%) completecomplete