28
COMA Tables (COnditions MetadatA) Elizabeth Gallas June 2014

COMA Tables (COnditions MetadatA)

  • Upload
    inga

  • View
    41

  • Download
    0

Embed Size (px)

DESCRIPTION

COMA Tables (COnditions MetadatA). Elizabeth Gallas June 2014. Oracle Database: TAG Services Schemas TAG DB, TAG Catalog, COMA DB. RUNS. ATLR DB. COMA. TAG DB Event-wise metadata tables Stores Event-wise attributes: electron (Et, eta, phi …) muon (Et, eta, phi …) … - PowerPoint PPT Presentation

Citation preview

Page 1: COMA Tables (COnditions MetadatA)

COMA Tables(COnditions MetadatA)

Elizabeth Gallas

June 2014

Page 2: COMA Tables (COnditions MetadatA)

2

TAG DB Event-wise metadata tables Stores Event-wise attributes:

electron (Et, eta, phi …) muon (Et, eta, phi …) …and references to RAW, ESD, AOD files

Official data processing chain:RAW ESD AOD TAG files

‘COMA’ (COnditions MetadatA) tables Conditions of data taking

Beam conditions Trigger and DAQ conditions Magnetic field …

Various sources:Conditions DB, Log files, xml files, email…

Data Catalogue tables Stores information on file and dataset

processing and location Project name AMI tag (what processing occurred)…

Sources: AMI (ATLAS Metadata Catalogue, Tier0 …

…EventLBRun …EventLBRun

…EventLBRun …EventLBRun

…EventLBRun …EventLBRun

Oracle Database: TAG Services SchemasTAG DB, TAG Catalog, COMA DB

RUNS

COMA

TAG Catalogue

TAG DB: Event-wise metadata

ATLR DB

ATLARC DBCOMA (copy)

TAG Catalogue (copy)

Page 3: COMA Tables (COnditions MetadatA)

3

Notes on the Run Metadata Schema Warning: The schema tables, relations, content shown in the diagram

Reflect how I think the schema will evolve. “*” on table and column names Parts of the schema in flux

May not be exactly as implemented on the database instances See database instance for the current implementation

Boxes in the diagram show the Oracle tables Table names in the implementation have the common ‘COMA_’ prefix

COMA COnditions database MetadatA Note: most but not all data is loaded from the Conditions Database or the Trigger Database

Tables with a single integer primary key have key column name *_INDEX Many tables adopt the RUN_INDEX column from RUNS

In this case, the child table includes additional columns to form its Primary (or Unique) Key(s) Except for DETECTOR_MASK which follows the Conditions DB payload column name

Relationships are indicated with arrows connecting the tables Arrow direction indicate parent to child relationship (foreign key constraint) Example: Runs must use a TRIGKEY_INDEX from MASTER_KEYS (defining it’s trigger configuration)

and must have DATA_SOURCE, RUN_TYPE and DETECTOR_MASK from those parent tables respectively.

Stone gray tables and/or dashed relationships are empty and some stage of development Act as placeholders

Tables shown might have columns or new indexes added. The elongated diamond shaped tables are many-to-many relationships. Refer to the database instance or the ddl for column types

SVN: offline/Database/RunLumi/ConditionsToTags/ddl/ The latest version is not guaranteed to be checked into SVN

Column Names follow COOL payload or Trigger DB column names found in the associated folders or tables (but with underscores and avoiding Oracle reserved words) https://twiki.cern.ch/twiki/bin/view/Atlas/CoolOnlineData

Page 4: COMA Tables (COnditions MetadatA)

4

Abbreviations which might be on the diagram P == Primary Key (all are mandatory) F == Foreign Key (can be optional) FI == Indexed Foreign Key (R) == Redundant (but useful) M == Mandatory O == Optional

Note: columns are by default optional U == Comprise Unique Keys for this table

Unique keys are constraints forcing uniqueness like PKs. ? == questionable value

kept only for backward compatibility (MC) == populated only for Simulation

MC* == populated only for MC09+ (not FDR, MC08) (t) == populated by Database Trigger

NULL values vs default values – prefer NULL when unknown

Relationships:

One to many

Many to many

One to oneStone gray tables:

Not filled yet.

Page 5: COMA Tables (COnditions MetadatA)

5

COMA Schema

PD_OBSRun_LBN_Times

Events

RUN_TYPESSOURCES

MASKS

Fills

RUNS Run_LBN_Lums

Transitions

Run_Load_Status

ShortNames

Explains

Run_Fills

Stream_Events

Run_Magnets

Run_Stream_LBN_Events

Period_Defs

Period_P1_To_Runs

Run_Lums

Det_Status

Detectors

Det_Codes

Groups

Virtual_Flags

Vflag_Dets

HLT_PSLVL1_PS

MK_Triggers

PS_Sets

Master_Keys

Prescale_Keys

LVL1_PSK HLT_PSK

Run_Agg_PS

Trig_Agg_PS

MKT_Physics,Others

Stream_Keys

Mkt_To_Sks

Period_Phi_To_Plos

COOL_Connections

P2R_OBS

PH2L_OBS

PGRL_OBS

Load_Logs

Run_Stable_Beams

Run_ATLAS_Readys

Owner_Instances

Nodes

Schemas OnOffsInstances

PColumns

References

GTag_Infos

GTags

GT_TO_FTS

FTags

AMI_GTags

GT_TO_OIS

Run_Stream_Overlaps

Page 6: COMA Tables (COnditions MetadatA)

6

RUNSP - RUN_INDEXF - DATA_SOURCEF - RUN_TYPEF - DETECTOR_MASKF - COOL_INDEXF - PRESCALE_INDEXF - TRIGKEY_INDEXU - RUN_NUMBERU - FILENAME_TAGDAQ_CONFIGURATIONRECORDING_ENABLEDCLEAN_STOPSTART_TIMEEND_TIMEDURATIONSTART_LBNEND_LBNCONFIG_SOURCEMC_TRIG_KEY (MC09)U – AMITAG_NAME_MC (MC*)

See next slides for:

Data Periods

Triggers, Prescales, andAggregate Prescales

Trigger Chain Tables

Runs, Streams, Triggers

Fills, Beams, Magnets, Ready

CB Tables: Conditions DB Management Metadata for support of Conditions DB Browsing

DQ and Virtual Flags

Run_LBN_TimesPF – RUN_INDEXP – LBNSTART_TIMEEND_TIMEDURATION

Run_EventsPF - RUN_INDEXP – PARTITION_NAMEL1_EVENTSL2_EVENTSL3_EVENTSRECORDED_EVENTS

RUN_TYPESP - RUN_TYPERUN_TYPE_DESC

DATA_SOURCESP - DATA_SOURCEDATA_SOURCE_DESC

DETECTOR_MASKSP - DETECTOR_MASKMASK_DECODED

Conditions and Configurations Metadata for ATLAS (COMA)

Time_TransitionsP - TRANS_TIMEPU - TRANS_TYPEU - TRANS_NUMBER

Run_Load_StatusP – RUN_INDEXP - CONDITION_TYPEP – COLUMN_AFFECTEDSEVERITYCOLUMN_NEWCOLUMN_OLD

SHORTNAMESP – SHORTNAMEU –ENTITIY_NAMEENTITY_TYPE

TABLE_VALUE_EXPLAINSP – TABLE_SNP – COL_NAMEP – COL_VALUE COL_DESC

Cool_ConnectionsP - COOL_INDEXCOOL_IMPLCOOL_SERVERCOOL_SCHEMACOOL_INSTANCEU - COOL_TAGCOOL_FOLDERCOOL_DATECOOL_DESC

FILENAME_TAGSP – FILENAME_TAGFNT_DESC

Load_LogsP – LL_INDEXLL_PROGRAMCOMA_INS_DATECOUNT_INSERTCOUNT_UPDATECOUNT_DELETECOMA_UPD_DATE

Page 7: COMA Tables (COnditions MetadatA)

7

Data Periods (Tables and Views)

PERIOD_DEFSP – P_INDEX (t)

F – PDOM_NAMEU – P_LEVEL ( 1, 2, 3 )U – P_PROJECT_PERIOD

P_PROJECTP_PERIODP_DESCP_STATUS

CREATE_USERCREATE_DATE (t)MODIFY_USERMODIFY_DATE (t)

PERIOD_PHI_TO_PLOSP – PH2L_INDEX (t)

F – PHI_INDEXF – PLO_INDEX

CREATE_USERCREATE_DATE (t)

PERIOD_DOMAINSP – PDOM_NAMEPDOM_DESC

PERIOD_P1_TO_RUNSP – P2R_INDEX (t)

F – P_INDEXF – RUN_INDEX

CREATE_USERCREATE_DATE (t)

RUNSP -- RUN_INDEX

PERIOD_GRLSP – PGRL_INDEX (t)

UF – P_INDEXU – PGRL_NAME

CREATE_USERCREATE_DATE (t)

V_PHI_PLOSPDOM_NAME

PHI_LEVELPHI_INDEXPHI_PROJECTPHI_PERIOD

PLO_INDEXPLO_PROJECTPLO_PERIOD

V_P1_RUNSPDOM_NAME

P1_INDEXP1_PROJECTP1_PERIOD

RUN_INDEX

V_P2_RUNSPDOM_NAME

P2_INDEXP2_PROJECTP2_PERIOD

P1_INDEXP1_PROJECTP1_PERIOD

RUN_INDEX

V_P3_RUNSPDOM_NAME

P3_INDEXP3_PROJECTP3_PERIOD

P2_INDEXP2_PROJECTP2_PERIOD

P1_INDEXP1_PROJECTP1_PERIOD

RUN_INDEX

V_PERIOD_RUNSPDOM_NAMEP_LEVELP_INDEXP_PROJECTP_PERIODRUN_INDEX

Page 8: COMA Tables (COnditions MetadatA)

8

Data Periods (tables for storing updated/deleted rows)

PD_OBSPDO_INDEX (t)

P_INDEX (t)PDOM_NAME (t)P_LEVEL (t)P_PROJECT_PERIOD (t)P_PROJECT (t)P_PERIOD (t)P_DESC (t)P_STATUS (t)CREATE_USER (t)CREATE_DATE (t)MODIFY_USER (t)MODIFY_DATE (t)DELETE_USER DELETE_DATE (t)

PH2L_OBSPH2L_INDEX (t)PHI_INDEX (t)PLO_INDEX (t)CREATE_USER (t)CREATE_DATE (t)DELETE_USERDELETE_DATE (t)

P2R_OBSP2R_INDEX (t)P_INDEX (t)RUN_INDEX (t)CREATE_USER (t)CREATE_DATE (t)DELETE_USERDELETE_DATE (t)

PGRL_OBSPGRL_INDEX (t)P_INDEX (t)PGRL_NAME (t)CREATE_USER (t)CREATE_DATE (t)DELETE_USERDELETE_DATE (t)

PERIOD_DEFSP_INDEX (t)PDOM_NAMEP_LEVELP_PROJECT_PERIODP_PROJECTP_PERIODP_DESCP_STATUSCREATE_USERCREATE_DATE (t)MODIFY_USERMODIFY_DATE (t)

PERIOD_PHI_TO_PLOSPH2L_INDEX (t)PHI_INDEXPLO_INDEXCREATE_USERCREATE_DATE (t)

PERIOD_DOMAINSPDOM_NAMEPDOM_DESC

PERIOD_P1_TO_RUNSP2R_INDEX (t)P_INDEXRUN_INDEXCREATE_USERCREATE_DATE (t)

RUNSP -- RUN_INDEX

PERIOD_GRLSPGRL_INDEX (t)P_INDEXPGRL_NAMECREATE_USERCREATE_DATE (t)

On DELETE triggers (red) DELETE_USER must be updated by AMI Entry program

On UPDATE trigger (green) No need to populate columns DELETE_%

Page 9: COMA Tables (COnditions MetadatA)

9

Good Run List

RUNSP - RUN_INDEX…FILENAME_TAG…

Run_LBN_TimesPF – RUN_INDEXP – LBNSTART_TIMEEND_TIMEDURATION

Load_LogsP – LL_INDEXLL_PROGRAMCOMA_INS_DATE…

GRL_FILESP – GRLF_INDEXU – GRLF_FILEU – GRLF_PATHGRLF_DATEGRLF_SIZEGRLF_CHECKSUMGRLF_SOURCEGRLF_NAMEGRLF_VERSIONGRLF_STYLESHEETGRLF_PROJECTGRLF_PERIODINTEG_LUMIGRLF_INTEG_LUMIGRLF_ERROR

PERIOD_DEFSP – P_INDEXP_PROJECT_PERIODP_PROJECTP_PERIOD…

GRL_Criteria_OthersP – GRLCO_INDEXF – GRLF_INDEXGRLCO_NAMEGRLCO_VALUE

GRL_StreamsP – GRLS_INDEXFU – GRLF_INDEXU – STREAMRAW_EVENTSSFO_TOT_EVENTSSFO_SEL_EVENTSXML_TOT_EVENTSXML_SEL_EVENTS

GRL_RUNSP – GRLR_INDEXFU – GRLF_INDEXFU – RUN_INDEXINTEG_LUMI (R)GRLR_INTEG_LUMI

GRL_RUN_LBSP – GRLL_INDEXF – GRLR_INDEXF – LBN_BEGINF – LBN_ENDGRLL_INTEG_LUMI

RUN_LBN_LUMSP – RLL_INDEXUF – RUN_INDEXUF – LBNUF – LUM_COOL_INDEXLBAVINSTLUMILVAVEVTSPERBXVALIDRLL_INTLUMI

GRL_XMLSP – GRLX_INDEXGRLF_INDEXGRLF_FILEGRLF_PATHGRLF_DATEGRLF_CLOB

GRL_Run_StreamsP – GRLRS_INDEXFU – GRLS_INDEXFU – RUN_INDEXSFO_TOT_EVENTS (R)SFO_SEL_EVENTS

RUN_LUMSP – RL_INDEXUF – RUN_INDEXUF – LUM_COOL_INDEXINTEG_LUMI…

GRL_DQSP – GRLDQS_INDEXFU – GRLF_INDEXU – GRLDQ_STRING

Page 10: COMA Tables (COnditions MetadatA)

10

Runs, Triggers, Prescales and (Run,Trig)_Aggregate Prescales

HLT_PRESCALESPF - HLT_PRESCALE_INDEXP(F) – TRIGGER_LEVELP(F) – CHAIN_COUNTER

PRESCALEPASSTHROUGH

LVL1_PRESCALESPF - LVL1_PRESCALE_INDEXP(F) – CHAIN_COUNTER

PRESCALE

RunsP - RUN_INDEXDATA_SOURCERUN_TYPESTART_TIME

…F – PRESCALE_INDEXF - TRIGKEY_INDEX

MK_TriggersP - TRIGGER_INDEXUF TRIGKEY_INDEXU - TRIGGER_LEVELU - CHAIN_COUNTERCHAIN_NAMECHAIN_VERSIONLOWER_CHAIN_NAME? - PRESCALE_SIM? - PASSTHROUGH_SIMRERUN_PRESCALESTREAM_INFOTRIGGER_ELEMENTS

Prescale_SetsPF - PRESCALE_INDEXP – LBN_BEGINP – LBN_ENDF - HLT_PRESCALE_INDEXF - LVL1_PRESCALE_INDEXF – TRIGKEY_INDEX (R)

Master_KeysP – TRIGKEY_INDEXMK_DESCRIPTIONSMT_NAMESMT_VERSIONLVL1_DESCRIPTIONHLT_DESCRIPTION

Prescale_KeysP – PRESCALE_INDEX

LVL1_PS_KeysP – LVL1_PRESCALE_INDEXL1PS_MODIFIED_TIME

HLT_PS_KeysP - HLT_PRESCALE_INDEXHPR_MODIFIED_TIME

Run_Agg_PSPF – PRESCALE_INDEXPF – TRIGGER_INDEXF – TRIGKEY_INDEX (R)MIN_PSMAX_PSPS_FLAGPT_FLAGRR_FLAG

Trig_Agg_PSPF – LVL1_PRESCALE_INDEXPF – HLT_PRESCALE_INDEXPF – TRIGGER_INDEXF - TRIGKEY_INDEX (R)AGG_PSPT_FLAG

Page 11: COMA Tables (COnditions MetadatA)

11

Runs, Trigger Chain Tables, Run Aggregate Prescales

RunsP - RUN_INDEXDATA_SOURCEFILENAME_TAGRUN_NUMBER

…F – PRESCALE_INDEXF - TRIGKEY_INDEX

MK_TriggersP - TRIGGER_INDEX

UF TRIGKEY_INDEXU - TRIGGER_LEVELU - CHAIN_COUNTER

CHAIN_NAMECHAIN_VERSIONLOWER_CHAIN_NAMERERUN_PRESCALESTREAM_INFOTRIGGER_ELEMENTS

Prescale_SetsP – PRESCALE_INDEXP – LBN_BEGINLBN_ENDTRIGKEY_INDEX (R)LVL1_PRESCALE_INDEXHLT_PRESCALE_INDEX

Master_KeysP – TRIGKEY_INDEX…

Prescale_KeysP – PRESCALE_INDEX

Run_Agg_PSPF – PRESCALE_INDEXPF – TRIGGER_INDEXF – TRIGKEY_INDEX (R)MIN_PSMAX_PSPS_FLAGPT_FLAGRR_FLAG

MKT_PHYSICS:“Physics (L1-L2-EF) chains”replaces obsolete V_MK_Triggers

(get total prescale from Run_Agg_PS)

MKT_OTHERS“Commissioning chains”(Aggregate PS for these rows

also in Run_Agg_PS)

MKT_PhysicsP - TRIGGER_INDEXF TRIGKEY_INDEX

L3_BITL2_BITL1_BITL3_NAMEL2_NAMEL1_NAMESTREAM_INFO

MKT_OthersP - TRIGGER_INDEXF TRIGKEY_INDEX

L3_BITL2_BITL1_BITL3_NAMEL2_NAMEL1_NAMESTREAM_INFOULCN

Page 12: COMA Tables (COnditions MetadatA)

12

Stream_KeysP -- STREAM_INDEXSTREAM_NAMESTREAM_TYPEOBEYLBSTREAM_DESC

MKT_TO_SKSPF – TRIGGER_INDEXPF – STREAM_INDEXSTREAM_PRESCALE

Runs, Triggers, Streams, and Events

Master_KeysP – TRIGKEY_INDEXMK_DESCRIPTIONSMT_NAMESMT_VERSIONLVL1_DESCRIPTIONHLT_DESCRIPTION

Run_Stream_EventsP – RSE_INDEXUF – RUN_INDEXU – STREAM

STREAM_EVENTSSUM_FILESIZECOUNT_FILESIZE

F – STREAM_INDEXSTREAM_PRESCALE

RunsP - RUN_NUMBERDATA_SOURCERUN_TYPESTART_TIME…F – PRESCALE_INDEXF - TRIGKEY_INDEX

MK_TriggersP - TRIGGER_INDEX

UF TRIGKEY_INDEXU - TRIGGER_LEVELU - CHAIN_COUNTER

CHAIN_NAMECHAIN_VERSIONLOWER_CHAIN_NAMERERUN_PRESCALESTREAM_INFOTRIGGER_ELEMENTS

Run_Stream_LBN_EventsP – RSLE_INDEXUF – RSE_INDEXU – LBN

STREAM_EVENTS

Run_LBN_TimesPF – RUN_INDEXP – LBNSTART_TIMEEND_TIMEDURATION

Run_Stream_OverlapsP – RSO_INDEXUF – RSE_INDEXU – OVERLAP_STREAM

OVERLAP_EVENTS

Run_PSA_StreamsP – RPAS_INDEXRUN_INDEXSTREAM_INDEXLBN_BEGINLBN_END

Page 13: COMA Tables (COnditions MetadatA)

13

Run_Stable_BeamsP – RSB_INDEX

FU – RUN_INDEXFU – LBN_BEGINLBN_ENDRSB_STATEFI – FILL_INDEX

BEAM_ENERGY_GEV

NUM_BUNCH_COLL

MIN_BUNCH_DT

MACHINE_MODE

RUNSP - RUN_INDEX

F - DATA_SOURCEF - RUN_TYPEF - DETECTOR_MASKF - COOL_INDEXFI - PRESCALE_INDEXFI - TRIGKEY_INDEXU - RUN_NUMBERU - FILENAME_TAGDAQ_CONFIGURATIONRECORDING_ENABLEDCLEAN_STOPSTART_TIMEEND_TIMEDURATIONSTART_LBNEND_LBNCONFIG_SOURCEMC_TRIG_KEY (MC09)U – AMITAG_NAME_MC (MC*)

Run and Beam-related information

RUN_LBN_TimesPF – RUN_INDEXP – LBN

START_TIMEEND_TIMEDURATION

RUN_FILLSP – RF_INDEX (t)

FU – RUN_INDEXFU – FILL_INDEX

Run_Atlas_ReadysP – RAR_INDEX

F – RUN_INDEXF – LBN_BEGINF – LBN_ENDRAR_STATE

Cool_ConnectionsP - COOL_INDEXCOOL_IMPLCOOL_SERVERCOOL_SCHEMACOOL_INSTANCEU - COOL_TAGCOOL_FOLDERCOOL_DATECOOL_DESC

RUN_LBN_LUMSP – RLL_INDEX

UF – RUN_INDEXUF – LBNUF – LUM_COOL_INDEXLBAVINSTLUMILVAVEVTSPERBXVALIDRLL_INTLUMI

RUN_LUMSP – RL_INDEX

UF – RUN_INDEXUF – LUM_COOL_INDEXINTEG_LUMIMIN_INST_LUMIMAX_INST_LUMIMIN_AVG_MUMAX_AVG_MUREADY_FRACTION

FILLSP – FILL_INDEX

START_TIMEEND_TIME

LUM_DETS ?P – LUM_CHANNELLUM_CHANNEL_NAMELUM_CHANNEL_DESCLUM_SUBSYSTEM

Run_MagnetsP – RM_INDEX

FU – RUN_INDEXFU – LBN_BEGINLBN_ENDSOL_STATETOR_STATE

Page 14: COMA Tables (COnditions MetadatA)

14

Lum Views *

Run_Stable_BeamsP – RSB_INDEXFU – RUN_INDEXFU – LBN_BEGINLBN_ENDRSB_STATE

FI – FILL_INDEX

BEAM_ENERGY_GEV

NUM_BUNCH_COLL

MIN_BUNCH_DT

MACHINE_MODE

RUNSP - RUN_INDEXF - DATA_SOURCEF - RUN_TYPEF - DETECTOR_MASKF - COOL_INDEXFI - PRESCALE_INDEXFI - TRIGKEY_INDEXU - RUN_NUMBERU - FILENAME_TAGSTART_TIMEEND_TIMEDURATIONSTART_LBNEND_LBN…

Run_LBN_TimesPF – RUN_INDEXP – LBNSTART_TIMEEND_TIMEDURATION

Run_Atlas_ReadysP – RAR_INDEX

F – RUN_INDEXF – LBN_BEGINF – LBN_ENDRAR_STATE

Run_LumsP – RL_INDEXUF – RUN_INDEXUF – LUM_COOL_INDEXINTEG_LUMIMIN_INST_LUMIMAX_INST_LUMIMIN_AVG_MUMAX_AVG_MUREADY_FRACTION

V_Stable_Lums

RUN_INDEX

MIN_FILL_INDEXMAX_FILL_INDEXBEAM_ENERGY_GEV (MIN)NUM_BUNCH_COLL (MIN)MIN_BUNCH_DT (MIN)MACHINE_MODE (MAX)

MAX_RAR_STATE

LUM_COOL_INDEXINTEG_LUMIMIN_INST_LUMIMAX_INST_LUMIMIN_AVG_MUMAX_AVG_MUREADY_FRACTION

V_Run_Fills

RUN_INDEX

MIN_FILL_INDEXMAX_FILL_INDEXMAX_RSB_STATEMAX_RAR_STATE

V_Period_Lums *P_PROJECTP_PERIODP_LEVELP_DESCP_STATUS

MIN_RUN_INDEXMAX_RUN_INDEXNUM_RUNSMIN_START_TIMEMAX_END_TIMEMIN_FILL_INDEXMAX_FILL_INDEX

LUM_COOL_INDEXINTEG_LUMIMIN_INST_LUMIMAX_INST_LUMIMIN_AVG_MUMAX_AVG_MU

READY_LUMIREADY_FRACTION

BEAM_ENERGY_GEVNUM_BUNCH_COLLMIN_BUNCH_DT (MIN)MACHINE_MODE (MAX)

Period_DefsP_INDEX (t)PDOM_NAMEP_LEVELP_PROJECT_PERIODP_PROJECTP_PERIODP_DESCP_STATUS…

Period_P1_To_RunsP – P2R_INDEX (t)F – P_INDEXF – RUN_INDEX…

Page 15: COMA Tables (COnditions MetadatA)

15

CRT_RunsP – CRTR_INDEXU1 – COLLECTION_NAMEU1,U2 – RUN_INDEXU2 – STREAMU2 – AMI_TAGFRX_TAGFILENAME_TAGSCHEMA_NAMESTREAM_TYPESTREAM_NAMENB_EVENTSTAGDB_EVENTSTAGFILE_EVENTSAODFILE_EVENTSESDFILE_EVENTSRAWFILE_EVENTSCRTR_FLAGCRTR_CHECKSUMCRTR_ACTIVECRTR_SECSCRTR_ERRORCRTR_INFO

CRT_CountsP – CRTC_INDEXFU – CRTR_INDEXFU – TRIGGER_INDEXRTC_COUNTRTC_FLAG

Event Count, PSCorr Lumi by Run/Trig

Prescale_SetsPF - PRESCALE_INDEXP – LBN_BEGINP – LBN_ENDF - HLT_PRESCALE_INDEXF - LVL1_PRESCALE_INDEXF – TRIGKEY_INDEX (R)

Trig_Agg_PSPF – LVL1_PRESCALE_INDEXPF – HLT_PRESCALE_INDEXPF – TRIGGER_INDEXF - TRIGKEY_INDEX (R)AGG_PSPT_FLAG

Run_LBN_LumsP – RLL_INDEXUF – RUN_INDEXUF – LBNUF – LUM_COOL_INDEXLBAVINSTLUMILVAVEVTSPERBXVALIDRLL_INTLUMI

MKT_ALLsP - TRIGGER_INDEXF TRIGKEY_INDEXMKTA_TYPETOP_NAMEL3_BITL3_NAME …

Run_Agg_PSPF – PRESCALE_INDEXPF – TRIGGER_INDEXF – TRIGKEY_INDEX (R)MIN_PSMAX_PSPS_FLAG …

RUNSP - RUN_INDEX…F – PRESCALE_INDEXF - TRIGKEY_INDEX

Run_LBN_TimesPF – RUN_INDEXP – LBNSTART_TIMEEND_TIMEDURATION

Run_Trig_LumsP – RTL_INDEXFU – RUN_INDEXFU – TRIGGER_INDEXPSC_INTLUMI_SBPSC_INTLUMI_SBRMIN_AGG_PS_SBRMAX_AGG_PS_SBR

Run_Stable_BeamsP – RSB_INDEXFU – RUN_INDEXFU – LBN_BEGINLBN_ENDRSB_STATE…

Run_Atlas_ReadysP – RAR_INDEXF – RUN_INDEXF – LBN_BEGINF – LBN_ENDRAR_STATE

Run_Stream_EventsP – RSE_INDEXUF – RUN_INDEXU – STREAMSTREAM_EVENTS…

COOL_TCountsP – CTC_INDEXFU – RUN_INDEXFU – TRIGGER_INDEXEVENT_COUNT

Page 16: COMA Tables (COnditions MetadatA)

16

RUN_COOL_CountsPF – RUN_INDEXPF – CBFT_INDEXP – CHANNEL

ROW_COUNT

Count entries in COOL by Run, FolderTag, Channel for selected COOL Folderssuch as Luminosity and Data QualityRUNS

P - RUN_INDEXF - DATA_SOURCEF - RUN_TYPEF - DETECTOR_MASKF - COOL_INDEXFI - PRESCALE_INDEXFI - TRIGKEY_INDEXU - RUN_NUMBERU - FILENAME_TAGSTART_TIMEEND_TIMEDURATIONSTART_LBNEND_LBN…

CB_FTAGSP – CBFT_INDEXFU – CBF_INDEXU – TAG_NAMETAG_LOCK_STATUSTAG_DESCRIPTIONSYS_INSTIMECBFT_INSTIMECBFT_NODE_IDCBFT_ROWCOUNTCBFT_LAST_OBJTIMECBFT_SINCE_TIMECBFT_UNTIL_TIMECOMA_INS_DATE (t)COMA_UPD_DATE (t)

CB_NODESP – CBF_INDEXFU – CBOI_INDEXU – NODE_FULLPATHI – NODE_NAMENODE_IDNODE_PARENTIDNODE_ISLEAFNODE_INSTIMELASTMOD_DATECBF_LASTMOD_DATECOMA_INS_DATE (t)

NODE_DESCRIPTIONFOLDER_VERSIONINGI – CBF_NODE_PATHCBF_IOV_BASECBF_ATT_TYPECBF_CHAN_COUNTCBF_PAYLOAD_COUNTCOMA_UPD_DATE (t)

Project_FTag_RanksPF – FILENAME_TAGPF – CBFT_INDEX

FTAG_RANK

Page 17: COMA Tables (COnditions MetadatA)

17

Conditions DB Management Metadata Table Overview

For details (columns) see:http://www-pnp.physics.ox.ac.uk/~gallas/TAGs/1203_COMA_CB.ppt

Owner_Instances(Schema and Instances)

Nodes(Folders and Foldersets)

Schemas OnOffsInstances

PColumns

References

GTag_Infos

GTags(Global tags)

GT_TO_FTS

FTags(Folder Tags)

AMI_GTags

GT_TO_OIS

Page 18: COMA Tables (COnditions MetadatA)

18

Data Quality and Virtual Flags (DQ Stoplights: deprecated!)

DETSTATUSPF - RUN_INDEXPF - COOL_INDEXPF - DETECTOR_INDEXP – LBN_BEGINLBN_ENDCODEDEADFRACTHRUSTDESCRIPTION

COOL_CONNECTIONSP - COOL_INDEXCOOL_IMPLCOOL_SERVERCOOL_SCHEMACOOL_INSTANCEU - COOL_TAGCOOL_FOLDERCOOL_DATECOOL_DESC

DETSTATUS_DETECTORSP - DETECTOR_INDEXDETECTOR_DESCDETECTOR_LONGSYSTEMSUBSYSTEM

DETSTATUS_CODESP - CODECODE_DESCCODE_COLORCODE_HEX

RUNSP - RUN_INDEXF - DATA_SOURCEF - RUN_TYPE…U - RUN_NUMBER…START_TIMEEND_TIME…START_LBNEND_LBN…

DETSTATUS_GROUPSP - DETECTOR_INDEXCDG_MINCDG_MAX

DETSTATUS_VFLAGSPF – VF_COOL_INDEXPF – VF_CHANNELP – VF_RUN_BEGINVF_RUN_ENDVF_DESCRIPTIONVF_EXPRESSIONVF_DETECTOR_STRINGVF_RUN_STRING

DETSTATUS_VDETSP – VF_CHANNELVF_CHANNEL_NAMEVF_CHANNEL_DESC

Cool_H_TagsP – CHT_COOL_INDEXP – CHT_SUB_INDEX

V_RUN_DQTAGS

RUN_INDEX

COOL_INDEX

COOL_TAG

Page 19: COMA Tables (COnditions MetadatA)

19

Old ideas

Next slides contain old/obsolete Tables Views Ideas Pictures

Page 20: COMA Tables (COnditions MetadatA)

20

Obsolete ?

Event Counts by Collection/Run/Trigger

CRT_TCountsP – CRTT_INDEXFU – CRTR_INDEXFU – TRIGGER_INDEXRTC_COUNTRTC_FLAGCRTS_INDEX

CRT_RunsP – CRTR_INDEXUF – CRTC_INDEXUF – RUN_INDEXNB_EVENTSCRTR_FLAGCRTR_CHECKSUMCRTR_ACTIVECRTR_SECS

CRT_CollectionsP – CRTC_INDEXU1 – COLLECTION_NAMEU2 – FILENAME_TAGU2 – CRTS_INDEXU2 – AMI_TAGSCHEMA_NAMEFRX_TAG

CRT_StreamsP – CRTS_INDEXU1 – STREAMSTREAM_TYPESTREAM_NAME

Run_LBN_Trig_LumsP – RLTL_INDEXFU – RUN_INDEXFU – LBNFU – TRIGGER_INDEX

RLTL_INTLUMI_PSCAGG_PS (R)

Too many rows (10B)

Page 21: COMA Tables (COnditions MetadatA)

21

Obsolete Tables

RUN_STREAMSPF - RUN_INDEXP - STREAM_NAMESTREAM_EVENTSSTREAM_NAME_DS

RUN_LBN_STREAMSPF - RUN_INDEXP - LBNP - STREAM_NAMESTREAM_EVENTS

RUN_LBN_TRIGGERS•RUN_INDEX•LBN•L3_BIT•L3_ACCEPTS•L2_ACCEPTS•L1_ACCEPTS

RUN_LBN_LUMS*P - RUN_INDEXP - LBNP - COOL_INDEXINST_LUMLIVE_FRACTION ?

TRIGKEYSTRIGKEY_INDEXCOOL_INDEXHLT_KEYNAMELVLV1_KEYNAMETRIGKEY_DESC

TRIGGERSTRIGKEY_INDEXTRIGGER_LEVELCHAIN_COUNTERCHAIN_NAMECHAIN_VERSIONSTREAM_INFOPRESCALE_INITIALPASS_THROUGHLOWER_CHAIN_NAMETRIGGER_ELEMENTS

V_MK_TRIGGERS

TRIGKEY_INDEX

TRIGGER_INDEX

STREAM

L3_BIT

L2_BIT

L1_BIT

L3_NAME

L2_NAME

L1_NAME

L3_VERSION

L2_VERSION

L1_VERSION

L3_IPRESCALE

L2_IPRESCALE

L1_IPRESCALE

L3_PASS_THROUGH

L2_PASS_THROUGH

L1_PASS_THROUGH

L3_RERUN_PRESCALE

L2_RERUN_PRESCALE

RUN_LBNS (OLD)•RUN_INDEX•LBN•START_TIME•END_TIME•DURATION•INST_LUM•LIVE_FRACTION•COOL_INDEX*

GRL_Criteria_DQSP – GRLDQ_INDEXU – GRLDQ_CRITERIAU – GRLDQ_SYSTEM

GRL_TO_DQSP – GRL2DQ_INDEXFU – GRLF_INDEXFU – GRLDQ_INDEXGRL2DQ_GOOD

Page 22: COMA Tables (COnditions MetadatA)

22

STREAM_TAGS•STREAM_INDEX•STREAM_NAME•STREAM_TYPE•OBEY_LB

GROUPS

TRIGGER_ELEMENTS

HLT_PRESCALES

LVL1_PRESCALES

RUNSRUN_NUMBERRUN_TYPESTART_TIMEEND_TIME…TRIGKEY_INDEX

TRIGKEYSTRIGKEY_INDEXMASTER_KEYHLT_PRESCALE_KEYLVL1_PRESCALE_KEY

TRIGGERSMASTER_KEYTRIGGER_LEVELCHAIN_NAME CHAIN_VERSIONCHAIN_COUNTERLOWER_CHAIN_NAME…

COMA Runs and Triggers – Nov 2009 Fundamental

Secondary opts

Other options•This is schematic only from TAG meeting Nov 17, 2009 showing trigger related metadata that would/might be useful to include

•does not show the LB-wise changes in prescales

Page 23: COMA Tables (COnditions MetadatA)

23

STREAM_TAGS•STREAM_INDEX•STREAM_NAME•STREAM_TYPE•OBEY_LB

GROUPS

TRIGGER_ELEMENTS

TRIGGERSP - TRIGGER_INDEX

FU - MASTER_KEY_INDEXU - TRIGGER_LEVELU - CHAIN_NAME CHAIN_VERSIONCHAIN_COUNTERLOWER_CHAIN_NAME…

COMA Triggers, Streams, Elements, Groups – Nov 2009

Fundamental

Secondary opts

Other options

Notes:•Streams are definitely a piece of metadata (the association between streams and triggers) which would be useful selection criteria and help ELSSI to map selections to the CATALOG Metadata.•Elements and Groups look like they may also be of interests (but I don’t have a concrete sense for this data and its usage).

Page 24: COMA Tables (COnditions MetadatA)

24

Questions:1. Master Key Issue

1. Use negative MK for simulation ?1. Idea rejected (more than 2 sources) ?

2. Add column indicating ‘ConfigSource’ ?1. Adds PK to 2 tables with MASTER_KEY (ugh!)

3. Alt to 2. Add pseudo MK to MK table with source ? 2. Prescale type (int, float, string ?)3. PassThrough possible values/meanings ?4. Is Enable indicated by a prescale special value?

HLT_PRESCALESPF - HLT_PRESCALE_KEYPF – TRIGGER_INDEX

PRESCALEPASSTHROUGHENABLE ?

LVL1_PRESCALESPF - LVL1_PRESCALE_KEYPF – TRIGGER_INDEX

LVL1_PRESCALE

RUNSP - RUN_NUMBERDATA_SOURCERUN_TYPE…F - TRIGKEY_INDEX

TRIGGERSP - TRIGGER_INDEX

FU - MASTER_KEYU - TRIGGER_LEVELU - CHAIN_NAME

CHAIN_VERSIONCHAIN_COUNTERLOWER_CHAIN_NAME…

TRIGKEYSP - TRIGKEY_INDEXP - RUN_NUMBERP - START_LBP - END_LB

MASTER_KEYHLT_PRESCALE_KEYLVL1_PRESCALE_KEY

COMA Runs and Triggers – Nov 2009 – 2 Options:

HLT_PRESCALESPF - HLT_PRESCALE_KEYPF – TRIGGER_INDEX

PRESCALEPASSTHROUGHENABLE ?

LVL1_PRESCALESPF - LVL1_PRESCALE_KEYPF – TRIGGER_INDEX

LVL1_PRESCALE

RUNSP - RUN_NUMBERDATA_SOURCERUN_TYPE…F - MASTER_KEYF - PRESCALE_INDEX

TRIGGERSP - TRIGGER_INDEX

U - MASTER_KEYU - TRIGGER_LEVELU - CHAIN_NAME

CHAIN_VERSIONCHAIN_COUNTERLOWER_CHAIN_NAME…

PRESCALESP - PRESCALE_INDEXP - RUN_NUMBERP - START_LBP - END_LB

HLT_PRESCALE_KEYLVL1_PRESCALE_KEY

OPTION1 2

Page 25: COMA Tables (COnditions MetadatA)

25

Draft 1: Data Periods and Period GroupsRUNSRUN_INDEX

PERIOD_OBSP –CPO_INDEXPROJECT_PERIODPERIOD_NAMEFILENAME_TAGPP_FILE_TIME

PERIODSP –PROJECT_PERIODPERIOD_NAMEFILENAME_TAGPP_FILE_TIMEPP_START_TIMEPP_END_TIMEPP_RUN_STRINGPP_DESCPERIOD_RANK

PERIOD_RUNSP – PERIOD_NAMEP – RUN_INDEX

2010

implementation

PERIOD_NAMESP – PN_INDEXF – PD_NAME

PN_PROJECT_PERIOD

PN_PROJECTPN_PERIODPN_DESCPN_STATUS

PN_CREATE_USERPN_CREATE_DATEPN_MODIFY_USERPN_MODIFY_DATE

PERIOD_GROUPSP – PG_INDEXF – PD_NAME

PG_PROJECT_PERIOD

PG_PROJECTPG_PERIODPG_DESCPG_STATUS

PG_CREATE_USERPG_CREATE_DATEPG_MODIFY_USERPG_MODIFY_DATE

PERIOD_PG_TO_PNSP – PG_INDEXP – PN_INDEX

PERIOD_DOMAINSP – PD_NAME

PD_DESC

PERIOD_DOMAIN_ROLESPDR_INDEXPD_NAMEPDR_USERPDR_ROLE

PERIOD_PN_TO_RUNSP – PN_INDEXP – RUN_INDEX

RUNSP -- RUN_INDEX

PERIOD_PG_TO_PGSP – PG1_INDEXP – PG2_INDEX

PERIOD_GRLSP – PG_INDEXP – GRL_NAMEPGRL_STATUS

Page 26: COMA Tables (COnditions MetadatA)

26

Draft 1: Data Periods and Period Groups (tables and views)

PERIOD_NAMESP – PN_INDEXF – PD_NAME

PN_PROJECT_PERIOD

PN_PROJECTPN_PERIODPN_DESCPN_STATUS

PN_CREATE_USERPN_CREATE_DATEPN_MODIFY_USERPN_MODIFY_DATE

PERIOD_GROUPSP – PG_INDEXF – PD_NAME

PG_PROJECT_PERIOD

PG_PROJECTPG_PERIODPG_DESCPG_STATUS

PG_CREATE_USERPG_CREATE_DATEPG_MODIFY_USERPG_MODIFY_DATE

PERIOD_PG_TO_PNSP – PG_INDEXP – PN_INDEX

PERIOD_DOMAINSP – PD_NAME

PD_DESC

PERIOD_DOMAIN_ROLESPDR_INDEXPD_NAMEPDR_USERPDR_ROLE

PERIOD_PN_TO_RUNSP – PN_INDEXP – RUN_INDEX

RUNSP -- RUN_INDEX

PERIOD_PG_TO_PGSP – PG1_INDEXP – PG2_INDEX

PERIOD_GRLSP – PG_INDEXP – GRL_NAMEPGRL_STATUS

V_PG_TO_PNSPD_NAMEPG_INDEXPN_INDEXPG_PROJECTPG_PERIODPN_PROJECTPN_PERIOD

V_PN_TO_RUNSPD_NAMEPN_INDEXPN_PROJECTPN_PERIODRUN_INDEX

V_PG_TO_RUNSPD_NAMEPG_INDEXPG_PROJECTPG_PERIODPN_INDEXPN_PROJECTPN_PERIODRUN_INDEX

V_PPG_TO_RUNSPD_NAMEPG1_INDEXPG1_PROJECTPG1_PERIODPG2_INDEXPG2_PROJECTPG2_PERIODPN_INDEXPN_PROJECTPN_PERIODRUN_INDEX

V_PERIOD_TO_RUNSPD_NAMEP_LEVELP_INDEXP_PROJECTP_PERIODRUN_INDEX

Page 27: COMA Tables (COnditions MetadatA)

27

CB_FTAGSP – CBFT_INDEX

FU – CBF_INDEXU – TAG_NAME

COOL_CB Conditions DB Management Metadata

CB_NODESP – CBF_INDEX

FU – CBO_NAMEFU – CBS_NAMEFU – CBI_NAMEU – NODE_FULLPATH

NODE_NAMENODE_IDNODE_PARENTIDNODE_ISLEAFNODE_INSTIMELASTMOD_DATE

CBF_SCHEMA_NAMECBF_MODIFY_DATECBF_DESCCBF_TWIKI

CB_SCHEMASP – CBS_NAMECBS_SYSTEMCBS_DESC

CB_ONOFFS P – CBO_NAMECBO_DESC

CB_INSTANCESP – CBI_NAMECBI_DESC

CB_PAYLOADSP – CBP_INDEX

FU – CBF_INDEXU – CBP_COLUMN

CBP_TYPECBP_DESC

CB_REFERENCESPF – CBF_INDEX

FOLDER_IOVTABLENAMEFOLDER_TAGTABLENAMEFOLDER_IOV2TAGTABLENAMEFOLDER_CHANNELTABLENAME

CB_FOLDERSPF – CBF_INDEX

FU – CBO_NAMEFU – CBS_NAMEFU – CBI_NAMEU – NODE_FULLPATH

NODE_NAMENODE_DESCRIPTIONFOLDER_VERSIONING

CBF_NODE_PATHCBF_IOV_BASECBF_ATT_TYPECBF_CHAN_COUNTCBF_PAYLOAD_COUNT

CB_GTAG_INFOSP – CBGTI_INDEX

FU – TAG_NAMEU – STATUS_SINCE

STATUS_UNTILSTATUS_SUMMARYNEXTGTAG_IDUSED_FOR_TASKDEPLOYMENT_LOCATION

CB_GTAGSP – CBGT_INDEX

U – TAG_NAME

TAG_LOCK_STATUSTAG_DESCRIPTIONSYS_INSTIME

CBGT_INSTIMECBGT_RANK

CB_GT_TO_FTSP – CBG2F_INDEX

FU – CBGT_INDEXFU – CBFT_INDEX

CB_ALL_FTAGSP – CBFT_INDEX

FU – CBF_INDEXU – TAG_NAME

TAG_LOCK_STATUSTAG_DESCRIPTIONSYS_INSTIME

CBFT_INSTIMECBFT_ROWCOUNTCBFT_LAST_OBJTIMECBFT_SINCE_TIMECBFT_UNTIL_TIME

CB_GTAG_AMISP – TAG_NAME

IS_ACTIVEDATASET_COUNTProcessing_Date_Range ?

GRAY table

s

Not fil

led ye

t !

CB_GTAG_AMI_USESTAG_NAMEProject_NameRun_StartRun_End

AMITag_Array ?

Page 28: COMA Tables (COnditions MetadatA)

28

COOL Browser Support Tables

CB_NODESP – CBF_INDEX

FU – CBO_NAMEFU – CBS_NAMEFU – CBI_NAMEU – NODE_FULLPATH

NODE_NAMENODE_IDNODE_PARENTIDNODE_ISLEAFNODE_INSTIMELASTMOD_DATE

CBF_MODIFY_DATECBF_DESCCBF_TWIKI

CB_SCHEMASP – CBS_NAMECBS_SYSTEMCBS_DESC

CB_ONOFFS P – CBO_NAMECBO_DESC

CB_INSTANCESP – CBI_NAMECBI_DESC

CB_CHANNELSP – CBC_INDEXFU – CBF_INDEXU – CBC_CHANNELCBC_NAME

CB_PAYLOADSP – CBP_INDEX

FU – CBF_INDEXU – CBP_COLUMN

CBP_TYPECBP_DESC

CB_REFERENCESPF – CBF_INDEX

FOLDER_IOVTABLENAMEFOLDER_TAGTABLENAMEFOLDER_IOV2TAGTABLENAMEFOLDER_CHANNELTABLENAME

Too many channels

CB_Folders.CHAN_COUNT

contains the channel count

CB_FOLDERSPF – CBF_INDEX

FU – CBO_NAMEFU – CBS_NAMEFU – CBI_NAMEU – NODE_FULLPATH

NODE_NAMENODE_DESCRIPTIONFOLDER_VERSIONING

CBF_NODE_PATHCBF_IOV_BASECBF_ATT_TYPECBF_CHAN_COUNTCBF_PAYLOAD_COUNT