Ariadne: Tutorial

Ariadne:Tutorial

Illya ShapovalCERN, KIPT, UNIFE, INFN-FE

2nd LHCb Computing Workshop4th-8th November 2013

CERN

8-10-2013 2nd LHCb Computing Workshop , 4th-8th November, CERN 2

Content

● Introduction● Querying Ariadne● Updating Ariadne● Use case of compatibility


Introduction: terminology● Ariadne knowledge graph (AKG)

– A graph holding all LHCb metadata entities (nodes) and relationships (edges) between them (to the best of managers’ knowledge).

● Ariadne domain data model– An abstract graph, describing by its pattern topology of

relationships in a specific application domain. ● Ariadne solution

– Subgraph of the Ariadne knowledge graph, matching concrete graph pattern, requirements on nodes’ attributes and relationships types.

● Ariadne thread– An Ariadne solution that matches entirely domain data model.


Introduction: metadata identifiers● Raw identifier: dictionary of attributes, for use in low-level CL

tools, e.g.:{'type': 'application', 'name': 'DaVinci', 'version': 'v33r8'}

{'type': 'detector_type', 'name': '2013'}{'type': 'reco_type', 'name': 'Reco14'}

Special case: {'type': 'tag', 'partition': 'DDDB', 'name': 'dddb-20120831', 'release_date': '2012-08-31'}

● Shortcut identifier: Ariadne node index value, for use in high-level CL tools:

application:DaVinci:v33r8detector_type:2013reco_type:Reco14

tag:DDDB:dddb-20120831


Introduction: Ariadne CL tools● Querying Ariadne

– ariadne_query, high-level query tool, used by you up to now● available in the LbLogin environment● from anywhere with web access● no authentication and authorization (AA) required

● Updating Ariadne– ariadne_add, high-level admin tool, used by you up to now

● available in the LbLogin environment● from anywhere with web access● AA required: talks through CERN SSO (valid CERN Kerberos ticket)

– AriadneAdmin.py: powerful low-level admin tool, avoid using it;● available in the “SetupProject Tools --runtime LCGCMT --use Ariadne” environment● from anywhere with web access● AA required: talks through CERN SSO (valid CERN Kerberos ticket)

● New: generic tool for all tasks, recommended to use– ariadne, high-level admin tool for both updating and querying

● available in the LbLogin environment● from anywhere with web access● AA is conditional: no AA for querying, and AA for updating (valid CERN Kerberos ticket)

Note1: only members of lhcb-core-soft are authorized to update AKGNote2: where AA is required, cern-get-sso-cookie has to be installed


Querying Ariadne (with CL tool)ariadne q[uery] [OPTIONS] IDENTIFIER [IDENTIFIER2]

(see ariadne help, or ariadne query -h )

A. Check DaVinci v33r8 is tracked:ariadne query application:DaVinci:v33r8

B. Fetch entities, directly related to DaVinci v33r8:ariadne query -r application:DaVinci:v33r8

C. Fetch DetectorTypes, directly related to DaVinci v33r8:ariadne q -r application:DaVinci:v33r8 | grep detector

D. Fetch DDDB tags, directly related to DaVinci v33r8:ariadne q -r application:DaVinci:v33r8 | grep DDDB

E. Check DaVinci v33r8 and DetectorType 2013 have direct relationship:ariadne q application:DaVinci:v33r8 detector_type:2013

Let’s go around an application, for instance, DaVinci v33r8


Querying Ariadne (from Python)● A simple XMLRPC framework is provided to be able to send questions to

Ariadne, and get solutions from it:from xmlrpclib import ServerProxyariadne = ServerProxy("http://ariadne-lhcb.cern.ch/xmlrpc/", allow_none = True)solution = ariadne.aquery(question, criterion)

● A common question/solution skeleton provided to help users in communicating with Ariadne, e.g.:

question = { 'application': 'Brunel', 'application_version': 'v43r2p2', 'detector_type': '2012', 'reco_type': 'Reco14', 'DDDBTag': None, 'LHCBCONDTag': None, 'DQFLAGSTag': None }solution= { 'application': 'Brunel', 'application_version': 'v43r2p2', 'detector_type': '2012', 'reco_type': 'Reco14', 'DDDBTag': 'dddb-20120831', 'LHCBCONDTag': 'cond-20130404', 'DQFLAGSTag': 'dq-20121016' }Note: as criterion use 'latest_LHCBCOND_DDDB‘ to get this solution

● 'DDDBConf' already got the AutoTags feature, based on such query mechanism (>=LHCB_v35r6)from Configurables import DDDBConfDDDBConf( DataType = "2012", AutoTags = True )


Updating AriadneResponsibility of application manager

Compatibility data model

Consider the use case of compatibility. Imagine, new Brunel vXrY is being released.

A

TDTC

TQ

D

R

A

TDTC

D

R

New node: Brunel vXrY

? ??

?

Relationships, needed to be considered by application

manager

1-level deep only!


Updating AriadneResponsibility of CondDB manager

Compatibility data model

Consider the use case of compatibility. Imagine, new DDDB tag is being released.

A

TDTC

TQ

D

RNew DDDB tag

?

?

?

Relationships, needed to be considered by CondDB

manager

1-level deep only!

A

TDTC

D

R

?


Updating Ariadne (with CL tool)ariadne add [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]]

(see ariadne help, or ariadne add -h )

TDTC

D

R

Ariadne Knowledge Graph space

(zillions of other AKG nodes are not shown)

A. Compatible entities identified, and exist in AKG

A




TDTC

D

R




B. Brunel vXrY node added:ariadne add application:Brunel:vXrY

A

A New node: Brunel vXrY

B








C. Brunel vXrY is declared compatible with RecoType “Reco”

ariadne add application:Brunel:vXrY \ +reco_type:Reco

A


BA

TDTC

D

R

C










D. Brunel vXrY is declared compatible with the rest of entities:

ariadne add application:Brunel:vXrY \ +detector_type:2013 +tag:DDDB:dddb-ZZZ \ +tag:LHCBCOND:cond-WWW

A


B

C

A

TDTC

D

RD D D












A


B

C

A

TDTC

D

RD D D

Or, equivalently, in a single command:ariadne add application:Brunel:vXrY +reco_type:Reco +detector_type:2013 \

+tag:DDDB:dddb-ZZZ +tag:LHCBCOND:cond-WWW






E. Next day we figure out Brunel vXrY is NOT really compatible with RecoType “Reco”:ariadne add application:Brunel:vXrY -reco_type:Reco

A


B

E

A

TDTC

D

RD D D












vXrYA

If new Brunel vXrZ, instead, occurs to be entirely compatible with all entities the previous Brunel vXrY is compatible with, then:

ariadne add Brunel:vXrZ =Brunel:vXrY vXrZA


Visual Mining in Ariadne (with Neo4j WI)https://ariadne-lhcb.cern.ch/webadmin/


Updating Ariadne (with Neo4j WI)https://ariadne-lhcb.cern.ch/webadmin/

Documents

Ariadne: Tutorial