3D Testing and Monitoring

Lee Lueking

LCG 3D Meeting Sept. 15, 2005

Aug. 25, 2005 Conditions DB Deployment 2

Outline

• Overview• Schedule and Milestones (CMS Side)• Workplan (w/ 3D side)

Aug. 25, 2005 Conditions DB Deployment 3

Software Stack

• EDM EventSetup assures the correct non-event data is accessed and available for the user application.

• POOL-ORA (Object Relational Access) is used to map C++ objects to Relational schema.

• A POOL-RAL/FroNTier-Oracle plug-in is used to to enable a middle-tier proxy/caching service for read-only access. Have working example.

• The central DB is ORACLE, but other technologies can be used for testing ( MySQL, SQLite, …)

User Application

EDM Framework

EDM EventSetup

POOL-ORA

Frontier Cache

Database

Optional

POOL-RAL Plugin

Aug. 25, 2005 Conditions DB Deployment 4

N-Tier Deployment• Redundant Tomcat + Squid servers are

deployed at Tier 0 (launchpad). • Squids are deployed at Tier 1, and Tier N

sites. Will be an “edge service” for Grid computing centers. Configuration includes:– Access Control List (ACL)– Cache management (Memory and Disk)– Inter Cache sharing (if desired).

• Applications needing access to conditions data are configured with a list of servers, and proxies. For Example:

– FRONTIER_SERVER1=http://host1.domain.net:8000/Frontier export

– FRONTIER_SERVER2=http://host2.domain.net:8000/Frontier export

– FRONTIER_PROXY1=http://proxy. domain.net:10000

Squid(s)Tomcat(s)

Squid Squid Squid

DB

Squid Squid Squid

Tier 0

Tier 1

Tier N

FroNTierLaunchpad

Aug. 25, 2005 Conditions DB Deployment 5

Fall 05 Testing Plan

1. Load estimated data for ~1 year’s running (~500GB)

2. Exercise various loading and access patterns to simulate stable operations and worst case scenarios. Limited distributed deployment (eg CERN, FNAL and HLT Farm @ CERN).

3. Deploy Squid servers to Tier 1 and Tier 2 centers. Repeat testing under various loads and access patterns. Looking for sites interested in participating.

Aug. 25, 2005 Conditions DB Deployment 6

Integration Schedule

• Oct. 1: Central DB Configured and Loaded for DB test Activity.

• Oct. 15: Small scale test CERN+FNAL of distributed DB.

• Nov. 1-30: Large scale test of distributed DB.

• Dec. 20: DB test Evaluation Report. Milestone

Aug. 25, 2005 Conditions DB Deployment 7

Application

Object Diagram Radovan Chytracek

IMonitoring

IMonitoringService

IRelationaService

IRelationaSession

FrontierAccessCORAL PLUG-IN

MySQLAccessCORAL PLUG-IN

OracleAccessCORAL PLUG-IN

CSVReporter

DBReporter

XMLReporter

ApMon

Aug. 25, 2005 Conditions DB Deployment 8

MonALISA ApMon• http://monalisa.caltech.edu/dev_ap_mon.html• ApMon is a set of flexible APIs that can be

used by any application to send monitoring information to MonALISA services.

• The monitoring data is sent as UDP datagrams to one or more hosts running MonALISA services.

• Applications can periodically report any type of information the user wants to collect, monitor or use in the MonALISA framework  to trigger alarms or activate decision agents.

• We provide ApMon implementations for 5 programming languages: C, C++,  Java, Perl and Python.

Aug. 25, 2005 Conditions DB Deployment 9

Next Steps (w/ LCG3D)• Monitoring

– Prepare CORAL client monitoring framework Radovan    described a few weeks ago.

– Prepare udp communication with MonALISA based message collector. – prepare information strings to monitor who is the requester?, where is the request

coming from?, what is the request?, when was the request made?, how long did it take to reply?, how large was the reply?, possibly what was the intermediate routing requested?

• Testing repository, Frontier plug-in, etc. – Continue testing with actual CMS POOL data objects. Can use existing DB and

frontier server at FNAL to start. Need to be able to build with CMS software. – Start using 3D test infrastructure at CERN. We can start with devdb10 and existing

3d frontier server. – Set up CMS POOL repository for stress and access pattern testing. Need to move to

test DB server for a while to understand loading needs better. – Populate repository and test.

The End