Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
AMACAA Web Application for an AFS Cell monitoring
A. Rocchi, G. Bracco, S. Migliori, S. Podda, A. Santoro, C. Sciò
OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Summary Motivations Operating scenery
ENEA-GRID infrastructure
AMACA Software core modules
Main features
Conclusions
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Motivations
OpenAFS monitoring in a complex environment can be tricky
No visual tools for analysis
It is impossible to track state modifications of core components through time.
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Why not a NAGIOS plugin? Some OpenAFS control components have already
been written to be integrated in NAGIOS.
We choose not to pursue this strategy:
Nagios is designed to be a “quasi-realtime” general purpose monitor
AMACA, instead, aims to be a “periodic photographer” of OpenAFS, with sophisticated features of analysis and comparison among snapshots taken in different moments.
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Operating scenery ENEA (Italian National Agency for New Technologies,
Energy and the Sustainable Economic Development)
12 research sites, 6 computer centers
Computational resources are integrated in a global grid infrastructure (ENEA-GRID)
AFS is the lower layer, providing information distribution across multiple sites
One cell (enea.it), multiple served locations
See G. Bracco presentation (AFS in a grid context)
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AFS Memorize And CheckApplication
• Initial development in collaborationwith CASPUR
Two cooperating modules
• CRAWLER ↦ scans state ofOpenAFS core components, using the N. Gruener’s Perl API
• EXPLORER ↦ Provides a convenient visualizationinterface to the crawling results
Access to application modules isvalidated by Unified Access APIs
AMACA - Architecture
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – Database splitting The DB backend is splitted in two
• A production portion is used to store results of nightly indexing. Users can’t modify its content.
• A scratch portion, filled up by custom invocations of the crawler. Only users belonging to specific (customizable) pts groups
can write on it
For example, convenient for admins who want to see the effect produced by management tasks!
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Crawler stores unstructured data in a MySQL backend
• fine-grained data retrieval operations
Differentiation among subsequent crawler invocations is done by identifying every resultset with a unique ID (snapshot)
Explorer implements business logic of data aggregation and display
• Historical analysis on file system variations
• Parameterized search queries
AMACA - Details
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Why?
• OpenAFS + K5 integration: strong authentication over the entire GRID realm.
One account, many services: It would be good to keep only one set of credentials to access anywhere.
In particular, UAA is meant to be used to protect web resources (i.e.: webapps and static HTML pages)
• However, we need to be sure that only selected users will use some applications or software features
allowed users don't have to be system administrators!
Unified Access API (1)
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
How?
• Server-side Inter Process Communication with PAG shell
We need to be sure that every race condition on tokenacquisitions is avoided
• Requests for login and PTS membership are sent, and results are compared with user-defined entries.
• If access is granted, an encrypted cookie is generatedin order to store information
AMACA – UAA (2)
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – Explorer The Crawler can be ran in a stand-alone way (by command line invocation)
Explorer provides a convenient way to dig into indexed results.
• Web application, PHP/AJAX
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – GUI explained The explorer module exposes 4 groups of
information and controls:
• Global system state (sync site, volumes and partitions occupation stats, max Volume ID, …)
• Customization components
Admins can restrict the whole view to a specific cell subset (i.e.: to a single site)
• Alarms Highlight of (more or less) critical situations
• Navigation menu
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – Main page
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA - Alarms Alarms are a way to highlight potentially critical
situations.• Locked and unattacheable volumes• Servers send/receive ratio threshold (via rxdebug)• Overbooked partitions (i.e.: sum of volumes reserved
quotas is greater than actual partition size)• Volumes mapped only in Volume servers OR in file
servers• Volumes which don’t respect the numbering schema
(RO=RW+1, BK=RW+2)• …
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – Alarms (2) Alarms (as everything else in AMACA) are
discrete-time evaluated It could happen that criticality is really low
E.g.: a server, offline at the moment when snapshot has been taken, can become up and running at the very next moment
In any case, admins have to verify if there is some issue waiting to be solved.
For that reason, AMACA allows “power users” to run crawling directly from the Explorer
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Using AMACA – Login form
Using AMACA – Online crawling
Press the camel-shaped icon, wait
for the
crawler to finish, and see if there’s
some alarm still alive
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Using AMACA – Online crawlingCrawler invocation:
an asyncronouspopup appears. Waitfor the run to finish.
After a while (time dependson network conditions and cell size), results are shownin form of text output.Close the window and explore results.
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
AMACA – Analysis features The upper part of the navigation menu exposes 4 voices
• “List Differences” is a true diff between 2 snapshots Given two referring instants, AMACA searches what is changed
between them.
• “Volume or server search” Finds volume or server in every snapshot, using wildcards and size constraints
• “Time search” For a given entity, shows how its state has changed through time.
• “Volume history” Volume details (similar to vos ex) with respect to a single snapshot.
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Conclusions AMACA has been in production within our
infrastructure for 2.5 years.
• Downloadable tarball (source code + documentation) will be soon available.
Beta-tester/contributor anyone? :-)
• It’d be interesting to run AMACA on bigger cells than enea.it
• Performance improvement and features extension
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome
Thank you!!!
Any Questions?
A. Rocchi et al. - OpenAFS European Meeting – September 2009 – Uniroma3, Rome