5/17/2004VOMRS Technical Overview1 VOMRS Technical Overview John Weigand

5/17/2004 VOMRS Technical Overview 1

VOMRSTechnicalOverview

John Weigand


VOMRS Technical Overview

1. VOMRS Architecture

2. VOMRS database

3. Testing

4. Open Issues


VOMRS Architecture

1. Access (Client / Web)CLI

GSI authentication (Java Cog Kit)SOAP/SSL authentication (EDG Trust Manager)Java

Web UI HTTP/SSL authentication (EDG Trust Manager)Java / Java script

VOMRS Admin Java / Axis / Tomcat WEB service

2. VOMRS Server (Multi-threaded daemon process) : Client Manager

Handles remote client request to perform VO membership serviceEvent /Manager

Handles member event and interface notificationsInterface Manager

Handles interface notificationsVOMS Synchronizer

Synchronizes VOMS with VOMRS database

3. ServiceBroker / ServicesAuthorizationBusiness Logic

http://www-unix.globus.org/cog/java/

http://edg-wp2.web.cern.ch/edg-wp2/security/edg-java-security.html

http://edg-wp2.web.cern.ch/edg-wp2/security/edg-java-security.html


VOMRS ArchitectureWeb Access

1. Two ways of accessing the VOMRS besides VOMRS client:a. web UI (HTTP+SSL authentication)

http://computing.fnal.gov/docs/products/vomrs/#manual.

b. web services (SOAP+SSL authentication)

https://vomrs_host:8443/vo-<VO_NAME>/services/VOMRS?wsdl

2. All access methods utilize the same implementation of the business logic.

3. Authentication provided by the EDG Trust Manager(http://edg-wp2.web.cern.ch/edg-wp2/security)

4. WEB UI behavior controlled by configuration file parameters: a. The duration of the session

b. The name of organization that oversees all grid resources for a particular VO

c. The url location of usage rules for this organization

Configuration file entries:<web>  <log>VOMRS_DIR/var/etc/vomrs_VO_NAME_CNVRT/log4j_webui.properties</log>   <gridResourceName>LCG</gridResourceName>  <gridUsageRules>https://edms.cern.ch/file/428036/LAST_RELEASED/LCG_Usage_Rules.pdf</gridUsageRules></web>

http://computing.fnal.gov/docs/products/vomrs/#manual

http://computing.fnal.gov/docs/products/vomrs/#manual

https://vomrs_host:8443/vo-%3CVO_NAME%3E/services/VOMRS?wsdl

http://edg-wp2.web.cern.ch/edg-wp2/security


VOMRS Architecture Client / Server Access

1. ClientManager is a thread listening for connection from remote clients

2. When connection is established, a Client Manager starts a temporary new thread (ClientIF)

a. authenticates a client by using GSI authentication mechanism b. execute the requested VO membership services c. returns information regarding the success or failure of a requested service as

well as the execution result of this service if any

3. Thread will be stopped either whena. client closes the connection or b. the allowed connection time is expired whichever comes first. c. The allowed connection time can be set up in configuration.

Configuration file entries:<client> <!—change to on to enable remote client access, client should use VOMRS API and GSI authentication <status>off</status> <port>14000</port> <!—allowed connection interval in minutes <connectionTime>10</connectionTime></client>


VOMRS Architecture Service Broker / Services

1. All 3 access methods utilize a common ServiceBroker and a common set of Services.

2. All use the same business logic to satisfy a requested service.

Service Arguments Return Values Authorized Roles----------------- ------------ ------------- ------------------------GetGroupOwners GROUP DN,CA Visitor,Applicant,MemberAddGroupOwner DN,CA,GROUP NULL GroupOwner,VOAdminRemoveGroupOwner DN,CA,GROUP NULL GroupOwner,VOAdminGetLRPs INSTITUTION DN,CA Visitor,Applicant,MemberAddLRP DN,CA NULL SiteAdmin,VOAdminRemoveLRP DN,CA NULL SiteAdmin,VOAdmin

3. Services define:a. The action performed within VOMRS database

b. Administrative roles authorized to perform this action

c. List of input and output arguments

4. Authorization restrictions:a. Basic is based on role and arguments implemented in the ServiceBroker

b. Additional restrictions on some implemented in the Services code. (e.g. – AddLRP by SiteAdmin can only be performed on entities associated with his site, while VOAdmin can perform this for any site.)


VOMRS Architecture Service Broker / Services


5. ServiceBroker performs:a. Service availability

b. Service authorization based on role and arguments

c. Execution of the authorized service

d. Return data/success/failure of the service to the requestor.


VOMRS Architecture Event Manager

1. Thread of the VOMRS Server that processes all the notification events stored in VOMRS database.

2. Frequency controlled by the VOMRS configuration file.

3. Performs the following tasks:a. Checks database for any event that is ready to be processed

b. Determines current subscribers to this event

c. Formats and sends e-mail notification

d. Updates database with the outcome of event processing and notification (“Completed”/”Failed”)

Configuration file entries:<eventProcessor> <!—change to off to disable event processing <status>on</status>

<!—command use to send mail <mailCommand>/usr/sbin/sendmail</mailCommand>  <url>https://hotdog62.fnal.gov:8443/vo-VO_NAME/services/VOMRS?</url> <from>vomrs-VO_NAME_CNVRT-admin</from> <!—update period in minutes <updateTime>2</updateTime></eventProcessor>


VOMRS Architecture Interface Manager

1. Incomplete at this time. VOMS in the only interface support and it is done via the VOMS Synchronizer

2. Thread of the VOMRS Server that processes all the events that are relevant to a particular interface

3. Frequency will be controlled by the VOMRS configuration file.

4. Will performs the following tasks:a. Checks database for any interface event that is ready to be processed

b. Determines current interface subscribers to the event

c. Formats event and contacts a subscribed interface by using appropriate API

d. Updates database with the outcome of event processing and notification (“Completed”/”Failed”)

5. Only an interface to VOMS Admin is currently implemented


VOMRS Architecture VOMS Synchronizer

1. Thread of the VOMRS Server that transfers information from the VOMRS database to the VOMS database using EDG VOMS Admin API.

2. Behavior is controlled by the VOMRS configuration

3. Two (2) modes of operations:a. event driven mode - level 4

- using the Interface Manager to handle each event relevant to VOMS (subscribed by VOMS interface).

b. Synchronization mode

- The other mode allows a Synchronizer to synchronize the VOMRS and VOMS databases.

4. Four (4) levels of synchronization:a. Level 0 - Addition of all “approved” VOMRS members to a root group of the

VOMS database

b. Level 1 - Addition of all “approved” VOMRS members to a root group and deletion of all “suspended” VOMRS members from the of VOMS database

c. Level 2 - Synchronization of all members. Includes deletion of all members that are listed in the VOMS database but do not exist in the VOMRS database.

d. Level 3 - Complete synchronization. Includes synchronization of groups and roles, member assignment to the subgroups and member role within subgroups.


VOMRS Architecture VOMS Synchronizer

Configuration file entries:<synchronizer> <!—change to on to enable syncronization, will use VOMS API and SOAP/SSL authentication <status>off</status> 

 <vomsUrl>https://VOMS_hostname:8443/edg-voms-admin/VOMS_VO_NAME/services/VOMSAdmin</vomsUrl>









</synchronizer>


VOMRS Database

1. Currently MySql with intent to support Oracle.

2. 3 categories of tables:a. Core tables

• contain the basic updatable VO membership data

b. Code related tables

• Tightly coupled with the application’s code

• Only modified in conjunction with the code

• Not updated with available VOMRS services

c. Log tables

• Basically log ‘files’ in table form documenting changes in a member’s registration status and roles.

• Events/subscriptions/notifications for members and system Interfaces

3. ‘Code related’ and ‘Log’ tables used to support:a. Concept of Services

b. Event (member and interface) notifications

c. It is an attempt to reduce the amount of ‘hard-coding’ that would have otherwise been required.


VOMRS Database ERD

Code related tables Core tables Log tables

Disclaimer: The above ER diagram was created using the Oracle Designer tool. It does not include all the data relationships present in the VOMRS database, but only the most relevant. This is a variation from the normal use of this tool and was done so intentionally to simplify the diagram. Since this was originally designed for use in a MySql DBMS, the foreign key relationships are in the software rather than the DDL.


VOMRS DatabaseVO Membership Data

(Core Tables)

1. Registration Statusa. Within VO (New, Approved, Denied, Suspended, Revoked)

b. By site (New, Approved, Denied)

c. By local resource provider (New, Approved, Denied)

2. Personal Dataa. Configurable by VO

b. Private and public data

3. Multiple DN/CA per member

4. Rolesa. Visitor, Applicant, Member, Group Manager, Group Owner, Site

Administrator, Local Resource Provider (LRP), VO Administrator

b. Grid/Group roles (no validation at this time, just repository)

5. Groupsa. Modifiable by VO

b. Can assign roles for specific groups (both admin and grid roles)


VOMRS DatabaseServices

(Code Related Tables)

1. Defines the authorized functions that can be performed by a VO member in the VOMRS database.

2. Tablesa. Services table

Defines the service, required arguments and return_values

b. Service_roles table

Defines the VO member administrative roles that can perform the service

3. Tight coupling of code and tablesa. Member role dictates menus presented in the web interface

b. Service, arguments and roles used in determining if member is authorized to perform the service. (the authorization issue is in not knowing the user’s intended role)



VOMRS DatabaseEvents / Event Logs

(Code and Log Tables)

1. Each service that modifies VOMRS data generates one or more events.

2. The service “dumps” all relevant data for an event / service to a “log” in the form of a database table.

3. The event_log / event_value tables are ‘instances’ of an event.

4. This data is recorded in the form of name/values pairs

4. Each event can have 1 or more event types associated with it allowing for different forms of member and interface notifications to be managed. Just one set of values.

5. Coupling to the code:a. The service has to know what event to generate (e_tag is used in the code)

b. By dumping most everything, we don’t have to modify the code for additional data.

EVENTSE_ID 020E_TAG MemberAssignedToGroup

EVENT_VALUESEL_ID 357E_ID 020DN /DC=org/DC=DOEGrids../CN=…..CA DOEGrids GROUP /root/atlasIID 10MID 1001REPID 10RIGHTS fullROLE MemberSELFDN /DC=org…../CN=WeigandSELFCA /DOEGrids


VOMRS DatabaseEvents / Event Types

(Code and Log Related Tables)

1. Each event can have 1 or more event types associated with it, allowing for different forms of member and interface notifications to be managed.

2. Event description (et_descr)a. For members, it is generally the ‘subject’ of the email

b. For interfaces, it is less well defined at this point, but generally might be like an ‘action’ indicator.

3. Event template (et_template)a. For members, it is the body of the email.

b. For interfaces, it would generally be the protocol or format of the interface trasaction.


VOMRS DatabaseEvents / Event Types

(Example)

Event MemberDNDeleted has 3 event types

1. For notifying the affected Member: Description:

An alias has been removed for you Template:

Dear VO Member,Your alias with Distinguished Name %MAINDN%issued by Certificate Authority%MAINCA% has been deleted by %SELFDN%, %SELFCA%.VO Administrator</html>

2. For notifying an administrator / representative: Description:

Member has removed an aliasTemplate:

<html>Dear Administrator,The VO member with Distinguished Name %DN% issued by Certificate Authority %CA% deleted an alias with Distinguished Name %MAINDN% issued by Certificate Authority %MAINCA% . Alias has been deleted by %SELFDN%, %SELFCA%.VO Administrator</html>

3. For an interface: Description:

VOMSAction="Member has deleted an alias“ Template:

deleteVOMSUser;%DN%;%CA%


VOMRS DatabaseEvent Types / Event Subscribers(Code and Log Related Tables)

1. Each event can have 1 or more event types associated with it allowing for different forms of member and interface notifications to be managed.

2. Members subscribe/ unsubscribe to the events available for their role in the VO using the VOMRS service.

3. New interfaces are like code. There is no UI. Must be managed by an administrator or new release. Framework is such that we think this design will make addition of interfaces easier.

4. All yellow tables are pre-populated and are treated as part of the code.

5. You will notice we violate the normal parent-child forwarding of primary keys in tables such as event/event_types and event_type/role_event.

a. This was done intentionally

b. The services are only interested generating events and event_values. The services do not care about event_types.

c. The subscribers only care about the event description (er_descr) and the message/data associated with it (et_template).

d. This made it easier to maintain the table data which is effectively source code.


VOMRS DatabaseEvent Notifications – Members / Interfaces

(Log Related Tables)

1. The member_event_recipients and interface_event_recipients tables area used to track the notification attempts and feedback.

2. When an event occurs these tables are populated based on subscriptions of each event type.

3. The Event Manager / Interface Manager / VOMS Synchronizer then:a. Perform a substitution of the event’s values in the template

b. Send the notification.

c. Update the recipients tables accordingly.


VOMRS Technical OverviewTesting

1. Automated testing of business logic (services) using client / server.

a. Since web access side uses same services, business logic is covered for both.

b. They also share a “service broker” which performs the authorization functions.

2. 700+ tests performeda. it is a rudimentary test (i.e., pass/fail). Has some data validation

checks: 1) add something… add in again expect failure 2) test service created to verify templates substitutions are satisfied.

b. Lacks automation for all 9 administrative roles. Just tests VOAdmin roles.


VOMRS Technical OverviewOpen Issues / Additional Enhancements?

1. Servicesa. Currently have a tight coupling of the code with the return values.

The code MUST return all data in the same order as specified in the service table.

2. Event schedulera. No means of evaluating returned email.

b. Email only notification mechanism. Are other methods needed?

c. No UI at this point if notification done manually

3. VOMS synchronizationa. Synchronization not the most efficient means of interfacing.

However, handshakes on interfaces are difficult to implement.

b. No real means of processing feedback/handshake with VOMS if synchronization not performed.

4. Other interfacesa. None defined. Will require analysis on an interface by interface

basis most likely. Template concept could minimize protocal issues.

b. In most case will have to be serialized.

5. Testinga. Manual testing of the Web interface very, very time consuming

b. Client/server testing needs to be more robust. Pass/Fail is OK, but not adequate.

c. Lacks ability to test all 9 roles.

6. Support for ‘Group (Grid)’ roles.a. No validation criteria. Just a repository. Adequate?

Documents

5/17/2004VOMRS Technical Overview1 VOMRS Technical Overview John Weigand