The National Grid Service User Accounting System Katie Weeks Science and Technology Facilities Council

The National Grid Service User Accounting System

Katie Weeks

Science and Technology Facilities Council

Katie Weeks – [email protected] 13th September 2007

The National Grid Service

• The NGS is the UK’s grid for academics• Free and easy to apply online• Providing a number of heterogeneous compute and data

resources on the grid• Providing help and support to those using the grid • The NGS is led and coordinated by the STFC in

collaboration with the University of Manchester, the University of Oxford, the University of Edinburgh and the White Rose Grid at the University of Leeds


The National Grid Service

• Over 590 registered users

• 4 core sites, 6 partner sites and 3 affiliate sites

• Entered production in Autumn 2004

• Now in phase 2

• Predominantly focussed on compute and storage at present


Grid Accounting

• Accounting for any production grid is an important part of the monitoring process– Pricing policies may be introduced to grids in the future– To uphold policies relating to grid use and allocated hours– To monitor systems – particularly important for funding and

future planning– To have an overview of the system – how much are we

allocating? How much is being used? How much spare capacity do we have? How much are our biggest users using?

• JISC has recently funded a review of accounting and usage monitoring

• It’s an issue many grids now face


Grid policing

• Users are allocated limited resources• Important to know how much of those resources

have been consumed• Users tend to go over quota even when

monitored• Need to ‘lock-out’ users who go over quota• There is an important distinction between

accounting and policing• Retain integrity of application and peer-review

process


Our problem

• Situation at beginning of 2006:– Applications under go light-weight peer review

process– Ability to monitor individual usage– Policies in place for usage– No tools to use the monitoring data to enforce the

policies– Needed a system to enable us to manage users and

enforce allocation policies


Other solutions

• APEL (EGEE)– Accounting Processor for Event Logs– Accounting information such as CPU time, Wall Time and DN– Virtual Organisation emphasis with no individual policing

• TeraGrid– Account monitoring available via command line– No architecture in place for automatic policing

• OGF– Resource Usage Service (RUS)– Stores monitoring information that can be queried– NGS has had a RUS instance since early 2006

No current solution met all the needs of the NGS – needed our own solution


Our Solution

• Resource usage data obtained by querying the Resource Usage Service (RUS) at Manchester

• RUS already in place and had functionality the NGS required• Accounting system based on an Oracle database• Interface to the Oracle database was created using Oracle

Application Express (Apex)• Information collected

– User details– Application information– Account status– Resource allocation– Resource usage

• Usage and allocation is collated over all core sites• Records historical information• Records changes in Distinguished Name (DN)



Policing the NGS

• User Accounting System (UAS) queries the RUS every day for total CPU and disk space for every user

• A warning email is sent out when you reach 90% of your CPU allocation

• The account is automatically locked and an email sent when you reach 100% of your CPU allocation


Policing the NGS (2)

• The Lightweight Directory Access Protocol (LDAP) queries the User Accounting System every hour

• It’s only populated with users whose account has ‘active’ status

• Users whose accounts are locked or pending are not included in the LDAP

• Maximum of hour before accounts are active again once they’ve been unlocked


Policing the NGS (3)

• When an account is locked, you can apply for more resources– Via application form– Via your account details

• When your application is successful, your account is automatically updated with your new allocation and account is ‘active’ again

• An email is sent to you letting you know you’re back within your limits

• Your account will be active within the hour once the LDAP has queried the UAS


Integration with other systems

• NGS helpdesk is run using the commercial application Footprints

• Application process went through Footprints• Reviewers were used to Footprints• UAS sends applications directly to Footprints as well as

putting them in the database• System unchanged from reviewers point of view• Systems need to be synchronised

– Approved in Footprints so ticket can be closed– Approved in UAS so account is active

• Far from ideal situation


Using Oracle and Apex

• Oracle is robust, scalable and efficient• Oracle is used for hosting projects on the NGS

e.g. CPOSS data• Expertise, experience and Oracle support

already available to us• Apex allows web forms and reporting to be done

very quickly• Apex also allows graphs to be produced

dynamically and data queried in a variety of ways


Accessing your details

• Users wanted to know how much of their allocation they had used

• https://www.ngs.ac.uk/useraccountinfo.php • Certificate access to account details

– Not supported by Oracle Apex– Needed a workaround to take certificate details from

browser• Also provides ability to change contact details • Renewals can be done through their own

account




NGSUser

PK NGSUserID

SRBUserName OracleUserName DN Email Name AddressFK1 ProjectID AccountStatusStartDateFK2 AccountStatusTypeID Institution Telephone Funding InfoSource

NGSUserApplication

PK NGSUserApplicationID

FK1 NGSUserID ApplicationDate Case

ResourceApplication

PK ResourceApplicationID

FK1 ResourceTypeIDFK2 NGSUserApplicationID Limit

ResourceType

PK ResourceTypeID

Description

ResourceAllocation

PK ResourceAllocationID

FK2 ResourceTypeIDFK1 NGSUserID Limit StartDateFK3 ResourceApplicationID

ResourceAllocationHistory

PK ResourceAllocationHistoryID

FK1 ResourceTypeIDFK2 NGSUserID Limit StartDate EndDateFK3 ResourceApplicationID

AccountStatus

PK AccountStatusID

Description

NGSUsageRecord

PK NGSUsageRecordID

FK1 NGSUserIDFK2 ResourceTypeID UsageDate CumulativeValue

ResourcePolicing

PK ResourcePolicingID

FK2 NGSUserIDFK1 PolicingStateTypeIDFK3 ResourceTypeID StartDate

PolicingStatus

PK PolicingStatusID

Description

ResourcePolicingHistory

PK ResourcePolicingHistoryID

FK1 NGSUserIDFK3 PolicingStateTypeIDFK2 ResourceTypeID StartDate EndDate

NGSUserAccountStatusHistory

PK NGSUserAccountStatusHistoryID

FK1 NGSUserIDFK2 AccountStatusTypeID StartDate EndDate

Project

PK ProjectID

DescriptionFK1 NGSUserID

NGSUsageRecordHistory

PK NGSUsageRecordHistoryID

FK1 NGSUserID StartDate EndDateFK2 ResourceTypeID CumulativeValue

NGSApplicationReviewComments

NGSApplicationID CommentDate NGSReviewer ReviewCommentsFK1 NGSUserApplicationID

NGSUserCertificate

PK NGSUserCertificateID

FK1 NGSUserID DN

X509_authorisation

PK session_id

FK1 ngsuserid ipaddress time


Problems

• Limitation on field length restricted case entries and caused some problems when users wrote more than 2200 characters

• Oracle Apex doesn’t support X509 certificate access – our own workaround had to be implemented for the user to access their account details


Results

• Entered production in October 2006• Begun locking out users at 10,000% over quota • Gradually reduced to 100% of quota• 64 accounts have been locked• 73% have successfully reapplied• Over 150 new users have applied using the

system


Future Work

• The UAS is continuously developing• Peer-review process is being integrated into the

Oracle system• Incorporation of external system that creates the

SRB accounts and subscribes users to JISCmail.

• Inclusion of accounting of storage usage by users

• Extension to support Virtual Organisations


Summary

• Oracle and Apex offered the best solution for the NGS UAS

• Monitoring and policing accounts is all done automatically now

• Application process is handled automatically

• Room for expansion of the system


Further information

• Ask a question now!

• Talk to me after the presentation

• Visit the NGS stand (booth 20)

• Contact the NGS support centre [email protected]

• Visit the NGS website www.ngs.ac.uk

Documents

The National Grid Service User Accounting System Katie Weeks Science and Technology Facilities Council