Embed Size (px)
Citation preview
Frascati, October 9th, 2001 [email protected]
Accounting in DataGrid
Initial ArchitectureAlbert Werbrouck
Frascati, October 9, 2001
Frascati, October 9th, 2001 [email protected]
Two accounting approaches
Existing accounting systems follow basically two approaches:
• The passive approach: the accounting system only records the resource usage.
• The active approach: the accounting system actively uses information for workload management.
The second one seems to fit better our needs.
Frascati, October 9th, 2001 [email protected]
Advantages of the active approach
• A tighter integration with the Resource Broker (RB). This integration can help to optimize resource load.
• Prevention of unlimited usage and, consequently, resource saturation.
Frascati, October 9th, 2001 [email protected]
Computational economy
• In the context of an active accounting system, we chose an economical approach.
• In this economical view the resource usage is regulated by the exchange of a kind of ‘virtual money’ (Grid Credits).
• The user ‘pays’ the resource for its usage.
• The resource ‘earns’ from being used.
Frascati, October 9th, 2001 [email protected]
Price-setting policies
• Different price-setting mechanisms can be adopted in this framework.
• Between a flat model with constant price and one of pure supply & demand, one can envision a ‘pricing authority’ set up to help reaching and maintaining equilibrium.
• A good price-setting policy should avoid ‘virtual’ inflation, speculation and ‘price wars’.
Frascati, October 9th, 2001 [email protected]
Elements to be charged
• It is important to define which elements should be charged; a minimum set could be:
• Priority in a queue (if any)• CPU time• Memory usage• Disk and tape storage usage• Networking parameters
• Other elements could be considered too, but the definition of the full set is still an open issue.
Frascati, October 9th, 2001 [email protected]
Resource Value
• We define value as corresponding to the effective resource capabilities.
• The value is estimated via a properly defined benchmark suite.• The benchmark suite is run on the resource when
it joins the Grid.
• It must be impossible to falsify the benchmarks.
• The resource value should be published.
Frascati, October 9th, 2001 [email protected]
Resource Price
• The resource price is set according to its value by means of a mechanism, which is driven by the economical model adopted.
• This mechanism is the core of the economical approach. The final behavior of the whole system relies on this choice.
• The system is general enough to allow any pricing mechanism.
Frascati, October 9th, 2001 [email protected]
Cost algorithm
• The cost algorithm is responsible for computing the cost of a job. To do this it uses:• The information about the resource value.
• The chosen price setting mechanism.
• Information about the job resource usage obtained from the job monitoring service.
Frascati, October 9th, 2001 [email protected]
Working scheme
• Our system has to manage economical interactions between producers and consumers of a service.
• To address this problem the GSM mobile phone system uses a so-called “Home Location Register” to store info about the user (identity, “resource usage”, ...).
• We applied this concept to our accounting model.
Frascati, October 9th, 2001 [email protected]
The HLR working scheme
• Every user and resource belongs to an HLR.
• Normally the user and the resource belong to separate HLRs.
• In this context:• The user (belonging to HLR_A) submits a job.
• The resource (belonging to HLR_B) executes the job.
• HLR_A pays HLR_B for the resource usage.
Frascati, October 9th, 2001 [email protected]
Job Submission scheme
• Within the DataGrid Workload Management scheme:• User submits a job to Resource Broker specifying its
requirements using the Job Description Language.
• The RB finds a suitable resource.
• The resource HLR requests funds availability in the user HLR.
Frascati, October 9th, 2001 [email protected]
Job Submission scheme
• If the user has a satisfactory Grid Credit availability:• The resource runs the job and collects usage info
• Once completed, the resource sends to its HLR the info for computing the real cost of the job using the cost algorithm
• The Resource HLR sends the payment request to the user HLR
• The user HLR transfer credits for the job to the Resource HLR
• Else the job is rejected and sent back to the RB
Frascati, October 9th, 2001 [email protected]
HLR Scheme
HLR BHLR AJob costcomputation
Job submission
Job runAnd information gathering
Payment:From user account On HLR_A To resource account On HLR_B
Fund earned arePeriodically Redistributed to The HLR_B users.
PaymentRequest
Frascati, October 9th, 2001 [email protected]
HLR Authorization scheme
• The resource availability is related to funds availability.
• To obtain this we identified two different approaches:• Cost Forecast: the resource, using the information given by
the user with the Job Description Language, forecasts the cost of the job. The user HLR then reserves the funds foreseen to be paid at job completion.
• Fund consistency check: if the user has a positive balance he can run. Else not.
Frascati, October 9th, 2001 [email protected]
Local accounts on Grid resources
• Two opposite approaches:• Many-to-one: more than one user is mapped to a single local
user in the gridmap_file.
(Difficult tracking of the real user)
• One-to-one: every user is mapped to his/her own account.
(Not scalable)
• We think a good solution could be the ‘template accounts’ proposed within GGF.
Frascati, October 9th, 2001 [email protected]
Conclusions
• A preliminary document discussing more aspects of the problem in greater detail is available at: http://www.to.infn.it/grid/accounting/
• We developed a proof-of-principle prototype of the proposed architecture in preparation for the integration with the DataGrid project architecture. The feasibility results seem to be encouraging.
