View
34
Download
3
Category
Tags:
Preview:
DESCRIPTION
CernVM News. Predrag Buncic (CERN/PH-SFT). CernVM Project. Approved in 2007 as R&D activity under extra funding (“White Paper”) Portable Analysis Environment using Virtualization Technology (WP9) Led by SFT Group in CERN PH Department Started in January 2008 Funding for 4 (2+2) years - PowerPoint PPT Presentation
Citation preview
CernVM NewsPredrag Buncic (CERN/PH-SFT)
CernVM ProjectApproved in 2007 as R&D activity under extra funding (White Paper)Portable Analysis Environment using Virtualization Technology (WP9)Led by SFT Group in CERN PH DepartmentStarted in January 2008Funding for 4 (2+2) yearsFellow, student + 0.5 staff
MotivationEmerging technologiesMulti and many core CPUsMaking laptop/desktop ever more powerful and underutilisedHardware support for virtualizationMinimal performance penalty for application running in Virtual MachineVirtualization software is quickly becoming integral part of OS or given as a free add-onKVM, Xen, Vmware Player & Server, Suns VirtualBox
Old problemsMillions of lines of codeComplicated software installation/update/configuration procedure, different from experiment to experimentOnly tiny portion of it is really used at runtime in most of the casesOften incompatible or lagging behind OS version on desktop/laptop
Project GoalsUse virtualization and extra cores to get extra comfortHoping that this will come at acceptable cost in terms of performanceDecouple application lifecycle from evolution of system infrastructureClose to zero effort to install, maintain and keep up to date the experiment softwareReduce the cost of software development by reducing the number of compiler-platform combinations
How do we want to do that?Build a thin Virtual Software Appliance for use by LHC experiments
This appliance should provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Gridbe independent of physical software and hardware platform (Linux, Windows, MacOS)
Key Building BlocksrPath Linux 1 (www.rpath.org)Slim Linux OS binary compatible with RH/SLC4rAA - rPath Linux Appliance AgentWeb user interfaceXMLRPC API CVMFS - CernVM file systemRead only file system optimized for software distributionAggressive cachingOperational in offline modeFor as long as you stay within the cacherBulderBuild VM images for various platforms
CernVM componentgroup-cernvm(core packages)group-cernvm-devel(development tools)group-slc4(SLC4 compatibility libs)group-groups and extra packages required by experiment100 MBcompat-db4compat-opensslcompat-linstdc++slc3compat-libxml2compat-readlinecompat-tclcompat-tkgroup--desktopLightweight X working environemnt
Thin Software ApplianceCacheHTTPDSoftwareRepository10 GB1 GB0.1 GBLAN/WAN(HTTP)
CernVM File SystemAppOn same host:/opt/-> /mnt/cmfs/open(/opt/lcg)On File (Web) Server /opt/CacheKernelNFSLFSFUSECernVMFuseHTTP /opt/aliceRSYNCV.O. ADMINUSERS /opt/alice /opt/alice /opt/alice
Web UI
XML-RPC APIimport xmlrpclibimport osurl = 'http://' + user + ':' + password + '@' + host + ':8004/rAA/xmlrpc'server = xmlrpclib.ServerProxy(url)r = server.cernvm.Config.configUpdate('http://cern.ch/cernvm/config', 'ALICE', # (can be None,ATLAS, ALICE, LHCB, CMS) 'off', # (on/off - if on it will make cvmfs locally writeable) 'off', # (on/off - enable/disable grid UI) 'off') # (on/off - enable VO profile)print r["message"]##### Here we create initial user 'test' with the same password as Web admin####r = server.cernvm.User.userUpdate('test', password,password)print r["message"]##### Select grid UI version####r = server.cernvm.Config.configGridUIVersion("")print r["message"]
Developing CernVM
2-4 developersMonthly releasesBi-weekly meetings with experimentsGood collaboration with experiments ATLAS, ALICE, LHCb CMS is joining nowInitial feedback from the expertsFrom version 1.0 (end of October) it will be offered to wider publicThanks to Yves for constructive comments Hope to get soon more input from ALICE
CernVM Release TimelineJANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC-Preparation Release 0.5- Release 0.6On time!Release 1.0 Scheduled for end of OctoberKickoff Workshop- Release 0.8- Release 0.7- Release 0.91 (RC1)-Release 0.92 (RC2)
0.92 Release (RC2)Available for download fromhttp://rbuilder.cern.ch/project/cernvm-devel/releasesCan be run on Linux (KVM, Xen,VMware Player, VirtualBox)Windows(WMware Player, VirtualBox) Mac (Fusion, Parallels, VirtualBox) Release Notes http://cernvm.web.cern.ch/cernvm/index.cgi?page=ReleaseNotesHowTohttp://cernvm.web.cern.ch/cernvm/?page=HowTo
Appliance can be configured and used with ALICE, LHCb, ATLAS (and CMS) software frameworks
Content distribution networkCVMFSFile ServerCVMFSFile ServerCVMFSFile ServerHTTPProxy
More than a User Interface..BOINC Open-source software for volunteer computing and grid computingCernVM is now extended to include BOINC clientSummer student project in OpenLab based on LHC@HOME experience and CernVM image to run ATLAS simulation using BOINC infrastructurehttp://boinc.berkeley.edu/ Successfully run PanDA job in CernVMGlobus Virtual Workspace ServiceAn open source infrastructure for the deployment and management of virtual machinesGoogle Summer School (hosted at ANL) project to deploy a one-click, auto-configuring virtual Grid overlay for Alice/AliEnhttp://workspace.globus.org/Successfully created virtual AliEn site with one command
CernVM CoPilot0. Send host JDL (free disk space, free memory, available packages)1. Append framework specific information (e.g. CE name) and request a job3. Send input files and commands for execution(packages are already there) 5. Register output files4. When the job is done send back the output files (and the result of validation)2. Send user job JDL from Task QueeAliEn/DIRAC/PanDA
Mailing listscernvm-talk@cern.ch (Open list to discuss about design, user experience and related issues with the CernVM project)cernvm.support@cern.ch (End-user support for the CernVM project) Savannah PortalPlease submit bugs and feature requests to Savannah athttp://savannah.cern.ch/projects/cernvm
Web site(s) http://cernvm.cern.chhttp://rbuilder.cern.ch
Links
Extra slides
http://cernvm.cern.ch
http://rbuilder.cern.ch/project/cernvm-devel/
NOTE - initial settings are:
User Name: adminPassword: password
Please change Web admin password!
Select experiment for which VM will be configured
Status page
Select your top level group and migrate appliance configuration to get the latest version
Create local user account. First account created using this interface will be allowed to to executed sudo comand
Hands onpbuncic$ ssh -X pbuncic@192.168.197.129pbuncic@cernvm's password: Last login: Fri Jul 11 10:15:48 2008 from 192.168.197.1
[pbuncic@pb-d-128-141-224-235 ~]$ alienv listAliRoot/v4-13-Rev-02GEANT3/v1-9ROOT/v5-19-04
[pbuncic@cernvm ~]$ alienv use AliRoot/v4-13-Rev-02.---------------------------------------------------------------Great! Your compiler setup is ok!---------------------------------------------------------------******************************************************Attention: You don't have a valid grid proxy - doing grid-proxy-init******************************************************Your identity: /DC=ch/DC=cern/OU=.Enter GRID pass phrase for this identity:Creating proxy ....................................... DoneYour identity: pbuncicCreating token ..................................... DoneYour token is valid until: Sat Jul 12 10:18:52 2008
bash-3.00$ which aliroot/opt/alice/packages/AliRoot/v4-13-Rev-02/bin/tgt_linux/alirootbash-3.00$ aliroot -b ******************************************* * W E L C O M E to R O O T * * Version 5.19/04 14 May 2008 * * You are welcome to visit our Web site * * http://root.cern.ch * *******************************************ROOT 5.19/04 (tags/v5-19-04@23850, Jun 11 2008, 02:19:19 on linux)CINT/ROOT C/C++ Interpreter version 5.16.29, Jan 08, 2008Type ? for help. Commands must be C++ statements.Enclose multiple statements between { }.root [0]
Cache: ~200 MB
bash-3.00$ cernvm-cache -i
CONFIGURATION VARIABLES CERNVM_CONFIG_URL=http://cern.ch/cernvm/config CERNVM_ORGANISATION=ALICE CERNVM_PLATFORM=i686-pc-linux-gnu
CURRENT LOCATION Public Hostname: pb-d-128-141-224-235.cern.ch Domain: cern.ch Time zone: Europe/Zurich Location: Geneva
WEB FILE SYSTEM Server: cernvm-webfs.cern.ch Protocol: grow Cashe directory: /tmp/parrot.500 Cache size: 191 MB
*
Recommended