41
Cover from Linux magazine, BOINC

Volunteer Computing using BOINC

Embed Size (px)

Citation preview

Page 1: Volunteer Computing using BOINC

Cover from Linux magazine, BOINC

Page 2: Volunteer Computing using BOINC

Volunteer computing using

Berkeley Open Infrastructure for Network Computing

http://bit.ly/boinc_srbiauTo get more references visit:

Pooyan Mehrparvar

Page 3: Volunteer Computing using BOINC

i. A brief introduction to volunteer computing & BOINC

ii. BOINC’s applicationsiii. BOINC’s architectureiv. How to join a popular BOINC projectv. How to set-up your own BOINC project

This prensentation covers:

Page 4: Volunteer Computing using BOINC

I – A brief Introduction to volunteer computing & BOINC

Why do we choose this topic? What is volunteer computing? What is BOINC?

Page 5: Volunteer Computing using BOINC
Page 6: Volunteer Computing using BOINC

What is volunteer computing

An established technology that enables ordinary citizens donate their computing resources to one or more "projects".

BOINC is the most widely-used middleware system

a client program runs on the volunteer's computer

BOINC was introduced by David P. Anderson

Page 7: Volunteer Computing using BOINC

BOINC is designed to support applications thathave large computation requirements, storagerequirements, or both. The main requirement ofthe application is that it be divisible into a largenumber (thousands or millions) of jobs that canbe done independently.

Goal:Use all the computers in the world, allthe time, to do worthwhile things

Page 8: Volunteer Computing using BOINC

What is BOINC

Page 9: Volunteer Computing using BOINC

Basic overview of BOINC jobs

Security: BOINC uses code signing to prevent distribution of malicious executables. Each project has a key pair for code signing and the private key kept on network-isolated machine

Virtualization as a solution for client’s security

Page 10: Volunteer Computing using BOINC

Project: • An entity that does distributed computing using BOINC. Application: • A project can include multiple applications• an application includes several programs (for different platforms) and a set of

workunits and results

Application versions: • A particular application program version,compiled for a particular platform

Workunit: a computation to be performed. • associated with an application, not with an application version

Result: • an instance of a computation, either unstarted, in progress, or completed. • Each result is associated with a workunit.

Basic concepts of BOINC:

Page 11: Volunteer Computing using BOINC
Page 12: Volunteer Computing using BOINC

When BOINC operates? Cycle scavenging BOINC uses computers’s idle cycles of CPU/GPU and other resources to

operate (by default) To avoid high battery consumption/3G charge the BOINC android app runs on

A/C power & WiFi connection (by default)

Client can run BOINC as:Screensaver (with fancy graphics)Window service (running in the background)Application (displaying results in tabular form)

Page 13: Volunteer Computing using BOINC

Biref history of BOINC SETI@home development (1998) Ifrastructure issues, United devices (2000) Uited devices falling out, SETI@home failed, hacked(2001) BOINC was introduced (2002) Climateprediction.net, SETI@home, LHC@home were implemented on BOINC (2004) Rosseta@home, Einstein@home, IBM World community Grid, Primegride (2005) BOINCstats (BAM!), Gridrepublic, BOINC wrapper (2006) GPU support, Multi-core apps (2008) BOINC packages for debian (2010) Apps in virtual machines, vbox wrapper (Server-side) (2011) Android, Condor/OSG collaboration, Git (2012) Virtualbox client (2013) Samsung app (Power sleep), HTC app (Power to give) (2014)

Page 14: Volunteer Computing using BOINC

BOINC supported platforms (until december 2014) Server: Unix-based operating sytems (Debian based linux is recommended) Microsoft Windows Vista or later (POSIX ready) Client:

Linux (x86/x64) Microsoft windows (x86/x64) MacOS (x86/x64) Playstation3 Android

Virtualization as a solution for cross-platform distributed systems (Virtualbox, Vmware, VirtualPC,...)

Page 15: Volunteer Computing using BOINC

Performance Benchmarking: FLOPS as a measure for computer performance in scientific fields (Floating point operations per second) Some computer systems are unable to to run FLOPS benchamrk MIPS/MOPS: suitable for database query, word processing,

spreadsheets, or to run multiple virtual operating systems

Page 16: Volunteer Computing using BOINC

Benchmarking records : Fastest supercomputer: China's Tianhe-2 running 33.86 petaflops (June 10, 2013)

BOINC: Active: 232,691 volunteers, 718,577 computers. 24-hour average: 8.308 petaFLOPS (December 19, 2014)

Page 17: Volunteer Computing using BOINC

Why to use GPU & Playstation General-purpose computing on graphics processing units (GPGPU) Playstation 4 (ATI) :

18 compute units, 64 cores per unit = 1,152 coresTheorical peak performance = 1.84 TFLOPS

Playstation 3’s CUDA (NVIDIA) is widley used in BOINC projects e.g Folding@home

Page 18: Volunteer Computing using BOINC

Computational Science : rosseta@home

Virtual campus supercomputing : univ. of Westminster in London

Desktop grids for business : Slicify project

Integration with HTCondor to allow Globus-based grids to run jobs for BOINC projects : Einstein@OSG

II – BOINC’s aplications

Page 19: Volunteer Computing using BOINC

What is computational Science?

Page 20: Volunteer Computing using BOINC

What is computational Science?

Computational science is concerned with constructing mathematical models and quantitative analysis techniques

and using computers to analyze and solve scientific problems.

Page 21: Volunteer Computing using BOINC

BOINC is used in:Physics, Astrophysics MathematicsBiology and medicince (Protein folding)Distributed sensingClimate modelingGames, 3D animation Rendering,...

Page 22: Volunteer Computing using BOINC

BOINC popular projects:Physics, Astrophysics: LHC@home, Einstein@home, SETI@homeMathematics: SZTAKI Desktop Grid, PrimegrideBiology and medicince: Folding@home, Rosseta@homeDistributed sensing: Quake Catcher NetworkClimate modeling: Climateprediction.netGames, 3D animation Rendering,... : Chess@home,

Enigma@home

Find more projects on:http://boinc.berkeley.edu/projects.php

Page 23: Volunteer Computing using BOINC

LHC@homeSETI@home

Page 24: Volunteer Computing using BOINC

What is protein fodling?

Amino Acids, Proteins, DNA Protein structure prediction / Modeling proteins

structure Rosseta@home helps to find a cure for:

Alzheimer’s diseaseHIVMalariaAnthraxHerpes simplex virus 1

Page 25: Volunteer Computing using BOINC

Rosetta@home’s screensaver

Page 26: Volunteer Computing using BOINC

Crowdsourcing Crowdsourcing is the process of getting work or funding,

usually online, from a crowd of people. Human vs Computer

Page 27: Volunteer Computing using BOINC

Foldit : Solve puzzles for science! From the creators of Rosseta@home (Washington Univ.) Similar to Rosetta@home, Foldit is aimed as a means of

discovering native protein structures faster, through a combination of crowdsourcing and distributed computing.

Game with a purpose In 2011 gamers helped to decipher the crystal structure of

M-PMV (virus causing AIDS in monkeys)

Page 28: Volunteer Computing using BOINC

Snapshot of Foldit game

Page 29: Volunteer Computing using BOINC

III – BOINC’s architecture

Challenges while using a BOINC system:

How can we depend on clients? Are they permanent?Will they reach the deadlines? Is the produced result valid?How can we trust a BOINC server?How can we trust a BOINC client?How to send jobs to various platforms?

Page 30: Volunteer Computing using BOINC

Some features of a BOINC system:

homogeneous redundancy (sending workunits only to computers of the same platform—e.g.: Win XP SP2 only.)

workunit trickling (sending information to the server before the workunit completes)

locality scheduling (sending workunits to computers that already have the necessary files and creating work on demand)

work distribution based on host parameters (workunits requiring 512 MB of RAM, for example, will only be sent to hosts having at least that much RAM, We send more jobs to multi-core CPUs/GPUs)

Double checking: (server sends the same workunit to at least two clients, then it compares the result by validation techniques (bitwise, sample trivial, fuzzy) or a customized validation technique

Job scheduling is needed in both server & client: priority between different tasks, reaching deadlines

To avoid cheating, credits are given when after the job is validated by the server

Page 31: Volunteer Computing using BOINC

BOINC server DAEMONS:In multitasking computer systems, a daemon is a computer program that runs as a background process, rather than being under the direct control of an interactive user

Generator Transitioner Feeder Scheduler Validator Assimilator File deleter

Page 32: Volunteer Computing using BOINC

BOINC server DAEMONS:

Page 33: Volunteer Computing using BOINC

IV – How to join a popular BOINC project

Regiter at Boincstats (BAM) Choose your project(s) Join/create a team (optional) Download the BOINC client (BOINC manager) Log into software Immediately the project(s) will be attached Tasks will be downloaded Completed tasks will be uploaded to the server You will gain due to your performance New job will be downloaded

Step by step manual is available at: http://bit.ly/boinc_srbiau

Page 34: Volunteer Computing using BOINC

BOINC client manager running four tasks (three projects)

Page 35: Volunteer Computing using BOINC

V – How to set-up your own BOINC project

Project configuration in a nutshell!:

Configure a LAMP (Linux, Apache server, Mysql, Php/Phyton) A virtualbox debian linux is available at BOINC’s website

Develope your own project (mostly done in C++ (GCC), Fortran)

Attach your project url in boincmanager (Client) “your server ip/projectname” (exp: http://192.168.1.80/testproject)

Monitor your project perfomance/administaration at “your server ip/projectname_ops” (exp: http://192.168.1.80/testproject_ops)

Step by step video is available at: http://bit.ly/boinc_srbiau

Page 36: Volunteer Computing using BOINC

BOINC server (left) + BOINC manager client (right) using virualization

Page 37: Volunteer Computing using BOINC

Conceived to be used by scientists, not IT professionals

BOINC offers tools for◦ Creating, starting, stopping and querying projects◦ Adding new applications, new platforms, …◦ Creating workunits ◦ Monitoring server performance

(All these procedures could be done by UNIX shell commands directly)

Server administration’s page:

Page 38: Volunteer Computing using BOINC
Page 39: Volunteer Computing using BOINC
Page 40: Volunteer Computing using BOINC

What you probably need to implement a BOINC ecosystem A precise evaluation before choosing BOINC as a solution, comparing

to other alternatives (JPPF, other cloud or distributed systems) Fundamental undrestaning of BOINC architecture including it’s

daemons, repositories, shell-script commands, etc Good knowledge and experience of linux environment (shell) C/C++, Fortran programming skills (GCC, VC++, etc) Server/network configuration Security issues

Page 41: Volunteer Computing using BOINC

BOINC’s weakness: The architecture is centeriliezed in contrast of most Grid systems Insufficient Interest from Computer Science Insufficient Interest from scientists Insufficient Interest from funding agencies According to official reports volunteers are not increasing Complexity of server and job submission

Insufficient documentations (obsolete docs) Rare experiment in web society (You can’t find your answers in BOINC

forums, Stack-overflow, etc), Risk of failure