E516-ENGINEERINGCLOUDCOMPUTING

GregorvonLaszewski

(c)GregorvonLaszewski,2019

E516-ENGINEERINGCLOUDCOMPUTING

1PREFACE1.1Disclaimer☁�1.1.1Acknowledgment1.1.2Extensions

2SYLLABUS2.0.1SyllabusEngineeringCloudComputing☁�2.0.1.1Instructor2.0.1.2Audience2.0.1.3Coursesummary2.0.1.4References2.0.1.5Tools2.0.1.6CourseSchedule2.0.1.7Attendance2.0.1.8Assignments2.0.1.9GradeBreakdown2.0.1.10ProjectExamples2.0.1.11StatementonAcademicMisconduct

3OVERVIEW3.1Summary☁�3.2Communication☁�3.2.1ClassMaterial3.2.2Piazza3.2.3ClassResources3.2.4OnlineMeeting3.2.5Assignments3.2.6PostYourBio3.2.7EvolvingDocument3.2.8Books3.2.9YouGetCreditforImprovingtheBooks3.2.10OngoingAgenda

3.3QuickTips☁�3.3.1Requirements3.3.2TimeCommitment3.3.3CourseMaterialList

3.3.4Help3.3.5HowtoTakethisClass3.3.6Assignments3.3.6.1TechnologyReview3.3.6.2Project3.3.6.2.1License3.3.6.2.2ProjectReport3.3.6.2.3ProjectCode3.3.6.2.4ProjectData3.3.6.2.5WorkBreakdown3.3.6.2.6Bibliography3.3.6.2.7Reproducibility3.3.6.2.8ListofDeliverables3.3.6.2.9ExampleOutlineofaReport

3.3.7Submission3.3.8BonusProjects3.3.9Participation

4WEEKLYAGENDA4.1Week1:CourseIntroduction☁�4.1.1Assignments4.1.2OnlineMeetingRecording4.1.3ClassVideos4.1.4LabActivities4.1.4.1AccountCreation4.1.4.2Bio4.1.4.3Python3.7.44.1.4.4Questionnaire

4.2Week2:CloudDataCenters☁�4.2.1OnlineMeetingRecording4.2.2LectureMaterial4.2.3LabActivities4.2.3.1ReceiveyourHID4.2.3.2README.yml4.2.3.3Optional:PlagiarismCertificate

4.3Week3:CloudComputingArchitectures☁�4.3.1OnlineMeetingRecording4.3.2LectureMaterial

4.3.3LabActivities4.3.3.1UngradedActivities4.3.3.1.1Review:venvinpython34.3.3.1.2Review:Conda4.3.3.1.3Dicts4.3.3.1.4f-StringsinPython34.3.3.1.5Classes4.3.3.1.6PythonModules

4.3.3.2GradedActivities4.3.4Projectselection4.3.5WorkingAhead

4.4Week4:Openstack☁�4.4.1Video4.4.2LectureMaterial4.4.3Lab:OpenStack4.4.4Namingofvms4.4.5Horizon4.4.6CloudmeshOpenStackinterface4.4.7OpenStackCommandlineClient4.4.7.1InstallationofCloudmeshCloudBundle4.4.7.2SSH4.4.7.3Configuration4.4.7.4CloudmeshMongo4.4.7.5StartaVM

4.4.8WorkingwiththeVM4.4.9Exercise

4.5Week5:CloudVMComputeService☁�4.5.1Videos4.5.2LectureMaterial4.5.3Lab:VMComputeServices4.5.4Exercise

4.6Week6:REST☁�4.6.1LectureMaterial4.6.2LabActivities4.6.3GradedLabActivity

4.7Week7:MapReduce☁�4.7.1LectureMaterial

4.7.2LabActivity:HadoopInstallation4.7.2.1Exercises:

4.7.3LabActivity:PythonMapReduce4.7.4LabActivityMap/ReduceontheCloud4.7.4.1PythonWordCountinMapReduce4.7.4.2RunHadoopMapReduceinPython

4.7.5LabActivity:MapReduceonthecloud4.7.5.1HadoopClusterSetup4.7.5.2Configuration4.7.5.3OperatingtheHadoopCluster4.7.5.4RunHadoopMapReduceinPython

4.8Week8:ProjectReview☁�4.9Week9:Containers-Docker☁�4.9.1LectureMaterial4.9.2LabActivityDocker4.9.3LabActivityContainerRESTservice

4.10Week11:GraphQLandMessaging☁�4.10.1LectureMaterial4.10.2LabActivityGraphQL4.10.3OptionalLabActivityMQTT

4.11Week12:GO☁�4.11.1OptionalLabActivityGoRESTService

4.12WeekN-16☁�5APPENDIX5.1FAQs☁�5.1.1WheredoIfindthematerialforthisclass?5.1.2Wheretofindtheweeklyagenda?5.1.3Whendotheonlinemeetingstakeplace?5.1.4WherecanIfindtherecordingsoftheonlinemeetings?5.1.5CanIsearchintheePubreader?5.1.6Icouldnotfindananswertoanassignment?5.1.7HowmuchtimedoIneedtospendforeachquestion?5.1.8IdonotunderstandPiazza?5.1.9WheredoIfindthegithubdirectory?5.1.10WherecanIfindthehiddirectory?5.1.11WhyyoushouldnotreadthePiazzamailbutuseClickHereinstead?

5.2ePubReaders☁�

6REFERENCES

1PREFACE

SatNov2305:19:48EST2019☁�

1.1DISCLAIMER☁�ThisbookhasbeengeneratedwithCyberaideBookmanager.

Bookmanagerisatooltocreateapublicationfromanumberofsourcesontheinternet. It is especially useful to create customized books, lecture notes, orhandouts. Content is best integrated inmarkdown format as it is very fast toproducetheoutput.

Bookmanagerhasbeendevelopedbasedonourexperienceoverthelast3yearswith amore sophisticated approach.Bookmanager takes the lessons from thisapproachanddistributesatoolthatcaneasilybeusedbyothers.

The followingshieldsprovide some informationabout it.Feel free toclickonthem.

pypipypi v0.2.28v0.2.28 LicenseLicense Apache2.0Apache2.0 pythonpython 3.73.7 formatformat wheelwheel statusstatus stablestable buildbuild unknownunknown

1.1.1Acknowledgment

Ifyouusebookmanagertoproduceadocumentyoumustincludethefollowingacknowledgement.

“This document was produced with Cyberaide Bookmanagerdeveloped by Gregor von Laszewski available athttps://pypi.python.org/pypi/cyberaide-bookmanager. It is in theresponsibility of the user tomake sure an author acknowledgementsection is included in your document. Copyright verification ofcontentincludedinabookisresponsibilityofthebookeditor.”

Thebibtexentryis@Misc{www-cyberaide-bookmanager,

author={GregorvonLaszewski},

1.1.2Extensions

We are happy to discuss with you bugs, issues and ideas for enhancements.Pleaseusetheconvenientgithubissuesat

https://github.com/cyberaide/bookmanager/issues

Pleasedonotfilewithusissuesthatrelatetoaneditorsbook.Theywillprovideyouwiththeirownmechanismonhowtocorrecttheircontent.

title={{CyberaideBookManager}},

howpublished={pypi},

month=apr,

year=2019,

url={https://pypi.org/project/cyberaide-bookmanager/}

}

2SYLLABUS

2.0.1SyllabusEngineeringCloudComputing☁�

LearningObjectives

Getaquickoverviewwhattheclassisabout

2.0.1.1Instructor

GregorvonLaszewskilaszewski@gmail.comOfficehours:Byappointment

2.0.1.2Audience

Werecommendyouknowoneprogramminglanguage.Knowledgeofpythonisofadvantagebutnotrequired.Pythoniseasy.

2.0.1.3Coursesummary

This class will introduce you to state-of-the-art cloud computing concepts anengineering approaches. This will include virtual machines, containers andMap/Reduce.ThecoursewillhaveaLab inwhichyoucanpracticallyexploretheseconcepts.Youwillforexamplehavetheoptiontocreateacloudaspartofthiscourseandexplorecloudcomputingtoolsandframeworks.

2.0.1.4References

The course does not have required readings. We will provide the followingreferencesaspointerstowhatwewilldiscuss:

CloudComputing

LinuxPythonMarkdown

However,pleaserememberwewillselecttopicsandincludematerialthatisormaynotbecoveredinthesebooks.Thebookswillevolveduringthesemester.

2.0.1.5Tools

Youwillberequiredtohaveacomputertologintothecloud.WewillgiveyouaccesstoanOpenStackcloud.AccesstoAzure,AWS,Googleandotherscanbeachievedthroughtheirfreetier,

2.0.1.6CourseSchedule

Week References1 CourseIntroduction2 CloudDataCenters3 PythonforCloudComputing,StartofProjectSelection4 CloudArchitectures5 VirtualizationI-OpenStack6 VirtualizationII-AWS,Azure,Google7 MultiCloudEnvironment8 CloudTechnologyPresentation-ProjectReview9 Containers-Docker,Kubernetes10 MapReduce11 Messaging12 REST13 GO14 ProjectWork15 ProjectsDue16 ProjectImprovements

Foreachofthetopicsyouwillfindoneormorerelevantchaptersorsectionsinouronlinebook.

2.0.1.7Attendance

Attendanceaccountsfor10%ofyourfinalgrade. Ifyouneed toskipclassforanyreason,youneedtonotifytheinstructorandTAs

2.0.1.8Assignments

This course will not have exams. Instead, we have the following gradedassignmentcategories:

LabAssignments (pass/fail) are assignments that will be conducted on aweeklybasis.Theywillhelpyoumakingsureyounotonlyunderstandthematerialtheoretically,buttrythemout.

Cloud Technology Review and Examples (Graded): (This can besubstituted for more programming in your project). This is a documentabout a Cloud technology that is not yet included in our handbook tointroduceaninterestedpartytoit.Itshouldnotcontainadvertisementsbutbe a rational description of the technology with examples that you havetried yourself. You will have to give a non plagiarized presentation anddocumentaboutit.

Project Assignments (Graded): The most important part of the class forwhichyouwill beworking throughout the semester.Up to three studentscanworkinaproject.Incaseofgroupprojects,theprojectdeliverablesareincreased.

Theprojecthasthreesubmissionsthatarespreadthroughoutthesemester.Eachsubmissionbuildsonthepreviousoneandmodifiespreviousdocumentsintoaconsistentprojectreportanddocumentationforyourproject.

ProjectOutline

A description of what your project is about and how it relates to cloudcomputingandaddress:

Whatistheproblemyoutrytoaddress?Whatareyoudoingtoaddressthisproblem?Howareyouaddressingthisproblem?What is the architecture that adresses the problem that you willimplement?

CodeandDocumentationReview

YouwillbeaskedtohaveameetingwiththeTA’sand/orinstructortoshowcaseyour code andhave at least one reviewprior to your finalsubmission.

This will take usually take place through the Lab hours on regularbasis.

Afirstprojectdiscussionmusthavebeendoneatleastonceatmidtermtime.

FinalProjectSubmission

AllcodeanddocumentationmustbecheckedintoGitHubwellbeforethe semester is over. This allows us to give you feedback forimprovements.

Please note that the syllabus is subject to change. Students in this class oftencome from a wide variety of backgrounds and experiences. As such, theinstructorreservestherighttochangethecontentofthecoursetoaccommodatetheneedsandexpectationsofthestudents.

2.0.1.9GradeBreakdown

70%Project10%Chapter10%Assignments10%Particpation

Thereisnoexamormidterminthisclass,howevertherewillbeaprojectreviewatmidtermtime.

2.0.1.10ProjectExamples

https://cloudmesh.github.io/cloudmesh-manual/projects/index.html

2.0.1.11StatementonAcademicMisconduct

Students will be expected to uphold and maintain academic and professionalhonestyandintegrityasoutlinedintheCodeofStudentRights,Responsibilities,andConduct.Cases of academicmisconductwill be handled according to thestudentdisciplinaryproceduresdescribedinIU’spolicies.

3OVERVIEW

3.1SUMMARY☁�Pleasewatchthefollowingvideoandreadthischapter.

OverviewEngineeringCloudComputing2019

3.2COMMUNICATION☁�3.2.1ClassMaterial

Most class material is distributed as ePubs or PDF. ePub renders on manycomputers nicely, thus we recommend you use ePub instead of PDFs PleaseInstallanePubReaderifyoursystemdoesnothaveone.

macOS:BuildinuseBooksWindows10:Buildinuseedgeothers:Calibre

NaturallyyoucanreadtheePubsalsoonyourtabletcomputersifyouhaveonewithlotsofmemory.Pleasebeawarethatthedocumentswillbeupdatedweeklyandyouneedtodownloadthemaccordingly.Makesureyouclear thecacheifyourbrowserpreventsthedownload.

3.2.2Piazza

This classusespiazza for communication. It isyour responsibility to enroll inthePiazza for theclass.A link toourPiazza isprovided inCANVAS.Piazzaworksjustlikeaforuminwhichyoucanaskquestionsorpostnotes.Incaseofaquestion students and teachers can formulate answers.A thread system in thequestioncanbeusedtogathertheanswer.

https://piazza.com/iu/fall2019/e516fall19

YouwillgetagradeforparticipationintheclassdiscussionsonPiazza.PleasenotethatwedonotrecommendyougofromCANVAStopiazzaasCANVAShas a bad layout and in fact does not allowyou to see thePiazzaHyperlinks.InsteadgotoPiazzadirectly.

3.2.3ClassResources

Allclassmaterialwillbepostedat

https://piazza.com/iu/fall2019/e516fall19/resources

3.2.4OnlineMeeting

Ifyouareanonlinestudent:

ApollhasbeenpostedthatyoumustfilloutbeforeFridayofthefirstweekofthesemesterforpossiblemeetingtimes.

https://piazza.com/class/jzkfveoqwri3e4?cid=8

Thelinktothemeetinginformationisat

https://piazza.com/class/jzkfveoqwri3e4?cid=7

3.2.5Assignments

All Assignments are links in the piazza resource page as a simple link toCANVASintheresourcespage:

https://piazza.com/iu/fall2019/e516fall19/resources

3.2.6PostYourBio

ToassurethatyouhaveaccesstoPiazzaandcanposttoit,please,postaformalBio.

3.2.7EvolvingDocument

Thisdocumentisimprovedthroughoutthesemesterwithweeklyactivities.Werecommend that you regularly download a new version. Be aware that somebrowsers or ePub readersmay cache the previous version. Thusmake sure tocheckoutthenewestversion.

3.2.8Books

Thisclassdoesnotusebooksfrompublishersascloudcomputingisconstantlychangingandbythetimeyouplacethepurchaseorder,thebookmayalreadybeoutdated.Insteadwehavepreparedonlinebooksthatareconstantlyevolving.

Weliketoillustrateonasimpleexampleonwhathappenedtousinthepast.Weprepared with great detail information about Azure. However throughout thesemester thewayonhowto interactwith itwaschangeddramatically.Nowifyouwould have had a book, the entire chapter aboutAzurewould have beenoutdated.However,withinlessthanaweek,wewereabletoreplacethecontentsoeveryone inclasscanbenefit fromourupdatesveryquickly.Notonly that,becauseourmaterial isonlineyouwill likelyfindanupdatedversioninfutureevenafteryourclassiscompleted.Noprintedtextbookcanprovidethisservice.

Wehave the followingbooks inePubandPDFavailable.TheePubversion isourpreferedversion:

ListofBooks

TheweeklyLecturenotesarecontainedat

e516:CloudComputingEngineering

Thispagewillbringyoutothefollowingbooksthatweuseaspartofe516.ItwillalsoclearlyspecifywhichassignmentsyouneedtodoaspartoftheweeklyLabactivities.

CloudComputingLinuxPythonMarkdown

3.2.9YouGetCreditforImprovingtheBooks

PleasenotethatimprovingthedocumentinGitHubviapullrequestswillgetyoucredits.Thisisaneasyopportunityforyoutogetanexcellentgrade.

3.2.10OngoingAgenda

WewilladdhereonaweeklybasisthetopicsandLabsthatwewillcoverinthecomingweek(s).Pleasebeawarethatjustasinregularlecturesweaddtheagenda itemsand follow theSyllabus.Wemaychange theorderof the topicscoveredinthesyllabusasappropriate.

3.3QUICKTIPS☁�

3.3.1Requirements

We recommend that you know one programming language. Although mostactivities are done in Python, this programming language does not have to bePython as it can easily be learned throughout the semester.No background incloudcomputingisneeded.

3.3.2TimeCommitment

Anyclass at anuniversity requires a significant time commitment.Due to thedifferent background the students have it is difficult to predict the actual timeneeded.Onaveragewesee thatstudentsspend6hourson theclass if theydoparticipateonaweeklybasis.Studentswithlittleprogrammingexperiencespendupto12hours.

3.3.3CourseMaterialList

Course material will be distributed as ePubs, PDF, Video or Presentations.HoweveryouwillberequiredtoresearchsometopicsontheinternetasCloudComputing is a arapidly evolving field and we find the most up to dateinformationon theWeb.Wealsoaskyou tohelpusupdating theePuband touseadditionalresourcesasappropriate.

Thelistincludes:

CloudComputing[1]CloudTechnologies,GregorvonLaszewski[2]ProjectReportFormat,GregorvonLaszewski[3]IntroductiontoPython[4]LinuxforCloudComputing[5]ScientificWritingwithMarkdown[6]

3.3.4Help

If you take our class, please use piazza to ask for help. This is important asquestions may be answered by different Teaching Assistents (TAs) based onexpertise.Please,donotsende-mail to the instructors.TAsarenotallowedtoanswere-mailsendtothempersonally.

3.3.5HowtoTakethisClass

This class is attendedby studentswithgreatlydifferent backgrounds and timeschedules.Tobemost flexible andaddress all students there are twodifferentwaysonhowyoucantakethisclass.

Way 1: Free form. Here you simply look at the Syllabus table for thesemester and identify whatever section you feel like reading (when itbecomes available). However, make sure you conduct our weekly Labactivities.

Way2:Chronologicalorder.The lecturenotesareorderedchronological.Thusyoucanfollowourlecturealsoinchronologicalorder.

PleasenotethatwehavesetasidearecommendedsetofweeklyLabactivities.Theactivitiesarepass-failandwillbeintegratedinyourgrade.Youarecertainlyallowed to work ahead, but please be aware that based on feedback andobservationwemaymakemodificationstotheLabs.

Typically, Lab activities are supposed to be completedwithin oneweek as italertsusofproblemsyoumighthavethatwecanthanaddress.Thisassuresusthatweknowyouwillhavenoissueswithyourproject.

Labactivitieswillnotreceiveanycreditifyouarearesidentialstudentandtheactivityhasnotbeencompletedwithinoneweek.However,residentialstudentswillgettwoDelayaLabforOneWeekpassesthatyoucanapplytoanyoftheLabsandstillgetcredit.

IfyouareanonlinestudentwerecommendthatyoufinishtheLabsalsoinoneweek.However,youwillgeteightDelayaLabforOneWeekpassesthatyoucanapplytoanyoftheLabs.

Pleasenotethatifyouwouldneedtopostonealabfortwoweeks,youneedtouse twopasses.Labpassesexpireonemonthbefore the lastdayofclass.Youwillhavetocompletealllabsbythattime.NocreditwillbegivenatthattimeifthisdeadlineismissedforanydelayedLabsasTAsmustfocustheirattentiononprojectsupport.

Labpassesdonotapplytootherassignmentsandduedates.

3.3.6Assignments

BesidestheLab’s,wehaveonlytwomainassignmentsinthisclass.TheLab’swillprepareyoutowardsachievingtheseassignments.

3.3.6.1TechnologyReview

Studentsdoingprojectsrelatedtocloudmeshareexemptfromwritingatechnologyreview,butareexpectedtodomoreprogrammingandmakingsuretheprojecthasamanualandproperdocumentation.

Aspartofcloudengineeringyouwillbeexposedtoalargesetoftechnologies.Tosharpenyourskillsinanalyzingandevaluatingthesetechnologies,youwillbe asked to prepare a technology review that is being added to a classproceedings.

This includes a substantial non-plagiarized document that can be added as achaptertothelecturenotes.Thereviewmustbedoneonatopicthatisnotyetincludedinourbook.Thereviewwillnotincludeadvertisementstatementsformthose that have developed the technology, but will qualitatively describe thetechnologyandpotentiallycontrast it toother related technologies. Inaddition

youwillhavetodevelopanexampleshowcasinghowtousethetechnology.Theminimallengthofareviewisabout800words.

Anexampleforsuchasectionis

GraphQLintheCloudComputingbook

Alternativelyyoucanprepareseveraldifferentsmallersections(atleast5)thatmay not have an example in it but are more of descriptive nature. Samplesectionscontributedbystudentsinclude:

SectionMicrosoftNafikDataCenterintheDatacenterChapterSectionLambdaExpressionsinIntroductiontoPython

Itisexpectedfromyouthatyouselfidentifyasectionyourself,asthis shows competence in the area of cloud computing. If howeveryoudonotknowwhattoselect,youmustattendanonlinehourwithus inwhichwe identifya topicwith you.Technologies that arenotrepeatableduetoenormouscostorlicensingissuesneedtogetpriorapproval.

3.3.6.2Project

Theobjectiveoftheprojectistodefineaclearproblemstatementandcreateaframeworktoaddressthatproblemasitrelatestocloudcomputing.

Aprojectisthemajoractivitythatyouchoseaspartofyourclass.Thisincludesa project report* or manual and working project code. You will create asignificantnon-trivialprojectrelatedtocloudcomputingandcloudengineering.Up to three students can collaborate. The project could be built on top of aprevious project but must have significant additions or modifications. If aprevious project is used, a detailed discussion is to be held onwhat has beenimprovedandisdifferent.

In this class it is especially important to address the reproducibility of thedeployment.Atestandbenchmark,possiblyincludingadownloadabledataset,mustbeusedtoverifythecorrectnessofyourapproach.

3.3.6.2.1License

Allprojectsaredevelopedunderanopensourcelicense,suchastheApache2.0License.YouwillberequiredtoaddaLICENCE.txtfileanddescribehowothersoftware, if used, can be reused in your project. If your project uses differentlicenses,pleaseaddaREADME.mdfilethatdescribeswhichpackagesareusedandwhatlicensesthesepackageshave.

3.3.6.2.2ProjectReport

A project report is to be delivered and continuously improved throughout thesemester in GitHub. It includes not just the analysis of a topic, but a shortdescription of theArchitecture and code,withbenchmarks anddemonstrateduse. Obviously it is longer than a term paper and includes descriptions aboutreproducibility of the application. A README.md is provided that describeshow others can reproduce your project and run it. Remember that tables andfiguresdonotcounttowardsthepaperlength.Thefollowingminimallengthisrequired:

800words,onestudentintheproject1200words,twostudentsintheproject1400words,threestudentsintheproject

Projectswithmorestudentsareexpectedtodomoreprogramming.Thereportiswritten in markdown and checked into GitHub. The report will be madeavailableinaclassproceedings.AReportcouldbesubstitutedbyamanualandbenchmarks. In this case a one page extended abstract has to bewritten, thatincludesthelingtothemanual.

Forcertainprojects,therequirementofareportcanbewavedorissignificantlyreducedwhilereplacingitwithmoreprogrammingactivities.Thisincludes

AnyprojectthatenhancescloudmeshBuildingalargecloudclusterwithRaspberryPi’sAnyApplicationproject showcasingNISTbigdata reference architectureuse(thereisaharddeadlineoftheNISTprojectbyDec1st).

However you still have to do amanual andusage examples, benchmarks and

pytestsforthem.

3.3.6.2.3ProjectCode

You are expected to deliver a documented and reproducible code with unitteststhatallowsaTAtoreplicatetheprojectwithease.Incaseyouusevmorcontainer images, theymust be created from scratch locally andmay not beuploaded to services such as DockerHub. You can, however, reuse approvedvendor uploaded images such as from ubuntu or centos or other linuxdistributions. All code, scripts, and documentation must be uploaded togithub.comundertheclassspecificGitHubdirectory.

3.3.6.2.4ProjectData

DataistobehostedonIU’sGoogledrive,ifneeded.Ifyouhavelargerdata,itshould be downloaded from the internet. It is your responsibility to develop adownloadprogram.ThedatamustnotbestoredinGitHub.YouareexpectedtowriteapythonprogramthatdownloadsthedataeitherfromtheWeborIU’sdatastorage.

3.3.6.2.5WorkBreakdown

Thisisanappendixtothedocumentthatdescribesinabulletlistwhodidwhatintheproject.Ifyouareateamofonesuchasectionisnotneeded.Thissectioncomes after the references. It does not count towards the page length of thedocument. Itmust includesexplicitURLsto thegithistory thatdocuments thestatisticstodemonstratemorethanonestudenthasworkedontheproject.Ifyoucan not provide such a statistic or all check-ins have been made by a singlestudent,theprojecthasshownthattheyhavenotproperlyusedgit.Thus,pointswillbededucted fromtheproject.Furthermore, ifwedetect thatastudenthasnot contributed to a project,wemay invite the student to give a detailed oralpresentation of the project including a demonstration of the examples in realtime.

3.3.6.2.6Bibliography

All bibliography has to be provided in a BibTex file that must either bevalidated with jabref or with emacs. Please be advised doing references

correctly takes some time so you want to do this early and throughout thesemester.Whatwouldtakelessthan5minutesaweek,couldquicklyadduptomultiplehoursattheendofthesemester.PleasenotethatexportsofEndnoteorother bibliographymanagement tools donot lead to properly formattedbibtexfiles,despite theirclaimsofdoingso.Youwillhave toclean themupandwerecommend to do it the otherway around.Hence, the easiestway tomanageyour bibliography is with jabref or emacs. Make sure labels only includecharactersfrom[a-zA-Z0-9-].Usedashesandnotunderscoreandcolons(_,:)inthelabel.Yourlabelsmustbemeaningfulandunique.Wewilldeductpointsifyousubmitan invalidBibTex file toGitHub.Sopleasemake sureyour file isvalidated.Youcanevencreateyourowncheckswithtoolssuchasbiber.

Wewillteachyouwhattodoitiseasy.

3.3.6.2.7Reproducibility

Ingeneral,anyprojectmustbedeployablebytheTA.Ifittakeshourstodeployyour project, please talk to us before final submission.This should not be thecase.Also,ifittakes100steps,wearesureyoucanautomatethem…asyouarelikelydoingsomethingwrongorhavenotthoughtaboutcloudcomputingwherewetendtoautomatemostofthesteps.

Youhaveplentyof time toexecuteawonderfulprojectbutyouneed toworkconsistentlyonit.Startingoneweekbeforethedeadlinewillnotwork.

Thebestwaytoassurereproducibilityistousepytest.Wewilldiscusshowtodothatinclass.

Wewillteachyouwhattodoitiseasy.

3.3.6.2.8ListofDeliverables

In general your deliverables will include the following (We will address andexplaintheminaLab):

Providebenchmarks.TakeresultsinacloudservicesandyourlocalPC(ex:ChameleonCloud,echo kubernetes). Make sure your system can be created and deployed

basedonyourdocumentation.Each team member must provide a benchmark on their computer and acloudIaaS,wherethecloudisdifferentfromeachteammember.Werequireyoutowriteoneormorepytest’sthatdeploys,run,kill,view,clean that deploys your environment, runs application, kills it, views theresultandcleansupafterwards.Forpythonusearequirements.txtfileanddevelopasetup.pysoyourcodecanbeinstalledwithpipinstall.FordockeruseaDockerfile

Wewillteachyouwhattodo…

3.3.6.2.9ExampleOutlineofaReport

(Ifnotexempt)writeareportthattypicallyincludesthefollowingsections:

AbstractIntroductionDesign

ArchitectureImplementation

TechnologiesUsedResults

DeploymentBenchmarksApplicationBenchmarks

(Limitations)Conclusion(WorkBreakdown)

Your reportwillnot haveaFutureWork section as this implies that youwilldoworkinfutureandyourpaperisincomplete.Hencewewouldnotgradeit.Instead,youcanuseanoptional“Limitations”section.

Do communicate your status and add aWorkbreakdown section inwhichyououtlinewhichtasksneedtobedoneandbywhomincaseofagroupproject.Onceyouhavedoneatasksimplyincludemakerataskasfollows*[done,Gregor]Thiswasgregorstasktoshowcasehowtomarkit

Incaseyouhaveanexemptionfor theproject reportyouneed touse sphinxanddocumentyourcodeaspartofamanual.Wewillexplain thedetails inoneofourlabs.

3.3.7Submission

All submissions are conducted via GitHub if not otherwise instructed.TechnologyreviewsaretobeaddedtothebookGitHubrepowiththehelpofpullrequests.TheTA’swillworkwithyoutointegratethem.

AsweareworkingcontinuouslythroughoutthesemesteryoumustindicateyouractivitiesinaREADME.ymlfileinyourGitHubrepo.TheGitHubRepowewilldefineforyouinthefirst2weeksofthesemester.

AnexamplefortheREADME.yamlfileisshownnext

You MUST run yamllint on the README.yml file. YAML errors will cause pointdeductions. Any invalid yaml filewill result in point deductions. Please keepyouryamlfilevalidatanytime.Ourscriptsdependonit.TheyamlfilewillalsobeusedtocreatealistforTAstoreviewyourdeliverables.Ifit’snotintheyamlfileitwillnotbereviewed.Pleasenotethatitisnotsufficienttojustrunyamllint,but to compare your yaml file carefullywith the README.yml examples.Make sureyoudo the indentationwith2spaces,donotuse theTABcharacterandmakesure you use the list and attribute organization with proper dash placement.WorkwiththeTAsifyouhavedifficulties.Ifyoucopy,onlycopyfromtherawcontentinGitHub.Ifyouworkonmore

---

owner:

firstname:Gregor

lastname:vonLaszewski

hid:fa18-523-00

community:i523

semester:fa18

chapter:

-keyword:IoT

title:RoleofBigDatainIoT

url:https://github.com/cloudmesh-community/fa18-523-00/blob/master/chapter1/paper.md

group:fa18-523-00fa18-523-01

-keyword:Datacenter

title:GreenITdatacenteresintheUS

url:https://github.com/cloudmesh-community/fa18-523-00/blob/master/chapter2/paper.md

group:fa18-523-00

project:

-keyword:CloudCluster

title:RaspberryPICloudCluster

url:https://github.com/cloudmesh-community/fa18-523-00/tree/master/project-report

group:fa18-523-00fa18-523-01

code:TBD

Wewillteachyouwhattodoitiseasy.

3.3.8BonusProjects

Thisclasswillnothaveanybonusprojects,asalladditionalactivitiesshouldbeput inyourprojectorchapter/reviewcontribution.However,wewillrecognizeextraordinaryeffortsintheseactivities.

3.3.9Participation

Inadditiontotheseartifacts,therewillalsobeaparticipationcomponentinclassthatwillbedeterminedbasedonyourproductivecontributionstopiazzatohelpothers that have questions and contributions to the books to, for example,improve sections with spelling, grammer or content. We can see from theGitHubhistory ifyouconducted such improvements.Make sure that technicalcontributions, work on all OSes and are not just targeting a single OS if theimprovementisofgeneralnature(exceptionsapply).

4WEEKLYAGENDA

4.1WEEK1:COURSEINTRODUCTION☁�

4.1.1Assignments

Allgradedclassassignmentsarepostedat

https://iu.instructure.com/courses/1822529/assignments

Weprovideinthisdocumentallactivitieswedoineachweek

4.1.2OnlineMeetingRecording

The first online meeting for week 1 which will be repeated on Thursday 8-9:30pmhasbeenrecordedandisavailableat

OnlineMeetingRecordingforWeek1(1:29:39)

4.1.3ClassVideos

Pleasewatchthefollowingvideoand

OverviewEngineeringCloudComputing2019(21:05)

Next,reviewthefollowingchaptersinthebook

CloudComputing,GregorvonLaszewski,Ed.2019[1]:

Chapter:Preface,ePubReadersChapter:OverviewChapter:DefinitionofCloudComputing(anditsvideos)

4.1.4LabActivities

Summary:

PostyourprofessionalbiotopiazzaSetupyourcomputerCreatethecloudaccountsandfillouttheformCloudAccountsInstallpythononyourcomputer(python3.7.3or3.7.4)UpdatepipInstallanePubReader

macOS:BuildinuseBooksWindows10:Buildinuseedgeothers:Calibre

Nextweprovidesomemoredetails.IfyouhavequestionsaskonPiazza

4.1.4.1AccountCreation

Aspartoftheclassyouwillneedanumberofaccounts.Thisincludes:

piazza.com(usedforcommunication)github.com(usedforprojectandotherclassartifacts)chameleoncloud.org(freecloudaccount)

futuresystems.org(GPU&container)

Pleasejointhisproject

Afteryoucreatedtheaccounts,pleasefilloutthefollowingformsowecansetuptheclassaccountsandincaseofgithubwecreateyouaclassrepositoryforyou.

CloudAccounts

Optionalaccounts(applyonceyouneedthem,somearetimelimited):

google.com(optional)aws.com(optional)azure.com(optional)WatsonfromIBM(optional)

googleIaas(optional)

Wewillneveraskyouforyourpasswords

4.1.4.2Bio

Thisactivityservestwopurposes.First,ittellsuswecancommunicatewithyouwithin Piazza, and second, you can introduce yourself to others in piazza topotentiallybuildprojectorstudyteams.

4.1.4.3Python3.7.4

Please set up a computer on which you can do Python development. Werecommendthatyouusepythonfromhttp://python.org.Wewillnotprovideanysupportforconda,ascondahashundredsoflibrariesthatwearenotinterestedinusing. Also note that conda modifies your environment without telling you.Certainlyyoucanusevirtualbox,orcontainers incaseyoulike to isolateyourdevelopmentenvironmentfromyourothersystems.Pleaserememberthatcondahas significant disadvantages of often working with outdated libraries. Asdeveloperoffuturesoftwareyoumaywanttoavoidthis.Ifyouusepythonfrompython.orgwerecommendthatyouusevenv.

4.1.4.4Questionnaire

Inthisactivityyouwillbefillingoutaformwithinformationaboutyousowecan assess how to best integrate you in this class. It is not important that youknowanyofthetechnologiesweaskyouabout.

BackgroundQuestionnaire

4.2WEEK2:CLOUDDATACENTERS☁�

4.2.1OnlineMeetingRecording

TheonlinemeetingrecordingfromTueSep3,2019isavailablefromthislink:

OnlineMeetingRecordingforWeek2(48:38)

Thismeetinglastedactuallyhalfanhourlater,butincludedduplicatedquestionsthatweremoved.

4.2.2LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]:CloudComputing,GregorvonLaszewski,Ed.2019[1]:

Please read the section in Cloud Computing[1]: about Data Centers. Wedecidednottodoavideoasthematerialinthissectionfrequentlychangesandvideosbecametooquicklyoutdated.

Dothefollowingassignments:

E.Datacenter.2.aE.Datacenter.2.bE.Datacenter.3E.Datacenter.4E.Datacenter.5E.Datacenter.8

Optional:

E.Datacenter.9

4.2.3LabActivities

Makesureyouhavecompletedtheapplicationforthechameleoncloudaswewillusethatinourlabsoon.

Make sure you have Python set up. Read up on virtualized pythonenvironments as used in python 3.7.4. Its super simple.Understandwhatthecommand

Those using Windows, please find out ho to do it on Windows, provide adocumentvenv-windows.mdinyourHIDdirectorydescribingit.

Thoseusingconda/anaconda,please findoutho todo itwithconda/anaconda,provideadocumentvenv-conda.mdinyourHIDdirectorydescribingit.

MakesureyoucreateaseparatevenvforthisclasscalledENV3thatwewilluseforthisclassandshouldnotbeusedforotherclassesaswedonotwanttocreatesideeffects.Pleasealsobeawarethatattimesitmaybenecessarytodeleteyourpythonenvironmentincaseyoudosomethingwronganditwouldbeunwisetocombineallyourpythonactivitiesintoonepythoninstall.

Youwillnotreceiveanypointsforjustinstallingpython,withoutbeingabletocreateavirtualizedpythondeployment.

OurPythonbookandtheinternetwillhelp.ThisfeatureisbuildintoPython3!soitissupereasytouse.

4.2.3.1ReceiveyourHID

MakesureyouhavereceivedanHIDongithub.Lookitupat

https://github.com/cloudmesh-community

Makesuretoacceptthegithubinvitationandtryaddafile.YoucanuseeithertheGUIwaywithCreate newFile or if you are familiarwith github use thecommandlinetools.Nextweekwewillintroduceyoutoconvenienttoolssoyoucandevelopyourprogramsmoreeasilyingithub.

4.2.3.2README.yml

MakesuretheinformationinyourREADME.yamlfileisaccurate.Makesuretochange thevalue incommunityanduseyourclassnumber.Thiswillbeeither516or649.Pleasenotewhanweaskyouforyourhidnumberitistheentirehidnumbernotjustthelastthreedigits.

$cd~

$python-mvenvENV3

$sourceENV3/bin/activate

4.2.3.3Optional:PlagiarismCertificate

Whenwriting contributions that can be integrated into the classmaterial, it isimportant that it isnotplagiarized. It ismost important thatyounot just copycontentfromWebpages,butmakeappropriatemodificationsandprovidecredittowhereyoufoundthisinformation.

As you certainly will have learned form other classes what plagiarism is andhownottoplagiarizeyouwillprobablybefine.However,weoftenfindoneortwostudnetsinaclassthatdonotknowwhatitis.ThereforeweSTRONGLYRECOMMENDthatyoutaketheplagiarismcertificateofferedbyIU.Thishastheadvantagethatyoucanshowitinotherclassesandshowyouareinformed.PleasealsoreadthecodeofconductrulesatIU.

ForthisreasonweincludedthismaterialalsoinpartintheBook

ScientificWritingwithMarkdown

PleasereadthechapteraboutPlagiarism.

Please be aware that wemay conduct superior plagiarism tests. Our tests areevenbetterthan“Turn-it-in”whileweareabletoidentified“translations”fromotherlanguages,copiesoftextfromnon-publicexternaluniversityservers,andcompanyarchives that turn-it-indoesnotcheck. IUhasa strictpolicy thatwemust follow. Plagiarism/cheating could lead to expulsion from the university.TheargumentIdidnotknowwhatplagiarismisdoesnotcountaccordingtoIUrules.YoumustknowwhatitispriortoyoutakingacourseatIU.

Itisnotsubjectofthisclasstoteachyouwhatplagiarismis.Wedothisjusttoremindyoutoavoidanyuncertaintyaboutit.

4.3WEEK3:CLOUDCOMPUTINGARCHITECTURES☁�

---

owner:

firstname:Gregor

lastname:vonLaszewski

hid:hid-000

community:516

semester:fa19

StartofProjectSelection

4.3.1OnlineMeetingRecording

ThismeetingtookplaceSep10,2019,8-9pmEST

OnlineMeetingRecordingforWeek3(1:54:01)

4.3.2LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]CloudComputing,GregorvonLaszewski,Ed.2019[1]Introduction toPython forCloudComputing,Gregor vonLaszewski,Ed.2019[4]

ReadingAssignments:

1. Read in the book Cloud Computing [1] the chapter about CloudArchitectures.

2. Read in the book Introduction to Python for Cloud Computing [4] thechapteraboutCloudmesh

4.3.3LabActivities

Wehavethefollowinggoals

installpython3.7.4onyourcomputer.YOUcaneitherusepython.org,conda,orpyenv.Whichyouchoseisyourchoice.Forthisclassthepreferredwayofinstallingpythonisfrompython.org.

4.3.3.1UngradedActivities

Fortheungradedactivities,nosubmissionisneeded.

4.3.3.1.1Review:venvinpython3

Thislabisonlytobechosebythoseusingpython3frompython.orgwhichisourpreferredenvironment.

Pleaseunderstandthefollowingconcepts(nosubmissionneeded):

HowdoyouactivatethevirtualenvinyourOS?Howdoyoumodifyyour.bashrcfile?Whydoyouneedtousevenvforthisclass?

4.3.3.1.2Review:Conda

Thislabonlyhastobedonebythoseusinganaconda/conda.

What problems may you encounter when using anaconda as pythondeveloper?

Letusassumeyouuseanacondaonyourvirtualmachinesinthecloud.Letusassumeyoustart1000vmsallusinganaconda.WhatistheoverheadinwastedspaceonthesemachinesifyoujustwantedtouseassimpleregularpythonprogramontheseVMs?

How do you find out howmuch space is used by your program and itslibraries?

Howdoyouswitchbetweenanacondaandregularpython3.7.4

Whatisthedifferencebetweenconda,miniconda,anaconda?

Whydoyouwanttouseavirtualenvironmentevenforconda/anaconda?

4.3.3.1.3Dicts

Weliketoremindyouaboutdictsinpython.

Readupondictsandexperimentwiththem.

python-mvenv~/ENV3

Howdoyoumergethecontentoftwodicts?

4.3.3.1.4f-StringsinPython3

Python3providessomeverynicewayofusingvariablenamesaspartofstringmanipulations.

Locate the PEP 498 and study it:https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep498

Tryoutthefollowing

Inoneoftheexampleslocals()isused.

Whatdoeslocalsdo?Whatdoes**intheformatstatementdo?

4.3.3.1.5Classes

Thiscanbecompletedatalatertime

Whatisselfinclasses?Whydoesselfneedstobeusedinregularmethoddefinitionsinclasses?WhatcanIdowithinitandwhyisitused?Whatisclsand@classmethods?Whywouldoneuse@statusmethod?

4.3.3.1.6PythonModules

Thiscanbecompletedatalatertime

Whatisasetup.pyfile

test="Gregor"

msg=f"Thisisatest{test}"

print(msg)

deff(test):

msg=f"Thisisatest{test}".format(**locals())

print(msg)

fromcloudmesh.common.debugimportVERBOSE

d={"test":"Gregor"}

VERBOSE(d)

Whatisthedifferencebetweenpipinstall.andpipinstall-e.HowdoIuninstallapythonmodule?Mypythonvirtualenvironmentisbroken,Whatdoyoudonow?

WewillbediscussingthesequestionintheLabs(online/andresidential).

4.3.3.2GradedActivities

In the book Introduction to Python for Cloud Computing [4] please do thefollowingassignments

E.Cloudmesh.Common.1E.Cloudmesh.Common.2E.Cloudmesh.Common.3E.Cloudmesh.Common.4E.Cloudmesh.Common.5E.Cloudmesh.Shell.1E.Cloudmesh.Shell.2E.Cloudmesh.Shell.3

4.3.4Projectselection

Youwillbeselectingacloudrelatedprojectoverthenext2weeksthatyouwillbedevelopinguntiltheendofthesemester.Theprojectmusthavethefollowingrequirements:

Programming should be done using python, if another programminglanguageisused,pleasecontactusandjustifytheuse.Youwillalsohavetodopartofyourprogrammingto,forexample,coordinatedthedeploymentorthebenchmarklikelyinpython.

TheprojectmustuseOpenAPI3.0todefineaRESTservice.Wewillteachyouhowtodothatinafutureactivity.

The projectmust use conexion to automatically generate the rest service(pleasedonotuseswaggercodegen).Wewillteachyouhowtodothatinafutureactivity.

Theprojectmustuseatleast1+ncloudsforeachteammember.Wherenisthenumberofteammemberswithmaximum3teammembers.Wewillteachyouhowtodothatinafutureactivity.

The codemust usepytests.Wewill teachyouhow todo that in a futureactivity.

Thecodemustusecmssysgenerate.This isoneof the labassignments for thisweek.

The code must use the cloudmeshbenchmark/stopwatch class. This is one of the labassignmentsforthisweek.

If anAI service is usedwe recommend to use scikit-learn orKearos andcontrast it with AI services offered by cloud providers. This will bediscussedinupcominglectures.

Youmustprovideareportwithmeaningfulbenchmarks.

Youwillhavetowriteareport.Wewilldiscusswithyouhowtodothatinafuturelecture.

Pleasenote that the above requirements alsohold true if youuse technologiessuchasHadoop,Spark,Kubernetes,AWS,Azure,Google,OpenStack,…

Certainprojectswillhavecustomdeliverablesthatwewillrefinewithyouonceyouhavechosensuchaproject.Forexample,ifyouweretochoseacloudmeshprojectyouwillbeaskedtodevelopamanualinsteadofareport.

Apreliminarylistofprojectsisavailableat.

https://cloudmesh.github.io/cloudmesh-manual/projects/

Wewilladdadditionalprojectideasoncethebecomeavailable.Pleasenotethatweconsiderthecloudmeshrelatedprojectseasyasweintroduceyougraduallyinallaspectsaspartoftheclasstodeliverasuccessfulproject.

Those wanting to chose the Raspberry PI Cluster or the Robot Boat, pleasecontact instructorsviapiazza.Wewant tosetupameeting inMESHtobetter

discussthisproject.

When it comes to the scopeof theproject, rememberaproject takesup to12weekstobecompleted.Itisnotallowedtojustsearchongithuboranotherbookand“replicate”aprojectdonebysomeoneelse.Yourprojectmusthaveanovelcomponent. Please note this class is called Engineering Cloud Computingwemust see clearly an aspect that you engineer the cloud, e.g. the setup of areproducible cloud environment is mandatory. Please ask questions andunderstand this. While you are able to use all services, all images must becreatedfromscratchandwemustbeabletoreproducethem.Wewilldeclineallprojects that point us to images that you askus todownload anduploadedbyyouongithub,dockerhuborsimilar. Insteadyoumustprovideuswithscripts,dockerfiles,makefilesorsimilarthatcreatetheimages.Youareallowedtouseimageshostedbymajorvendorssuchasanubuntu19.04imageoronchameleoncloudtheimagesstartingwithcc-*andsoon…

4.3.5WorkingAhead

Obviouslywewillbeintroducingyoutosomemoreadvancedconceptsthatarenotyetfinalizedinthebooks.Youcancertainlyusedocumentsformtheinternettolearnaboutsuchconcepts.

ConceptswewillneedarelistedintheSyllabus.

Onthepythonside,wewillintroduceyouto

pytestgithubAPIasanexampleforaRESTserviceLibcloudAzurepythonAPIAWSboto…

On the cloudmesh side (still developed by current set of students we willintroiduceyouto

cloudmesh.yamlaspreliminarydocumentedinthecloudmeshmanualcloudmeshcloudbundle(untested)

cloudmeshstoragebundle(untested)

4.4WEEK4:OPENSTACK☁�4.4.1Video

Incaseyouhavenotyetcreatedansshkey,thefollowingvideoisuseful.Thiscan be replicated on anyLinux,macOS andWindows10machine, in case ofWindows10usegitbash

SSHkeygen(4:07)

A Video briefly summerizing an introduction to cloudmesh for multi cloudenvironments including pointedts to projects related to virtual directories,computeservices,andvirtualclusters

CloudmeshVersion4(44:01)

AvideoonhowtostartanloginintoavirtualmachineonHorizonispresentednext:

OpenStackHorizon(10:49)

Avideohowtostartvmswithcloudmeshisavailablehere:

Cloudmeshcmsvmboot(15:07)

Anumberofvideosonexplainingtheinternalsofcloudmeshisavalablehere:

The first video introduces you to cloudmesh common a library to executecommandsonyourosandinteractwithit

cloudmesh-common(13:40)

Thesecondvideoexplainsdetailsaboutcloudmeshcommputeandalsotellsyouaboutwhatisexpectedfortheprojecttobedeveloped.Howeverthevideodoes

notyetexplainhowtodoavirtualclusterfromtheProviders,whichisonegoalforthissemester.

cloudmesh-cloud(15:09)

Thethirdvideotellsyouaboutthecloudmesh-storageprojectthatissupposedtobedonintheproject,thisincludes

bettertestscompletionofadditionalprovidersusingofProcesspoolandmongodbformanagingfilestobecopied

thecreationof aRESTservice that includes thepool to allowcopy fromonecloudtoanother.

cloudmesh-storage(8:03)

4.4.2LectureMaterial

ABookforChameleonCloudisavailable

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]CloudComputing,GregorvonLaszewski,Ed.2019[1]Introduction toPython forCloudComputing,Gregor vonLaszewski,Ed.2019[4]

4.4.3Lab:OpenStack

ThisweekwewillbelearninghowtomanagevirtualmachinesonOpenStack.Youarerequested toexplore theGUIinterfacewhich iscalledhorizonsoyoucan.verifyyouractivitieseasilyincaseyouhaveissueswiththecommandlinetools. However, our main goal will be that you use command line tools tointeractwithChameleoncloud.

WhateveryoudoafteryouaredonewiththeVMsyouneedtoterminatethemsoyoudonotunnecessarilywastecomputetime.

4.4.4Namingofvms

YourHIDisoftheformfa16-516-NNNorsimilarforotherclasses.Pleasenotethatthenumberisuniqueacrossclasses.Thisidentifiesyouandifyoustartavminasharedspacesuchaschameleonwecanuseittoidentifypeopleandnotifythemeasily.

ThuspleasusethefollowingnamingschemeNNN-firstname-i

wheretopicisatopicforthevmsuchaswebserverandiisanumbersuchas1asyoumaystartmultiplevms.

Pleaseneverstartmorethan3vmswithoutconsultationwithGregoraswewillrunoutofnodehoursfortheclassifwedoso.

4.4.5Horizon

TheinformationonhowtouseHorizonisavailableintheChameleonbook.

AvideoofthemeetingonTuesday17Sepwillbemadeavailable

4.4.6CloudmeshOpenStackinterface

TheinformationonhowtouseitisavailableintheChameleonbook.

AvideoofthemeetingonTuesday17Sepwillbemadeavailable

4.4.7OpenStackCommandlineClient

TheinformationonhowtouseitisavailableintheChameleonbook.

4.4.7.1InstallationofCloudmeshCloudBundle

⚠� Do these only after you have completed the cloudmesh shell relatedassignmentfromlastweek.

Compute:

Thenext isoptionalandonly for those thatchoseacloudmeshstorage relatedproject.Theinstallationofthestoragebundleincludestheinstallationofthecomputebundle.

Toseetheavailablecommandstype

4.4.7.2SSH

Makesureyouhaveapasswordprotectedsshkey

4.4.7.3Configuration

1. Change the username and password for the chameleon cloud in~/.cloudmesh/cloudmesh.yaml

2. Changethepasswordinthemongodbsection

3. Changetheinformationintheprofilesection

4.4.7.4CloudmeshMongo

SettingupcloudmeshMongoisdiscussedinthecloudmeshmanual.Wesuggestyoudotheonediscussedinthedistributionsectionforyoursystem.Itisactuallybuiltintocloudmesh,butbeforeyoudoit,wesuggestyoubackupyormachine.

$cdcm

$cloudmesh-installergitclonecloud

$cloudmesh-installerinstallcloud-e

$cdcm

$cloudmesh-installergitclonestorage

$cloudmesh-installerinstallstorage-e

$cmshelp

$ssh-keygen

$cmsadminmongoinstall

Onceyousetitupusecmsinitwhichwipesthedbandshouldonlybeusedonce

afterthatalwaysuse

4.4.7.5StartaVM

OpenStackcouldbeover-utilizedandthataVMmaynotstartbeforeatimeout.Gregorobserved70%successrate.

Task:wheneveryoustartavm,pleasekeeparecordifitstartedornot

dothisinafileonyourGitHubcalledchameleon-success.md

makesurethefailuresarenotrecordedbasedonprogrammingerrorsorwrongparameters.

Usethecommands

⚠�switchondebuggingandtrace

⚠�Explainthedifferencebetween--refreshandnotusingit.

4.4.8WorkingwiththeVM

Logintothevmwith

4.4.9Exercise

$cmsinit

$cmsstart

$cmsstop

success:7

failure:3

cmssetcloud=chameleon

cmsimagelist--refresh

cmsflavorlist--refresh

cmsvmboot

cmsimagelist--refresh

cmsssh

1. StartavmwithHorizon,login,andterminateit2. StartavmwithCloudmesh,login,andterminateit3. Userobo3TorasimilarprogramtobrowsintheCloudmeshMongoDB

4.5WEEK5:CLOUDVMCOMPUTESERVICE☁�4.5.1Videos

Watchthevideosfromweek4relatedtoCloudmesh

4.5.2LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]CloudComputing,GregorvonLaszewski,Ed.2019[1]

PleasereadthechapterinthebookCloudComputing[1]about

HypervisorIaaS,pickonecloudyouliketogetfamiliarewithandfocusonthat

1. Donotreadthesectionsmarkedwith

Construction

2. Do not use PythonLibCloud we want you to find the vendors pythonlibraries whcih are typically superior and have better access to theinformation related to IaaS for theprovider.Thiswillmakeyourprojectsmucheasierinthelongrun.

4.5.3Lab:VMComputeServices

4.5.4Exercise

1. FindacloudyoulikeandidentifythenativePythonAPI(butnotlibcloud,incaseofopenstackusetheopenstacksdkwhichisnewandnotnovaandtheotherapi’s).

2. DemonstrateaPythonprogramtolistimages,flavorsandvirtualmachines.You can create a vm via the GUI on your cloud, name it with yourfirstname.

3. Whenitcomestocredentialmanagement,pleaseusecloudmesh.yamlandConfig().Learnhowtodothat.UnderNOCIRCUMSTANCESpostyourpasswords or add files to GitHub that include your passwords or othercredentials.

4. Developapythonprogramthataddskeysandsecuritygroups(difficult).5. Showcasethatyoucansshintothevm(difficult)

This exercise can be done openly in class and everyone can share code witheveryone,aslongasyouacknowledgethestudentwithnameandhid.

4.6WEEK6:REST☁�4.6.1LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]CloudComputing,GregorvonLaszewski,Ed.2019[1]ChameleonCloudIntroduction toPython forCloudComputing,Gregor vonLaszewski,Ed.2019[4]

Go through Chapter REST of the Cloud Computing [1] book. This chapterincludes,

AnintroductiontoRESTAnoverviewofframeworksusingRESTAnoverviewofOpenAPIandhowitrelatestoRESTAn introduction of using GitHub which is also a REST service from aPythonAPI.

ThechapteralsincludesasectiononOpenAPI2andEvewhichishoweverforthisclassnotrelevant.

4.6.2LabActivities

Implement theREST service explained in theSectionOpenAPI REST ServiceviaIntrospectioninyourlocalmachine

Completetheexercises:

OpenAPI.Conexion.1OpenAPI.Conexion.2OpenAPI.Conexion.3OpenAPI.Conexion.4OpenAPI.Conexion.5

Completetheexercises:

E.OpenAPI.1E.OpenAPI.2E.OpenAPI.3E.OpenAPI.4E.OpenAPI.5

4.6.3GradedLabActivity

In this lab activity you will be adding more functionality to the REST cpuexampleanddeployingtheserviceintheChameleonCloud.

Youmayneed to installpy-cpuinfo library tohelpyoucollect information fortheserviceimplementation.

1. ChangethecpuGETmethod,toworkinaoperatingsysteminvariantway(i.e. use python libraries to determine theCPU name, rather than systemcalls)

2. Playaroundwithpy-cpuinfo library inUbuntuenvironment just toget anidea about the information you could retrieve. I suggest you spawn an

Ubuntu18.04containeronyourlocalmachineforthis.

3. AddaGETmethod togetcachesizeof theCPU.URLparameter{level}shouldspecifythecachelevel.

GEThttp://localhost:8080/cloudmesh/cpu_cache/{level}

level=‘l3’forl3cachesizelevel=‘l2’forl2cachesize

Thereturnshouldbeadictionaryasfollows.

4. Ifcachelevelisnotspecified,thefollowingdictionaryshouldbereturned.

5. Addthesemethodstothecpu.yamldefinition.

6. Deploytheservicelocallyandtesttheservicesusingswagger-UIandcurl.

7. Createam1.smallinstanceinChameleonCloudwithubuntu18.04image.Deploy your new web service in the cloud instance. You should usecloudmeshcommandstostarttheseVMs.

8. Use curl to call thewebservice in the cloud. (You are NOT expected toexposetheservicethroughthepublicIPaddress)

9. Capture the outputs for each service paths in ameaningfulway (images,screenshots, etc) and compile a Markdown file. The Markdown shouldinclude, the new cpu.yaml file, server.py, and cpu.py files together withoutputs. Furthermore, you should include the cloudmesh commands thatyou have used in the process and youmay include improvements to theservicesuchashandlingmalformedrequests,etc.

10. IntheMarkdownfile,discusswhatarethewaysyoucouldaddsecurityfor

{

"l3":"8448KB"

}

{

"caches":{

"l3":"8448KB",

"l2":"1024KB"

}

}

theseservices(YoudoNOTneedtoimplementthis).

4.7WEEK7:MAPREDUCE☁�4.7.1LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]:CloudComputing,GregorvonLaszewski,Ed.2019[1]:

PleasereadtheHadoopsectionsinCloudComputing[1]

WatchtheseonlinevideosforHadoopMapReduce:

MapReduce,Hadoop,andSpark(19:02)HadoopA

Hadoop13:19HadoopB

Hadoop12:57HadoopC

Hadoop15:14HadoopD

Note:AllMapReduce/HadoopLabactivitiesareoptoional

4.7.2LabActivity:HadoopInstallation

Thefollowingexerciseswillguideyouto installHadooponasinglenodeandthen runaMapReduce job inPython.Please figureout the requiredcommandlines. These commands are available in the book sections: CloudComputing,GregorvonLaszewski,Ed.2019[1]

4.7.2.1Exercises:

InstallprerequisitesoftwareInstallJavaInstallSSH

InstallMavenConfigureprogrammingenvironments

JAVA_HOME

core-site.xml

hdfs-site.xml

SShlocalhostwithoutapassphraseStartHadoopHDFS

FormatHDFSfilesystemStartNameNodeandDataNodeviacommandlinesCheck NameNode status via Web Interface: NameNode -http://localhost:9870/

StartHadoopYARNConfigure

mapred-site.xml

yarn-site.xml

StartYARNviacommandlineCheck YARN status via Web Interface: ResourceManager -http://localhost:8088/

After finishing all these steps, you are good to move forward toMapReduceprogramming.

4.7.3LabActivity:PythonMapReduce

SeeinthemapSectionofthePythonbook:

Thebasicsyntaxofthemapfunctionexpectsafunctionobjectandanynumberof iterables like a list or dictionary. It executes the function object for eachelement in the sequence and returns a list of the elements modified by thefunctionobject.

4.7.4LabActivityMap/ReduceontheCloud

defmultiply(x):

returnx*2

map(multiply2,[2,4,6,8]

#Output[4,8,12,16]

4.7.4.1PythonWordCountinMapReduce

ThiscodeisusedasMapper.

ThiscodeisusedasReducer.

PleasenotethesecodesnippetsareusedtodemonstratetheideaofMapperandReducerinPython.Itleavessomebugsbyintentions.PleasedebugtheabovecodeorwriteyourownversionofMapReduce.

4.7.4.2RunHadoopMapReduceinPython

4.7.5LabActivity:MapReduceonthecloud

4.7.5.1HadoopClusterSetup

#!/usr/bin/envpython

importsys

forlineinsys.stdin:

line=line.strip()

words=line.split()

forwordinwords:

print('%s\t%s'%(word,1))

#!/usr/bin/envpython

fromoperatorimportitemgetter

importsys

current_word=None

current_count=0

word=None

forlineinsys.stdin:

line=line.strip()

word,count=line.split('\t',1)

try:

count=int(count)

exceptValueError:

continue

ifcurrent_word==word:

current_count+=count

else:

ifcurrent_word:

print('%s\t%s'%(current_word,current_count))

current_count=count

current_word=word

ifcurrent_word==word:

print('%s\t%s'%(current_word,current_count))

bin/hadoopjar<path_to_hadoop_libs>/hadoop-*streaming*.jar\

-file/<path_to_mapper>/mapper.py\

-mapper/<path_to_mapper>/mapper.py\

-file/<path_to_reducer>/reducer.py\

-reducer/<path_to_reducer>/reducer.py\

-input<input_file_path>\

-output<output_file_path>

Hadoop’sJavaconfiguration isdrivenby two typesof importantconfigurationfiles:

Read-onlydefaultconfiguration-core-default.xml,hdfs-default.xml,yarn-default.xmlandmapred-default.xml.Site-specific configuration - etc/hadoop/core-site.xml, etc/hadoop/hdfs-site.xml,etc/hadoop/yarn-site.xmlandetc/hadoop/mapred-site.xml.

Additionally,youcancontrol theHadoopscriptsfoundin thebin/directoryofthedistribution,bysettingsite-specificvaluesviatheetc/hadoop/hadoop-env.shandetc/hadoop/yarn-env.sh.

ToconfiguretheHadoopclusteryouwillneedtoconfiguretheenvironmentinwhichtheHadoopdaemonsexecuteaswellastheconfigurationparametersfortheHadoopdaemons.

HDFS daemons areNameNode, SecondaryNameNode, andDataNode.YARNdaemons are ResourceManager, NodeManager, and WebAppProxy. IfMapReduceistobeused,thentheMapReduceJobHistoryServerwillalsoberunning.Forlargeinstallations,thesearegenerallyrunningonseparatehosts.

4.7.5.2Configuration

ConfigureallHadoopdaemons:

NameNode:HDFS_NAMENODE_OPTSDataNode:HDFS_DATANODE_OPTSSecondaryNameNode:HDFS_SECONDARYNAMENODE_OPTSResourceManager:YARN_RESOURCEMANAGER_OPTSNodeManager:YARN_NODEMANAGER_OPTSWebAppProxy:YARN_PROXYSERVER_OPTSMapReduceJobHistoryServer:MAPRED_HISTORYSERVER_OPTS

Configure Namenode to use parallelGC and a 4GB Java Heap, the followingstatementshouldbeaddedinhadoop-env.sh:exportHDFS_NAMENODE_OPTS="-XX:+UseParallelGC-Xmx4g"

4.7.5.3OperatingtheHadoopCluster

To start a Hadoop cluster you will need to start both the HDFS and YARNcluster.ThefirsttimeyoubringupHDFS,itmustbeformatted.Formatanewdistributedfilesystemashdfs.

FormatNameNode(onadedicatenodeinthecluster)StartNameNode(onadedicatenodeinthecluster)StartDataNode(oneachnodeofthecluster)StartResourceManager(onadedicatenodeinthecluster)StartNodeManager(oneachnodeofthecluster)

4.7.5.4RunHadoopMapReduceinPython

4.8WEEK8:PROJECTREVIEW☁�Makesuretohavemademajorprogressinyourproject

Documentitin

githubbyupdatingreport.bibgithubbycheckinginyourcurrentcode

4.9WEEK9:CONTAINERS-DOCKER☁�

4.9.1LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]:CloudComputing,GregorvonLaszewski,Ed.2019[1]:

$bin/hadoopjar<path_to_hadoop_libs>/hadoop-*streaming*.jar\

-file/<path_to_mapper>/mapper.py\

-mapper/<path_to_mapper>/mapper.py\

-file/<path_to_reducer>/reducer.py\

-reducer/<path_to_reducer>/reducer.py\

-input<input_file_path>\

-output<output_file_path>

PleasereadthesectioninCloudComputing[1]:aboutContainers.Wedecidednot todoavideoas thematerial in this section frequentlychangesandvideosbecametooquicklyoutdated.

4.9.2LabActivityDocker

Dothefollowingassignmentswithallitsquestions:

E.Docker.1E.Docker.2E.Docker.3

Placeyoursolutionsinyourgithubhiddirectory

4.9.3LabActivityContainerRESTservice

Develop a simple REST service using OpenAPI. Use the “Introspection”principalwiththeconexionwebservice.DevelopacontainerthatrunstheRESTservice.WriteascriptorpytestthatcontactsthecontainerandreturnstheresultfromtheRESTservice.

WriteaDockerfilethatcreatetheservice

Writeacloudmeshcommand(rememberwedid thisbefore) thatmanagesandinteractswiththeserver,butitactuallyinteractswiththecontainerandnotjustthenativeservice.

Hereisanexamplelistsofcommands(replacemyservicewithacomandthatisnotusedalreadyincloudmesh-cloudorcloudmesh-storage:

Obviouslyyoucanuseservicesfromyourproject.Amoreelaborateversionoftiswill be in your final project.Develop commands thatmake sense for you,

cmsmyservicestart

startstheservice

cmsmyservicestop

stopstheservice

cmsmyservicePATH

ConnectstotherunningserviceandreturnstheObject

curl...

Documenthowyouinteractwithyourservice

Thiscanincludeget,upload,delete,andupdateactionsforexample.

4.10WEEK11:GRAPHQLANDMESSAGING☁�4.10.1LectureMaterial

Anewversionofthefollowingbookshavebeenreleased:

e516LectureNotesEngineeringCloudComputing,GregorvonLaszewski,Ed.2019[7]CloudComputing,GregorvonLaszewski,Ed.2019[1]

Please read the chapter in thebookCloudComputing[1] aboutGraphQL andMessaging.

We have spend significant time to teach you how to generate REST servicesfromOpenAPI specifications. This has the advantage that you do not have todevelop a server as it can be outomatically generated from the OpenAPIspecification.

FurthermoreclientscouldbedevelopedwithSwaggercodeGeninthesupportedlarge number of supported programming languages. However using SwaggercodeGenisnotrequiredforthisclassandgoesbejondthescopewelikeyoutoexploreaspartofthisclass.SoifyouareusingSwaggercodegen,youneedtocontactGregortoidentifywhyyouwouldneedthisforthisclass.Ingeneralitisnotneeded.

Insteadwe like to focuson twodifferent technologies.One is calledGraphQlandtheotheriscalledMQTT.

Theinformationaboutthemareincludedinthebook:

CloudComputing,GregorvonLaszewski,Ed.2019[1]

Pleaselocatethechaptersandread

4.10.2LabActivityGraphQL

Beforedoingtheseoptionalassignments,makesurethatyourprojectisinverygoodshape.Theseoptionalassignmentsareungraded.

E.GraphQL.1E.GraphQL.3

4.10.3OptionalLabActivityMQTT

Beforedoingtheseoptionalassignments,makesurethatyourprojectisinverygoodshape.Theseoptionalassignmentsareungraded.

E.MQTT.2

4.11WEEK12:GO☁�Asyouhavenoticedcloudcomputinghasa lot todowithparallelcomputing,servers,andmessaging.

Whilepythonisagreatlanguageforcloudcomputingwealsoliketomakeyouawareofsomeotherlanguages.OneoftheselanguagesisGO.GohasalotoffeaturesincludedfromCSP.

PleasereadthechapterinthebookCloudComputing[1]aboutGo.

4.11.1OptionalLabActivityGoRESTService

Beforedoingtheseoptionalassignments,makesurethatyourprojectisinverygoodshape.Theseoptionalassignmentsareungraded.

E.Go.1

4.12WEEKN-16☁�TheagendaforthenextweekswilllikelyfollowtheSyllabus.Adjustmentsmayneedtobemadebasedonpaceoftheclass.

WewilltrytoreleasenewmaterialonTuesdays’sbeforemidnight.Assignments

willthanbedueonTuesdaythefollwoingweekat9aminthemorning.IfthingsareunclearpleaseaskquestionsonPiazza.

5APPENDIX

5.1FAQS☁�

5.1.1WheredoIfindthematerialforthisclass?

https://piazza.com/class/jzkfveoqwri3e4

5.1.2Wheretofindtheweeklyagenda?

Wehavealecturenotes(likelythedocumentationyoujustread).Ifyouhavenotyetcreatedashortcutinyourbrowserforthisdocument,pleasedosonow.Asweposted it inCANVAS,piazzaandalssendyouemailabout it thereshouldnot be an issue finding the document However if you have you can use thefollowingmethodscannotfindthisdocument:

A. CANVAS:gotocanvas,gotoourcourse,gotoassignments,lookfortheweek,youwillfindthelinktotheLecturenotes,whichbringsyoutoC)

B. PIAZZA:gotohttps://piazza.com/iu/fall2019/e516fall19/resourceslookforWeeklyAgenda,thiswillbringyoutoC

C. URL:https://laszewski.github.io/book/e516/

Werecommendtobookmarktheurlinyourbrowser

5.1.3Whendotheonlinemeetingstakeplace?

Theonlinemeetingstakeplace

Tue8-9pmEST,StaffedbyGregorandNiranda

Sun8-9amEST,StaffedbyBo

https://piazza.com/class/jzkfveoqwri3e4?cid=7

Online meetings are for residential students not a substitute for the FridayMeetings.

5.1.4WherecanIfindtherecordingsoftheonlinemeetings?

The meeting recordings will generally be available on Thursday evening.HoweversofarwewereabletopostthemthenextmorningonWednesday.

TheSundaymeetngrecordingswillbeavailableTuesdayEvenings.

You can find the link to the recordings in theWeekly lecture notes that arepublishedasepuborpdf.

5.1.5CanIsearchintheePubreader?

Yes,consultthehelpinyourepubreader

5.1.6Icouldnotfindananswertoanassignment?

Insomecases itwill takeyouaconsiderableeffort toresearchtheanswer.Anexample is theDatacenterquestionaboutenergyconsumptionorcost. It isnotexpected that google may return a result. Yoo may need to consider otherpublicationssuchasgooglescholar,orprofessional journals.Ifyouaftersomehour you have not found an answer, maybe the answer there exists not suchinformation.Checkwahtotherstudentshavedoneandaskthemhowtheyfoundtheinformation.Rememberthisclassisacommunity.

5.1.7HowmuchtimedoIneedtospendforeachquestion?

Thisquestioncannotbeansweredasitdependsonthequestion.Somequestionsmaytakehours.Thetitaltimestudentstypicallyspendonthisclassis6hours.However, sometimes it may be less or more dependent on your background.please be reminded that this class is attended bymany students with a lot ofdifferent background. For example if we ask you to do some pythonprogramming,andyouhavenobackground inpython that taskemay takeyoulongerthansomeonewhohaspythonbackground.Howeverafteryouhavedonesomepythontasksthenextpythontaskwillbeeasier.Letusassumeyoudonot

knowpython,butittookyouonly1hourtodotheassignmentsfortehweek,werecommendthatyoudotherestofthetimetolearnpython.THiswaywhenthepythonassignmentstartsyouareapythonexpertalso.

5.1.8IdonotunderstandPiazza?

Atetnd the online hours and ask questions there orwatch the recorded onlinemeetingsasweexplainedittherealso.

5.1.9WheredoIfindthegithubdirectory?

Seehttps://github.com/cloudmesh-communityandsearchforyourname.

5.1.10WherecanIfindthehiddirectory?

Weoftenrefertothegithubdirectoryasthehiddirectory,seeWheretofindthegithubdirectory?

5.1.11WhyyoushouldnotreadthePiazzamailbutuseClickHereinstead?

Piazzasendsyoue-mails.Thefrequencycanbecontrolledbyyouinthepiazzasettings. However, sometimes we do make corrections to the post after youreceived the e-mail. To prevent any issues, we do recommend that instead ofreadingthee-mailyoujustusetheclickherefeaturewhichisplacedontopofeachmail.

Thiswayyouareuptodateinsteadoflookingatapoteniallyoutdatede-mail.Iactivatede-mailnotificationssoyouhavethebestfthesetwomodes.

5.2EPUBREADERS☁�This document is distributed in ePub format. Every OS has a suitable ePubreader to view the document. Such readers can also be integrated into aWebbrowser so thatwhenyouclickonanePub it is automaticallyopened inyourbrowser. As we use ePubs the document can be scaled based on the user’spreferenceIfyoueverseeacontentthatdoesnotfitonapagewerecommend

youzoomouttomakesureyoucanseetheentirecontent.

Wehavemadegoodexperienceswiththefollowingreaders:

macOSX:Books,whichisabuildinebookreaderWindows10:Microsoftedge,but itmustbe thenewestversion,asolderversionshavebugs.Alternatively,usecalibreLinux:calibre

IfyouhaveaniPadorTabletwithenoughmemory,youmayalsobeabletousethem.

Sometimes you may want to adjust the zoom of your reader to increase ordecrease it.Pleaseadjustyourzoomtoa level that iscomfortable foryou.OnmacOSwithalargermonitor,wefoundthatzoomingoutmultipletimesresultsinverygoodrenderingallowingyou tosee thesourcecodewithouthorizontalscrolling.

6REFERENCES

☁�

[1] G. von Laszewski, Cloud computing, Fall 2019. Bloomington, Indiana:Indiana University, 2019 [Online]. Available:https://laszewski.github.io/book/cloud/

[2] G. von Laszewski,Cloud technologies, Fall 2019. Bloomington, Indiana:Indiana University, 2019 [Online]. Available: https://cloudmesh-community.github.io/book/vonLaszewski-cloud-technologies.epub?raw=true

[3]G.vonLaszewski,“Projectformatexample.”Aug-2019[Online].Available:https://github.com/cloudmesh-community/proceedings-fa18/tree/master/project-report

[4] G. von Laszewski, Python for cloud computing, Fall 2019. Bloomington,Indiana: Indiana University, 2019 [Online]. Available:https://laszewski.github.io/book/python/

[5] G. von Laszewski, Linux for cloud computing, Fall 2019. Bloomington,Indiana: Indiana University, 2019 [Online]. Available:https://laszewski.github.io/book/linux/

[6] G. von Laszewski, Scientific writing with markdown, Fall 2019.Bloomington, Indiana: Indiana University, 2019 [Online]. Available:https://laszewski.github.io/book/writing/

[7] G. von Laszewski, E516 cloud computing engineering, Fall 2019.Bloomington, Indiana: Indiana University, 2019 [Online]. Available:https://laszewski.github.io/book/e516/