Btree project4 discussionpages.cs.wisc.edu/~jignesh/cs564/notes/BadgerDB_Btree_project.pdf ·...

BadgerDB::Btree• Goal:BuildkeycomponentsofaRDBMS

– Firsthandexperiencebuildingtheinternalsofasimpledatabasesystem

– Andhavesomefundoingso!

• Twoparts– Buffermanager[✔]– B+tree(DueDate:Mar27by2PM)

• FirstclassdayaftertheSpringbreak

Allprojectsareindividualassignments

2/23/17 CS 564: Database Management Systems, Udip Pant and Jignesh Patel 1

StructureofDatabase

Queryoptimizerandexecution

Relationaloperators

Fileandaccessmethods

Buffermanager

I/Omanager

Planfortoday

• ReviewofC++templatesandhelpfulfunctions– memset,memcpy andreinterpret_cast

• B+tree:insertion• <break>• BadgerDB::Btree

– Projectspecifications– Code

• Q&A

C++templates

Humanmiseryofduplicatecodes

• SupposeyouwriteafunctionprintData:

• lateryouwanttoprintdoubleandstd::string

void printData(int value) {std::cout << "The value is "<< value;

void printData(double value) {std::cout << "The value is "<< value;

void printData(std::string value) { std::cout << "The value is "<< value;

}2/23/17 CS 564: Database Management Systems, Udip Pant and Jignesh Patel 5

AndStroustrup said- lettherebetemplates

template<typename T>void printData(T value) {

std::cout << "The value is ” << value;}

AndStroustrup said- lettherebetemplates

template<typename T>void printData(T value) {

std::cout << "The value is ” << value;}

Templatesemantics

• Thesyntaxissimple:template< typename name ó class name >

• Functiontemplates• Classtemplates

Functiontemplates

template<typename T>void func() {}

int main() { func<int>();func<double>();

Functiontemplates

template<typename T>void func() {}

int main() { func<int>();func<double>();

template<typename T>void func(T value) {} template<typename T, typename U>T func2(U value) {

return T(value); }

int main() { // T=intfunc(3);// T=doublefunc(3.5);

// T=int, U=double func2(3.5);// T=std::vector, U=intfunc2<std::vector> (5); // specify both T and U // T=std::vector, U=intfunc2<std::vector, int>(5.7);

Classtemplates• Alsoworksonstructs

template<typename T, int i>struct FixedArray {

T data[i]; };

FixedArray<int, 3> a; // array of 3 integers

Classtemplates• Alsoworksonstructs

template<typename T, int i>struct FixedArray {

T data[i]; };

FixedArray<int, 3> a; // array of 3 integers

template<typename T>class MyClass { };

template<typename T1, typename T2=int> class MyClass{};

// specify all parameters MyClass<double, std::string> mc1;

// default value for T2MyClass<int> mc4;

Templaterequirements• Templatesimplicitlyimposerequirementsontheirparameters

• TypeThastobe:– Copy-Constructible if

T a(b);– Assignablei.e. definesoperator=() if:

a=b;– etc

• Forthisproject:operationssuchasa < b couldmeandifferentforint andstd::string

Pointersandarrays

• arraysworkverymuchlikepointerstotheirfirstelementsint myarray [20];int * mypointer;

Pointersandarrays

• arraysworkverymuchlikepointerstotheirfirstelementsint myarray [20];int * mypointer;

Canyoudo?mypointer = myarray;

Pointersandarrays• arraysworkverymuchlikepointerstotheirfirstelementsint myarray [20];int * mypointer;

Canyoudo?mypointer = myarray;

Can you do?myarray = mypointer;

Pointersandarrays• arraysworkverymuchlikepointerstotheirfirstelementsint myarray [20];int * mypointer;

Canyoudo?mypointer = myarray; ç Yes

Can you do?myarray = mypointer; ç No

Example// more pointers#include <iostream>using namespace std;int main () {

int numbers[5];int * p;p = numbers; *p = 10;p++; *p = 20;p = &numbers[2]; *p = 30;p = numbers + 3; *p = 40; p = numbers; *(p+4) = 50;for (int n=0; n<5; n++)

cout << numbers[n] << ", ";return 0;

}2/23/17 CS 564: Database Management Systems, Udip Pant and Jignesh Patel 19

Example// more pointers#include <iostream>using namespace std;int main () {

int numbers[5];int * p;p = numbers; *p = 10;p++; *p = 20;p = &numbers[2]; *p = 30;p = numbers + 3; *p = 40; p = numbers; *(p+4) = 50;for (int n=0; n<5; n++)

cout << numbers[n] << ", ";return 0;

Prints: 10, 20, 30, 40, 50,

Pointersandstringliteral

• const char*foo="hello";

Pointersandstringliteral

• const char*foo="hello";

• Foocontainsthevalue1702,andnot'h',nor"hello“• Whatistheoutputof?

*(foo+4)foo[4]

C/C++helpfulfuctions

memset• void * memset ( void * ptr, int value, size_t num );• Fillblockofmemory

– Setsthefirst num bytesoftheblockofmemorypointedby ptr tothespecified value (interpretedasan unsignedchar)

int main () { char str[] = "almost every programmer should know memset!"; memset (str,'-',6); print(str);return 0;

memcpy• void*memcpy (void*dest,const void*source,size_t num );• Copyblockofmemory

– Copiesthevaluesof num bytesfromthelocationpointedtoby source directlytothememoryblockpointedtoby destination.

• std::memcpy ismeanttobethefastestlibraryroutineformemory-to-memorycopy(usuallymoreefficientthan std::strcpy)

struct { char name[40]; int age; } person, person_copy;

int main () { char myname[] = ”Bucky Badger"; /* using memcpy to copy string: */memcpy ( person.name, myname, strlen(myname)+1 ); /* using memcpy to copy structure: */ memcpy ( &person_copy, &person, sizeof(person) ); return 0;

reinterpret_cast<new_type>(expr)• reinterpret_cast convertsanypointertypetoanyotherpointer

type,evenofunrelatedclasses.• Allpointerconversionsareallowed:neitherthecontentpointed

northepointertypeitselfischecked.

int main () { int i = 7;int* p1 = reinterpret_cast<int*>(&i);assert(p1 == &i);// type aliasing through pointerchar* p2 = reinterpret_cast<char*>(&i);// type aliasing through referencereinterpret_cast<unsigned int&>(i) = 42; std::cout << i << '\n';

B+tree

• Occupancy(d)– Minimum50%occupancy(exceptforroot)– Eachnodecontainsd<=m<=2dentries.

2/23/17 CS 564: Database Management Systems, Udip Pant and Jignesh Patel 281/29/17 CS 564: Database Management Systems, Jignesh M. Patel 5

(Ubiquitous)B+Tree• Height-balanced(dynamic)treestructure• Insert/deleteatlogF Ncost(F=fanout,N=#leafpages)• Minimum50%occupancy(exceptforroot).

Eachnodecontainsd <=m <=2d entries.Theparameterd iscalledtheorder ofthetree.

• Supportsequalityandrange-searchesefficiently.

Index Entries(Direct search)

Data Entries

Data EntriesEntries in the leaf pages:

(search key value, recordid)

Index EntriesEntries in the index (i.e. non-leaf) pages:

(search key value, pageid)

1/29/17 CS 564: Database Management Systems, Jignesh M. Patel 5

Data Entries

B+-Tree:InsertingaDataEntry• FindcorrectleafL.• PutdataentryontoL.

– IfLhasenoughspace,done!– Else,mustsplit L(intoLandanewnodeL2)

• Redistributeentriesevenly,copyup middlekey.• InsertindexentrypointingtoL2intoparentofL.

• Thiscanhappenrecursively– Tosplitnon-leafnode,redistributeentriesevenly,but

pushingup themiddlekey.(Contrastwithleafsplits.)• Splits“grow”tree;rootsplitincreasesheight.

– Treegrowth:getswider oroneleveltallerattop.

Inserting8*intoB+TreeRoot

17 24 30

2* 3* 5* 7* 14* 16* 19* 20* 22* 24* 27* 29* 33* 34* 38* 39*

Entry to be inserted in parent nodeCopied up (and continues to appear in the leaf)

2* 3* 5* 7* 8*

Inserting8*intoB+TreeRoot

17 24 30

2* 3* 5* 7* 14* 16* 19* 20* 22* 24* 27* 29* 33* 34* 38* 39*

Entry to be inserted in parent nodeCopied up (and continues to appear in the leaf)

2* 3* 5* 7* 8*

Inserting8*intoB+Tree

Insert in parent node.Pushed up (and only appears once in the index)

5 24 30

Root17

14*16* 19*20*22* 24*27*29* 33*34*38*39*

7*5* 8*

Inserting8*intoB+Tree

• Rootwassplit:heightincreasesby1• Couldavoidsplitbyre-distributingentrieswithasibling

– Sibling:immediatelytoleftorright,andsameparent

5minsbreak

https://www.youtube.com/watch?v=AxSdWhkMB_A

BadgerDB:Btree

Afteryouuntar theproject:• btree.h:Addyourownmethodsandstructuresasyouseefitbut

don’tmodifythepublicmethodsthatwehavespecified.• btree.cpp:Implementthemethodswespecifiedandanyothers

youchoosetoadd.• file.h(cpp):ImplementsthePageFile andBlobFile classes.• main.cpp:Usetotestyourimplementation.Addyourowntests

hereorinaseparatefile.ThisfilehascodetoshowhowtousetheFileScan andBTreeIdnex classes.

• page.h(cpp):ImplementsthePageclass.• buffer.h(cpp),bufHashTbl.h(cpp):Implementationofthebuffer

manager.• Exceptions/*:Implementationofexceptionclassesthatyou

mightneed.• Makefile – makefile forthisproject.

B+-treePageFormatLe

R1 K 1 R2 K 2 K n P n+1

data entries

record 1 record 2

Next PagePointer

record n

Prev PagePointer

geP1 K 1 P 2 K 2 P 3 K m P m+1

index entries

Pointer to apage with Values < K1

Pointer to a pagewith values s.t.K1≤ Values < K2

Pointer to apage with values ≥Km

Pointer to a pagewith values s.t., K2≤ Values < K3

Index• theindex willstoredataentriesintheform<key, rid> pair• storedinafilethatisseparatefromthedatafile• i.e.theindexfile“pointsto”thedatafilewheretheactualrecords

arestored

• storesalltherelations(actualdata)aswedidinthebuffermanagerassignment

• Youdon’tactuallyusethisoneforthisproject

PageFile BlobFile• pagesinthefilearenotlinkedby

prevPage/nextPage links• treatsthepagesasblobsof8KB

sizei.e doesnotrequirethesepagestobevalidobjectsofthePageclass

• usetheBlobFile tostoretheB+indexfile

• everypageinthefileisanodefromtheB+tree

• wecanmodifythesepagestosuittheparticularneedsoftheB+treeindex

FileScan class• TheFileScan classisusedtoscanrecordsinafile.• FileScan(const std::string&relationName,BufMgr *bufMgr)

– TheconstructortakestherelationName andbuffermanagerinstance• ~FileScan()

– Shutsdown thescanandunpinsanypinnedpages.• void scanNext(RecordId& outRid)

– Returns(viatheoutRid parameter)theRecordId ofthenextrecordfromtherelationbeingscanned.ItthrowsEndOfFileException()whentheendofrelationisreached.

• std::string getRecord() – Returnsapointertothe“current”record.Therecordistheoneina

precedingscanNext()call.• void markDirty()

– Youdon’tneedthisforthisassignment

BadgerDB:B+Tree IndexSimplifications:• assumethatallrecordsinafilehavethesamelength(soforagivenattributeitsoffsetintherecordisalwaysthesame).

• onlyneedstosupportsingle-attributeindexing• theindexedattributemaybeoneofthreedatatypes:integer,double,orstring

• inthecaseofastring,youcanusethefirst10charactersasthekeyintheB+-tree

• wewillneverinserttwodataentriesintotheindexwiththesamekeyvalue2/23/17 CS 564: Database Management Systems, Udip Pant and Jignesh Patel 41

B+Tree Index:Constructor

const string&relationName

Thenameoftherelationonwhichtobuildtheindex.Theconstructorshouldscanthisrelation(usingFileScan)andinsertentriesforallthetuplesinthisrelationintotheindex

String&outIndexName

Thenameoftheindexfile;determinethisnameintheconstructorasshownabove,andreturnthename.

BufMgr *bufMgrIn Theinstanceoftheglobalbuffermanager.

const intattrByteOffset

Thebyteoffsetoftheattributeinthetupleonwhichtobuildtheindex.Forinstance,ifwearestoringthefollowingstructureasarecordintheoriginalrelation:

And,wearebuildingtheindexoverthedoubled,thentheattrByteOffset valueis0+offsetof(RECORD,i),whereoffsetof istheoffsetpositionprovidedbythestandardC++library“offsetoff”.

const DatatypeattrType

Thedatatypeoftheattributeweareindexing.NotethattheDatatypeenumeration{INTEGER,DOUBLE,STRING}isdefinedinbtree.h

If the index file already exists, open the file. Else, create a new index fileParameters:

B+Tree Index:insertEntry

const void*key Apointertothevalue(integer/double/string)wewanttoinsert.

const RecordId &rid Thecorrespondingrecordidofthetupleinthebaserelation.

insertEntryinserts a new entry into the index using the pair <key, rid>.Input to this function:

B+Tree Index:insertEntry

const void*key Apointertothevalue(integer/double/string)wewanttoinsert.

const RecordId &rid Thecorrespondingrecordidofthetupleinthebaserelation.

insertEntryinserts a new entry into the index using the pair <key, rid>.Input to this function:

You will be spending bulk of your time /code in this method

B+Tree Index:startScan

const void*lowValue

Thelowvaluetobetested.

const OperatorlowOp

Theoperationtobeusedintestingthelowrange.YoushouldonlysupportGTandGTEhere;anythingelseshouldthrowBadOpcodesException.

const void*highValue

Thehighvaluetobetested.

const OperatorhighOp

Theoperationtobeusedintestingthehighrange.YoushouldonlysupportLTandLTEhere;anythingelseshouldthrowBadOpcodesException.

startScanThis method is used to begin a “filtered scan” of the index. For e.g. if the method is called using arguments (“a”,GT,”d”,LTE), the scan should seek all entries greater than “a” and less than or equal to “d”.Input to this function:

B+Tree Index:scanNext

• scanNext• fetchestherecordidofthenexttuplethatmatchesthescan

criteria.Ifthescanhasreachedtheend,thenitshouldthrowtheexceptionIndexScanCompletedException

RecordId &outRid

outputvaluethisistherecordidofthenextentrythatmatchesthescanfiltersetinstartScan.

B+Tree Index:endScan

• endScan• terminatesthecurrentscanandunpins allthepagesthathavebeenpinnedforthepurposeofthescan

• throwsScanNotInitializedException ifcalledbeforeasuccessfulstartScan call.

Implementationnotes• callthebuffermanagertoread/writepages• don’tkeepthepagespinnedinthebufferpoolunlessyouneedto

• Forthescanmethods,youwillneedtorememberthe“state”ofthescanspecifiedduringthestartScan

• insert doesnot needtoredistributeentries• Attheleaflevel,youdonotneedtostorepointerstobothsiblings.Theleafnodesonlypointtothe“next”(theright)sibling

FAQs• HowdoIgetstarted?

iftheindexfiledoesnotexist:createnewBlobFile.allocate newmetapageallocate newrootpagepopulate'IndexMetaInfo'withtherootpage numscanrecordsandinsertintotheBTree

elsereadthefirstpagefromthefile- whichisthemetanodegettherootpagenum fromthemetanodereadtherootpage(bufManager->readPage(file,

rootpageNum,out_root_page)onceyouhavetherootnode,youcantraversedownthetree

• howtocheckwhetheranindexfileexists?– Seefile.h:staticboolexists(const std::string&filename)

• How do I write a node to disk as a Page? e.g. how to write IndexMetaInfonode to the file?

Þ You first need to allocate a Page using the bufferManager.Page* metaPage;bufManager->allocatePage(..., metaPage);

Then you can either cast it as IndexMetaInfo* and update it's parameter. Another way is to first create and populate MetaIndexInfo node.Then you can allocate a new Page* using bufferManager as above. Then use 'memcpy' to copy tothe new Page:

memcpy(metaPage, &metaInfo,sizeof(IndexMetaInfo));

But you do not need to write it back as page to disk explicitly. Buffer Manager does it for you.Remember project 2?

• howtoconvertPage - thatyoureadfromthefiletoNode?– youcancaste.g.usingreinterpret_cast

Suggestions

• Startearly– 1000+linesofcodes

• Trytofinishbeforethespringbreak– NoTAhoursduringthebreak

• Makeincrementalprogress– Testaggressively

Btree project4 discussionpages.cs.wisc.edu/~jignesh/cs564/notes/BadgerDB_Btree_project.pdf ·...

Documents

Jignesh patel,ivivc

564 HERPETOCULTURE

Postgre sql linuxcontainers by Jignesh Shah

Jignesh Final)

Candidate Feedback Report Prepared for: Jignesh … Feedback Report Prepared for: Jignesh Suba Date: ... key factors that led to this success? ... Verbal Comprehension:

The Gujarat Government Gazette · The Gujarat Government Gazette ... Khunt Manish, 87, ... 84 Nayi Jignesh Rameshbhai Parekh Jignesh Rameshbhai Block No. 832,

CS 564 Database Management Systems: Design and Implementation Lecture 5: SQL Chapters 3 and 5 in Cow Book Slide ACKs: AnHai Doan, Jeff Naughton, and Jignesh

Jignesh portfolio

Jignesh nakum(c.s.)

Nutrition 564: Introduction Nutr 564: Management Summer 2007

HP Photosmart Plus series - B209 Englishh10032. · HP Photosmart Plus series - B209 2 1 Cyan Cartridge Magenta Cartridge Yellow Cartridge Black Cartridge 564 564 564 564 Apply sticker

Toll Free: (619)564-5600 | Direct: (619)564-5640 | Fax ... · Toll Free: (619)564-5600 | Direct: (619)564-5640 | Fax ... is issued solely for the purpose of facilitating the issuance

27. jignesh sir AQAR 2016-17.pdfMicrosoft Word - 27. jignesh sir.docx Author Jignesh Created Date 11/27/2016 2:41:44 PM

ENVIORNMENTAL APPRAISALS AND PROJECTS JIGNESH R PANCHAL LUHARIYA

Jignesh ich

Jignesh Electrical Notes

DR. JIGNESH ZALAVADIAdspace.sctimst.ac.in/jspui/bitstream/123456789/10882/1/6879.pdf · DR. JIGNESH ZALAVADIA Dissertation submitted in partial fulfillment of the requirements for

Meet the face of Brand India 2.0 Jignesh - 63 moons · Jignesh Exchange is to Financial Technologies, what Search is to Google–Platform for Mass Disruptive Innovation,” says Jignesh

Learning Generalized Linear Models Over …pages.cs.wisc.edu/~jignesh/publ/GLMs-over-joins.pdfLearning Generalized Linear Models Over Normalized Data Arun Kumar Jeffrey Naughton Jignesh

Jignesh Steel Maharashtra India