View
253
Download
5
Category
Preview:
Citation preview
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleDatabaseIn-Memory
AndyRivenesDatabaseIn-MemoryProductManagerSystemsTechnologyGroupDOAG- November2016
EnablingReal-TimeAnalytics
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.
3
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Date Title Location SpeakerTuesdayNov15th11:00-11:45
OracleDatabaseIn-MemoryOption
Seoul AndyRivenesProductManager,Oracle
WednesdayNov16th08:00-08:45
OracleDatabaseIn-Memory:ADeepDiveandaFuturePreview
Tokio TirthankarLahiri–VicePresident,Oracle
WednesdayNov19th11:00-11:45
UsingADOwithDatabaseIn-MemoryandStorage-Based Tiering
Tokio GreggChristmanAndyRivenesProductManagers,Oracle
OracleDatabaseIn-MemoryScheduleforDOAG
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
WhatisDatabaseIn-Memory
5
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleDatabase In-MemoryGoals
6
Real-TimeAnalytics TrivialtoImplement
NoApplicationChangesNotLimitedbyMemory
100X
AccelerateMixedWorkload
AnalyticsTransactions
RunanalyticsonOperationalSystems
EnableReal-TimeBusinessDecisions
Real-TimeAnalytics100X
Risk-Free
ProvenScale-Out,Availability,Security
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Breakthrough:DualFormatDatabase
• BOTH rowandcolumnformatsforsametable
• Simultaneouslyactiveandtransactionally consistent
• Analytics&reportingusenewin-memoryColumnformat
• OLTPusesprovenrowformat
7
BufferCacheNewIn-MemoryColumnStore
SALES SALESRow
FormatColumnFormat
SALES
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Howeasyisittogetstarted
8
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleDatabase12c• DatabaseIn-MemoryisanoptionforOracleDatabase12cEnterpriseEdition
• DatabaseIn-Memoryisincludedinthefirstpatchset (12.1.0.2)for12.1
• UsethelatestDatabaseProactiveBundlePatch–NotPSUsorCPUs
• Noadditionalstepsrequired
9
Note:DatabaseIn-Memoryisnot enabledbydefault
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Oracle Database In-Memory: Simple to Implement
1. Configure Memory Capacity• inmemory_size = XXX GB
2. Configure tables or partitions to be in memory• alter table | partition … inmemory;
3. Later drop analytic indexes to speed up OLTP
10
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleIn-MemoryAdvisor• NewIn-MemoryAdvisor• AnalyzesexistingDBworkloadviaAWR&ASHrepositories
• ProvideslistofobjectsthatwouldbenefitmostfrombeingpopulatedintoIMcolumnstore
Note:Diagnostic&TuningPacksrequired
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Howdoesitwork
12
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleIn-MemoryColumnarTechnology
13
SALES
SALES
PureIn-MemoryColumnar• Purein-memorycolumnformat
• Enableforsubsetofdatabase
• Cheaptomaintain– nologgingorIO• AllowsefficientOLTP
• Nochangetodiskformat
• Builtseamlessly intoOracleDatabase• Appearsasanewstoragetype• Transparent toApplications• AllEnterpriseFeatureswork..
• Availability– RAC,Flashback,DataGuard,etc.• Security– Encryption,Auditing,etc.
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
In-MemoryArea:NewStaticAreawithinSGA
SystemGlobalAreaSGA
BufferCache
SharedPool
LogBuffer
LargePool
In-MemoryAreaOther
• ContainsdatainthenewIn-MemoryColumnFormat
• ControlledbyINMEMORY_SIZEparameter- Minimumsizeof100MB
• SGA_TARGETmustbelargeenoughtoaccommodatethisarea
14
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
CompositionofIn-MemoryArea
• Containstwosubpools:– IMCUpool:StoresInMemoryCompressionUnits(IMCUs)
– SMUpool:StoresSnapshotMetadataUnits(SMUs)
• IMCUscontaincolumnformatteddata
• SMUscontainmetadataandtransactionalinformation
SMU SMU
SMU SMU
SMU SMU
SMU SMU
IMCU IMCU
IMCUIMCU
IMCU IMCU
IMCUIMCU
InMemoryArea
ColumnFormatData
Metadata
15
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
CompositionofIn-MemoryCompressionUnit(IMCU)
• Unitofcolumnstoreallocation– Columnarrepresentationofalargenumberofrowsfromanobject
– Rowsfromoneormoretableextents
• Actualsizedependsonsizeofrows,compressionfactor,etc.
• EachcolumnstoredasaseparatecontiguousColumn CompressionUnit(columnCU)– Rowids alsostoredasaColumnCU
IMCUheader
Extent#13Blocks20-120
Extent#14Blocks82-182
Extent#15Blocks201-301
ROWIDEMPID NAME
DEPT SALARYColumnCUs
16
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Population:Compression• Objectscompressedduringpopulation
• ControlledbyMEMCOMPRESSsubclause
• 2xto20xcompressiontypical
• Multiplelevelsofcompression
- FORDML
- FORQUERY(Default)
- FORCAPACITY
17
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• OrderinwhichobjectsarepopulatedcontrolledbyPRIORITYsubclause:ALTER TABLE sales INMEMORY PRIORITY HIGH;
• Levels:- CRITICAL>HIGH>MEDIUM>LOW
- Controlsorder(notspeed)ofpopulate
• DefaultPRIORITYisNONE- Populateonlyonfirstaccess
• Populationcompletedbynewsetofbackgroundprocesses
ora_w001_orcl
• Numberofprocessescontrolledbyparameter:INMEMORY_MAX_POPULATE_SERVERS
18
Population:Priority
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
WhyAreAnalyticQueriesFasterInTheIn-MemoryColumnStore?
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ScanningandfilteringdatamoreefficientlyReal-TimeAnalytics
StorageIndexesCompression
Scan&filterdataincompressedformat
Accessonlythecolumnsyouneed
ColumnarFormat SIMDVectorProcessing
ProcessmultiplecolumnvaluesinasingleCPUinstruction
VectorRegister
Loadmultipleregionvalues
VectorCompareallvaluesan1cycle
CPUCA
CACA
CA
Min1Max3
Min4Max7
Min8Max12
Pruneoutanyunnecessarydatafromthecolumn
20
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Speedofmemory• ScanandFilteronlytheneededColumns
• VectorInstructions
Real-timeAnalyticsImprovesallaspectsofanalyticqueries
DataScans
VectorRegisterCPU
CA
CACA
CA
In-MemoryAggregation
•CreateIn-MemoryReportOutlinethatisPopulatedduringFastScan
•RunsReportsInstantly
Joins
•ConvertStarJoinsinto10XFasterColumnScans
•Searchlargetableforvaluesthatmatchsmalltable
HASHJOIN
TableA TableB
SALES
STATE=CA
21
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
What’sNew
22
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Availablenow– Exadata ExpressCloudService
• Comingsoon– DatabaseCloudServices– Exadata CloudMachine
23
Announcing OracleDatabase12cRelease2onOracleCloud
OracleispresentingfeaturesforOracleDatabase12cRelease2onOracleCloud. WewillannounceavailabilityoftheOn-Prem releasesometimeafterOpenWorld.
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Analyticquerieshavecomplexjoinswithnofilterpredicatesspecified
• JoinGroupspecifiescolumnsusedtojointables– Columnsharecompressiondictionary– Joinsoccuron dictionaryvaluesratherthandata
• Enables2-3xfasterjoinprocessing
Real-TimeAnalytics:FasterIn-MemoryJoins
Sales
VEHICLENAM
E
Example:FindsalespriceofeachVehicleSalesVehicle
NAMEisjoincolumnN
AME
CREATE INMEMORY JOIN GROUP V_name_jg(VEHICLES(NAME),SALES(NAME));
24
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Analyticqueriescontaincomplexexpressions– Originallyevaluatedforeveryrow
• Expressions pre-computedandcachedin-memory–Userdefinedviavirtualcolumns–Or expressionsautomaticdetected
• AllIn-Memoryoptimizationsapply• 3-5x fastercomplexqueries
Real-TimeAnalytics:In-MemoryExpressions
Net=Price+Price*Tax
In-MemoryColumnStore
Sales
Example:ComputetotalsalespriceTax
Price
25
Price
Price+PriceXTax
Price+Price*Tax
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• FullJSONdocumentspopulatedusinganoptimizedbinaryformat
• AdditionalexpressionscanbecreatedonJSONcolumns(e.g.JSON_VALUE) &storedincolumnstore
• QueriesonJSONcontentorexpressionsautomaticallydirectedtoIn-Memoryformat- e.g.Findmovieswheremovie.name
contains“Jurassic”
• 60x performancegainsobserved
Multi-ModelAnalytics:In-MemoryJSON
Relational
In-MemoryColumStore
In-MemoryVirtualColumns
In-MemoryJSONFormat
{"Theater":"AMC 15","Movie":”Sully","Time“:2016-09-09T18:45:00","Tickets":{
"Adults":2}
}
Relational Virtual JSON
26
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• AutomaticDataOptimizationextendedtoIn-Memory
• In-Memorypoliciesallowoptimaluseofcolumnstorebasedonheatmap- Requiresonlyinmemory option,not
advancedcompression
• Policiescanbedefinedto:- BringdataintotheIMcolumnstore- Increasecompressionlevelsasdatacools
- EvictcolddatafromIMcolumnstore27
Sales_Q3
Sales_Q2
Sales_Q4
In-MemoryColumnStore
Sales_Q1
Automation:In-MemoryDataAutoPopulationPolicies
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
HowdoesitimpactOLTPenvironments
28
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• ComplexOLTPisSlowedbyAnalyticIndexes
• Insertingonerowintoatablerequiresupdating10-20analyticindexes:Slow!
• ColumnStoreReplacesAnalyticIndexes
• Fastanalyticsonany columns
• ColumnStorenotpersistentsoupdatecostismuchlower
29
DatabaseIn-MemoryAcceleratesMixedWorkloads
Table1– 3OLTP
Indexes
10– 20AnalyticIndexes
REPLACE
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-MemoryandotherOracleDatabasefeatures
30
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-Memory:ScalestoAnySize
31
Scale-Up
•Scale-UponlargeSMPs•NUMAOptimized
•Scale-OutAcrossServerstoGrowMemoryandCPUs
• In-MemoryQueriesParallelizedAcrossServers
Scale-Out CombinewithFlashandDisk
DISKPCIFLASHDRAM
ColdDataHottestData
ActiveData
•Easilyplacedataonmostcosteffectivetier
•SimultaneouslyAchieve:•Speed ofDRAM• I/OsofFlash•Cost ofDisk
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.| 32
RAC
ASM
RMAN
Data Guard & GoldenGate
• Pure In-Memory format does not change Oracle’s storage format, logging, backup, recovery, etc.
• All Oracle’s proven availability technologies work transparently
• Protection from all failures- Node, site, corruption, human error, etc.
DatabaseIn-Memory:IndustrialStrengthAvailability
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Similar to storage mirroring
• Duplicate in-memory columns on another node- Enabled per table/partition
• e.g. only recent data
- Application transparent
• Downtime eliminated by using duplicate after failure
33
OnlyAvailableonEngineeredSystems
DatabaseIn-Memory:UniqueFaultTolerance
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-Memory:TrivialtoImplement
34
SimpletoConfigure
NocomplexsetupSetcolumnstoresize
DeclareIn-Memorytables
FullFunctionality
NoSQLrestrictions
100%Compatible
Noapplicationchanges
Nodatamigration
EasytoDeploy
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
HowarecustomersusingOracleDatabaseIn-Memory
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
AT&TWiFi – DataWarehouse• BusinessObjectsreports100X faster
• ETLprocessesimprovedby50% faster
• NochangestoSAPBusinessObjectsreports
Villeroy &Boch – SAPBW• SAPBWCOPAqueries30– 33X faster
• SAPTransactionlistqueries4– 4,800Xfaster
• Avoidedexpensive&riskyupgradetoS4/Hana
36
BOSCH – SAPCRM• Dropped allcustomindexes
• Analyticqueries2-20Xfaster,DML2-3X faster
• Nochangestoapplicationrequired
DieMobiliar – MixedWorkload• RemovalofNetezza system
• Analyticqueries50-200X faster
• Databasesizereduced considerably
DatabaseIn-MemoryReferences
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
In-MemoryUseCasesOLTP
• Real-timereportingdirectlyonOLTPsourcedata
• RemovesneedforseparateODS• Speedsdataextraction
DataWarehouse• Staging/ETL/Tempnotacandidate
• Writeonce,readonce• AllorasubsetofFoundationLayer
• Fortimesensitiveanalytics• PotentialtoreplaceAccessLayer
37
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
WherecanIgetmoreinformation
38
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
JointheConversation
39
https://twitter.com/TheInMemoryGuyhttps://blogs.oracle.com/in-memory/
WhitePapers(otn.com)• OracleDatabaseIn-MemoryWhitePaper• OracleDatabaseIn-MemoryAggregationPaper• WhentouseOracleDatabaseIn-Memory• OracleDatabaseIn-MemoryAdvisor
Videos• OracleDatabaseIn-MemoryYouTubeChannel• oracle.com- PoweringtheReal-TimeEnterprise- IndustryExpertsSharePerspectivesoracle.com/us/corporate/events/dbim/index.html
• YouTube- JuanLoaiza:SoftwareinSilicon
AdditionalQuestions• In-Memoryblog:blogs.oracle.com/In-Memory• Myemail:andy.rivenes@oracle.com
https://www.facebook.com/OracleDatabasehttp://www.oracle.com/goto/dbim.html
AdditionalResources
Recommended