Upload
peter-cochrane
View
524
Download
0
Embed Size (px)
Citation preview
cochrane.org.ukPeter CochraneBIG
S m a l l D a t a
I M P A C T
Ahmed Elmagarmid
Mourad Ouzzani
Stochastic complexity on a massive scaleexhibiting fractal behaviours from a local
to a national, international and global scale
ManAgement StruggleGenera l l y confounded by complex i ty
Instinct and experience increasingly dangerousKnee jerk management is always dangerous
Dynamic connectivityand multi-species
non-linearity
L i m i t e d : - A s s u m p t i o n s - M e a s u r e s - T h i n k i n g - M o d e l s - D a t a
E v o l v e d f o r a n s l o w c h a n g i n g a n a l o g u e w o r l d - l i n e a r i t y r u l e d & o u r m a t h w o r k e d
- b u t i t a l l c r a s h e s a b o v e o r d e r ( 5 ) - c o m p l e x i t y n o w r u l e s w i t h 1 0 s t o
1 0 0 0 s o f v a r i a b l e s - a n d p e r h a p s w o r s e
w e c o p e w i t h < 5 - 7 s i m u l t a n e o u s i d e a s
“There are lots of simple solutions to complex problems
and they are all wrong”
tyranny of THE SIMPLEWe are l inear th inkers but the wor ld i s complex
p o l i t i c s M a n a g e r s ECO NOM I CS
“People love simple - it is soobvious and doesn’t involve thinking”
Technology
Ecology
dependency J IGSAW
Economics
Society
Industry
Care
Markets
Trade
Education
Policing
Defence
Commerce
Security
Logistics
Finance
Government
Companies
W e h a v e n o m o d e l o r u n d e r s t a n d i n g
“ S o m e t i m e s W E m i g h t o b s e r v e o r i n f e r s o m e l i m i t e d re l a t i o n s h i p s a l o n g w i t h t y p i c a l
c h a r a c t e r i s t i c s - h o w e v e r, A I c a n p ro d u c e a m o re c o m p re h e n s i v e a n a l y s i s a n d a
m u c h f u l l e r p i c t u re”
The BIG DATA DOMAIN links everything and records relationships and patterns of behaviour - in doing so it creates vast amounts of data vital to the modelling of the real world…
Law & Order
Defence
Education
Health
Institutions
AX I O MSV e r y o b v i o u s ?
AX I O MSV e r y o b v i o u s ?
I n c r e a s e d e f f i c i e n c y l e a d s t o i n c r e a s e d b r i t t l e n e s s a n d m o re f a i l u r e s … .
… m o re d a t a i s a m u s t f o r a c c u ra t e b u s i n e s s m o d e l s a n d wa r g a m e s …
… . B I G a n d s m a l l D ATA a re ke y t o a m o re p re d i c t a b l e t h e f u t u re
S t o p O ve r O p t i m i s i n g i f y o u wa n t yo u r
c o m p a n y t o s u r v i v e a n d p ro s p e r
Avo i d e r ro n e o u s K P I s a n d b o n u s d r i v e n
c u l t u r e s
E m b ra c e a d a t a a n d b u s i n e s s m o d e l d r i v e n
a p p ro a c h
B I G D A T A
M A Y N O T B E E N O U G H
- P o l i s h i n g w h a t w e h a v e w i l l n o t r e a l i s e s u s t a i n a b i l i t y - N e w m a t e r i a l s & m a n u f a c t u r i n g m e t h o d s a r e r e q u i r e d - N e w d i s t r i b u t i o n , s u p p o r t a n d m a i n t e n a n c e r e g i m e s - E n e r g y a n d m a t e r i a l u s a g e h a s t o b e m i n i m i s e d - R e u s i n g , R e p u r p o s i n g , R e c y c l i n g , h a s t o b e r e a l - E c o n o m i c s y s t e m s h a v e t o b e f i t f o r p u r p o s e
AX I O MSNot so obv ious ?
“ To b u i l d a n e r v o u s s y s t e m f o r t h e p l a n e t w i t h m o s t
p e o p l e a n d t h i n g s c o n n e c t i n g ( a t w i l l ) w i t h
d i s t r i b u t e d s t o r a g e o f d a t a a n d i n f o r m a t i o n ”
MISSIONor evo lut ion ?
“ To s t o p p r o d u c i n g m o r e a n d m o r e f o r t h e f e w a n d
s t a r t p r o v i d i n g s u f f i c i e n t f o r t h e m a n y ”
DATA Big & SMALL
Small Data Focussed Detail
Big Data Macro-View Relationships
Limited C o n t a i n e d C o n s t r a i n e d
Expanding Te n d i n g t o
T h e I n f i n i t e
Both impor tan t & dynami c
THE AUDIT TRAILRaw mater ia l s to ro l l out to user
Raw mater ia l extract ion, re f inement Raw mater ia l recover from scrap Shipping log ist ics & supply cha in Process ing : forming, machin ing Test ing and Q A assessment Assembly machine & people Software loading & test Assembly test ing Packaging Shipping Supply Sa le
Sel ler test ing Sa le process /prep
Ownership Registrat ion User habits & maintenance
User feedback to des ign team Onward sa l e ( repea t a l l above )
Disposa l - canniba l i sat ion for parts Recyc l ing/recovery of raw mater ia ls
HEALTH PrognosisHospitalPrime Centre of Expertise & Capability
Medical CentrePrime Point of Local Capability
Home BasedMeasurementsMonitoringMedication
Technology Push - Patient Pull
M i g r a t i n g c e n t r e o f g r a v i t y
Dev i c e to d ev i c eMonitoring the in:out & effect give a complete picture
Self contained home net with no internet connection - or an aggregated and sporadic net connect
My toothbrushWil l not want to connect to the in ternet
….nor w i l l i t use 5G ! !
The co f fee mach ineANECDOTE
PurchasedFa i led in month 12Repa ired under warrantyReturned wi th a user repor t
8000 cof fees consumed in one year
8200 hours in one year ~ 1 cof fee an hour
Hea l th impl i ca t ionsProduct impl i ca t ions
The co f fee mach ineANECDOTE
PurchasedFa i led in month 12Repa ired under warrantyReturned wi th a user repor t
8000 cof fees consumed in one year
8200 hours in one year ~ 1 cof fee an hour
Hea l th impl i ca t ionsProduct impl i ca t ions
What will our applianceshave to tell us ??
FridgeFreezer
MicrowaveDishwasher
Washing MachineTumbler dryer
Oven/HobSecurityAir-ConShower
+++Des igner Feedback
Forward looking radar and car-to-car comms
Rearward looking radar and car-to-car comms
3,4,5G Network Connection
WiFi and WiMax Net Connection to Gas Station, Repair Shop for OS, APP, Info upgrades, failure reporting and real time remote diagnostics
Self contained data storage and analysis Small Data DomainVEHICLE to vehicle Comms
VEHICLE CommsNetworks without infrastructure
BIG DATA = Vehicle type, location, speed, owner, insurance, taxation…
small DATA = Production and repair history, performance data, driver habits,
apps,…
3,4,5G netscannot supportthe demands of
future traffic andlogistic networks…
but vehicle to vehiclesystems can…by providing
the most efficient and relevantlinking of users reporting their status
and sharing local/pertinent information
distributionA u t o m a t e d & i n t e l l i g e n t
BIG DATA Where has the package come from and where is it going ? Where is it now and what does it contain ?
small DATA The deta i led product history of how it was created; where it has been s t o re d a n d d e t a i l e d handling history including humidity, temperature, G-force, damage +++
Item
Box
Pallet
Warehouse
Container
Supplier
Customer
Ship toshore
Containerto Ship
H I D D E N V A L U EI n eve ry conta iner opera t ion g loba l l y
B IG DATA = Where i s my conta iner,s chedu le and content s ?
Sma l l DATA = What i s the cond i t i on o fthe conta iner content s ?
No single solutioncan deal with all forms of attack....
Security
N o o n e s o l u t i o n i ss u f f i c i e n t : t h e r e a r en o g o l d e n b u l l e t s …
I o T a n d D a t a
B I G & s m a l l D ATA r e q u i r e dt o m o d e l & s y n t h e s i s e r e a c t i v ea n t i d o t e s a n d a n t i c i p a t o r y p r o t e c t i o n
… a n a u t o i m m u n e s y s t e mi s r e q u i r e d t o c o p e w i t ht h e I o T a n d s t o c h a s t i cc o m p l e x i t y / a t t a c k s !
N E E D T O S H A R E
T R U M P S N E E D T O K N O W
We m i g h t s u p p o s e t h a t t h i s w o u l d t a k e p l a c e i n s e c o n d s o r m i n u t e s a n d n o t h o u r s a n d d a y s …
How Many Devices ?I m p l a n t e d , w o r n , c a r r i e d b y p e o p l e
To d a y I w e a r a n d c a r r y 6 s m a r t c o m m u n i c a t i n g t h i n g s … b u t i n t h e f u t u r e i t i s l i k e l y t o b e > > 1 0 ! !
H o w m a n y h a v e y o u g o t … . ? ?
T h e y a r e a l l a p a r t o f t h e I o T - a n d m a y / c a n / d o c r e a t e B I G a n d s m a l l d a t a …
Forbes Projection Based on an Industry Average
Forbes Projection Based on an Industry Average
A N I o T O F 5 0 B n T H I N G S
I S C O N S E R V A T I V E !
Stored information growth
1986
I can’t draw a
dot small
enough!!
.1996
0.001ZB
2006
0.13ZB
2016
13ZB
IoT will accelerate this trend to promote ever more on device storage
ZetaByte = 1021 ExaB = 1018 PetaB = 1015 TeraB = 1012
Prognosis: IoT small data distributedstorage may exceed the
internet
2014 2016 2018 2020 2022 2024
2
6
10
14
16
18
20
22
% Data Stored on Things
Year
Projection
Actual
Data extracted from numerous repor t s , papers & presentations by IDC, EMC, ICS,
Stored ON THINGS
Thank You
cochrane.org.uk c a - g l o b a l . o r g