Embed Size (px)
Citation preview
1
Live Wire: The Next Dimension for Streaming Analytics Alternatives for collecting and using streaming data
Analyst commentary
December, 2015 Mark Madsen Third Nature @markmadsen
Copyright Third Nature, Inc.
Almost all data today flows on networks, mostly used after collection, much of it never stored
2 Source: Noumenal
Disconnected
Milliseconds Minutes Hours+
Copyright Third Nature, Inc. Source: Noumenal
Milliseconds Minutes Hours+
Events and monitoring occur in different time frames, follow different cycles of use
3
Disconnected
Data lives in multiple places, at multiple levels of detail, for differing durations. Unlikely to all be in one place. Nor should it be.
Copyright Third Nature, Inc.
Streaming infrastructure, collection systems, big data, tend to be viewed as standalone systems
Most talk about this topic focuses on streams and programs living on the network.
Usually viewed as independent from the rest of data use in the organization.
The problem is that the live and persisted data are used together, or have dependencies of use.
Copyright Third Nature, Inc.
Monthly
Production plans
Weekly pre-
orders for
bulk cheese
Availability
confirmation
and location
In store system
Store
Stock
Management
Store EPOS
dataCategory
Supervis
or
Stock
adjustments/
order
interventions
Order
adjustment
Stock/order
interventions
*
*
Orders
(based on 6
day
forecast)
Dallas
Distrib Centre
WMS
Picking/load
teams
Pos/Pick
lists/Load
sheetsConfirmed
Deliveries/
Confirmed
picks +
loads
FarmersMilk intake/
silos Cheese plant
Plant
Processor
In-house Cheese
store
Contract Cheese
store
Processor
Packing plant
Processor
National
Distribution
Centre
Retailer
RDC
Retailer Stores
(550)
Retailer HQ
Consolidated
Demand
Ordering
Processor NDC
Customer
Services
Daily order -
SKU/Depot/
Vol
Sent @ 12.30-13:00
Delivery
orders
Processor HQ
Sales
Team/
Account
Manager
Processor HQ
Forecasting
Team
Processor HQ
Bulk Planning
Team
Cheese plant
Planner/Stock
office
Processor HQ
Milk
Purchasing
Team
Cheese plant
Transport
Manager
Actual
daily
delivery
figures
Daily
collection
planning
Weekly order for delivery to
Packing plant
Daily &
weekly Call-
off
Daily Call-off
15/day
22 pallet loads 15/day
A80
Shortages/
Allocation
instructions
Annual
Buying plan
Milk Availability
Forecast
Annual
prediction
of milk
production
Shortages/
Allocation
instructions
Daily milk
intake
Weekly milk
shortages
shortages
Spot mkt or
Processor
ingredients
Packing plant
Planning
Team
Processor HQ
JBA Invoicing
and
Sales Monitor
FGI and Last 5
weeks sales
Expedite
Changes
to existing
forecast -
exceptions
Retailer HQ
Retailer Buyer
Meeting
every 6
weeks
Packing plant
Cheese
ordering
10 day stock
plan
On line
stock info
7 day order
plan for bulk
cheese
Arrange
daily
delivery
schedule
Emergency
call-off
Daily
optimisation
of loads
Service
Monitor
Despatch and
delivery
confirmations
Processor NDC
Transport
Planning
Transport
Plan
Processor NDC
Inventory
MonitoringStock and
delivery
monitoring
Processor NDC
Warehouse
management
syatem
Operation
Instructions
Key
Shaded Boxes = Product flow system
Un-shaded boxes = Information flow system
Retailer Cheese ProcessorFarms
Schedule
weekly &
Daily
10 Day
plan(wed) and
daily plan
15/day
Changes
to existing
forecast -
exceptions
Stock
availabilityMonthly
review
Annual
f/cast
Source: IGD Food Chain Centre, February 2008
What about all the systems that already exist, built up over years?
A typical value chain diagram, showing the supply chain for cheese. Notice the extreme complexity of information exchanges (in red)
Copyright Third Nature, Inc.
Even within one company it’s complicated
Off-the-shelf software, custom applications, SaaS applications, logs, event streams, 3rd party datasets…
Copyright Third Nature, Inc.
Order Entry
Order Database
Customer Service
Integration Program
Inventory Database
Distribution
Integration Program
Receivables Database
Accounts Receivable
Data Warehouse
Analysts & users
Data is already moving between many applications but it isn’t streaming
Copyright Third Nature, Inc.
The shortcomings of the log-centered approach
Copyright Third Nature, Inc.
Supplement log-based models with non-intrusive collection
Copyright Third Nature, Inc.
Streaming infrastructure needs to enable reuse
Copyright Third Nature, Inc.
Image Attributions
Thanks to the people who supplied the images used in this presentation:
ignore_no_parking.jpg - http://www.flickr.com/photos/vermininc/2352068440
Streaming environment - http://www.noumenal.com/
phone booth red snow - http://www.flickr.com/photos/wwarby/5907617001/
lots of screens to watch – Cprbis stock photo
Slide 11
Copyright Third Nature, Inc.
About Third Nature
Third Nature is a research and consulting firm focused on new and emerging technology and practices in analytics, business intelligence, and performance management. If your question is related to data, analytics, information strategy and technology infrastructure then you‘re at the right place.
Our goal is to help companies take advantage of information-driven management practices and applications. We offer education, consulting and research services to support business and IT organizations as well as technology vendors.
We fill the gap between what the industry analyst firms cover and what IT needs. We specialize in product and technology analysis, so we look at emerging technologies and markets, evaluating technology and hw it is applied rather than vendor market positions.
