Embed Size (px)
Citation preview
‹#›
DRIVEN
Rob Mueller, Ben MurrayJune 24th 2015
TOINNOVATE
‹#›
NASCAR StoryArchitectureHP HavenIntegrationQ&A
NASCAR StoryArchitectureHP HavenIntegrationQ&A
Agenda
‹#›
NASCAR Environment
1948 – Bill France Sr. Creates NASCAR, the National Association for Stock Car Auto Racing
“Stock car racing has distinct possibilities for Sunday shows and we do not know how big it can be if it’s handled properly….”
- Bill France Sr.
The National Association for Stock Car Auto Racing
‹#›
…and Red Hat
Technology…
‹#›
‹#›
NASCAR Objectives
To be the first of its kind in the industry (i.e. Social and Traditional Media Monitoring), acting as a “state of the art” technical showpiece to garner attention from the technology sector and global sports properties
Establishing a more informed picture of both fan demographic and engagement with the sport of NASCAR
Near-time fan engagement though the monitoring and collation of data from social media, traditional media, broadcast, and in-house sources
Management of social profile and perception, delivering actionable insights and initiatives, to better engage fans, and meet strategic business objectives
‹#›
Event ManagementRaceTV spotPress conference
Incident ManagementRace – unforeseen circumstancesUnscheduled celebrity appearanceBreaking news
AnalyticsEventIncidentEntity
Near time
Retrospective
SentimentShare of voiceTrendsMessage volumesInfluenceGeographic profilingHot and breaking
Sentiment profilingMessage pull-throughComparative profiling
+
NASCAR Use Case
‹#›
Unstructured to structured
Near Time
‹#›
Source dataConstantly evolving and loose use of languageIndustry specific languageDisparate datasets and nomenclatureImplied contextAbbreviations, nicknames, mnemonics etc.SlangSarcasm and IronyProfanityEmoticonsEtc…
NormalizationSentiment Grammars
‹#›
Actionable analytics
Better ROI for Sponsors
Increased Fan Engagement
Increased Sales
Understanding
‹#›
NASCAR StoryArchitectureHP HAVEnIntegrationQ&A
Agenda
NASCAR StoryArchitectureHP HAVEnIntegrationQ&A
‹#›
EIP
‹#›
NASCAR logical
‹#›
System PathReference to Datasift needs to be removed
‹#›
Distributionremove datasift reference
‹#›
Broker View
remove the broker0 arrows pointing to the server2
‹#›
<networkConnector name="Prod1AMQ0toProd2AMQ0-2" networkTTL="1" duplex="false" uri="static:(tcp://10.221.9.32:62700)" conduitSubscriptions="false" >
<networkConnector name="Prod1AMQ0toProd2AMQ0-3" networkTTL="1" duplex="false" uri="static:(nio://10.221.9.32:62800)" conduitSubscriptions="false" >
<dynamicallyIncludedDestinations> <queue physicalName="preQ"/>…
AMQ0/ESB1 Server1 to AMQ0/ESB1 Server2 Load Balancing Network Connector
‹#›
CheckPoint service
‹#›
NASCAR StoryArchitectureHP HavenIntegrationQ&A
Agenda
NASCAR StoryArchitectureHP HavenIntegrationQ&A
‹#› (1) IDC Directions 2013: Why the Datacenter of the Future Will Leverage a Converged Infrastructure, March 2013, Matt Eastwood ; (2) & (3) IDC Predictions 2012: Competing for 2020, Document 231720, December 2011, Frank Gens; (4) http://en.wikipedia.org (a) Data Never Sleeps Domo.com (b) Deloitte Tech, media, and telecoms predictions 2014
Now By 2020
Volume, Velocity, Variety, Value
2,460,000 content shares (a)
277,000 tweets (a)
4,000,000 Google searches (a)
72 hours of video uploaded (a)
216,000 Instagram posts (a)
204,000,000 emails sent (a)
34,700,000 instant messages (b)
Every 60 seconds
40 Trillion(2)
10 Million(3)
… for 8 Billion(4)
30 Billion(1)
DATA
Mobile Apps
Devices
GB
HP Haven
Big Data Platform
‹#›
Relevant Data
Time to decision
1980 1990 2000 2010 2020
HP Haven
“ Current enterprise environments are focused on structured data. However, 90% of digital content created by 2015 will be of mixed data types” 1
¹Source: IDC Predictions 2012: Competing for 2020
‹#›
Contextualsearch
Data Normalization
Image/video analytics Geospatial
analytics
SQL on HadoopAccelerated
analyticsSentimentanalysis
Predictiveanalytics
And more...
On-premise On Demand
Haven Big Data platformCore big data business capabilities
Access Explore Enrich Analyze Predict Serve Act
HP Haven
‹#›
“HP FMEC, takes fan data, collects, it stores, and stitches it together…that helps us understand what is being talked about across the ecosystem of the sport.”-- Senior Director of IT, NASCAR
HP Haven in ActionPowering Real-time, multichannel fan engagementHaven solution:Hadoop + HP Vertica+ JBoss +HP IDOL + HP Enterprise Services
Data source:Twitter, Facebook, Google+, Instagram, Reddit, Tumblr, YouTube, Blogs, RSS, Corporate Data, etc…
Results:Understands sentiment, identifies emerging issues, uncovers trends that helps the NASCAR industry increase value for stakeholders, and enriches the fan and broadcast experience
‹#›
HP Haven in Action
‹#›
HP Haven in Action
Understand Market reactions and brand perceptionIdentify key buying factors and/or customer experiencesSegment populations to customize actionsEnable experimentationPredict outcomesEnable new business models
‹#›
NASCAR StoryArchitectureHP HavenIntegrationQ&A
Agenda
NASCAR StoryArchitectureHP HavenIntegrationQ&A
‹#›
Distributionremove datasift reference
‹#›
Broker View
remove the broker0 arrows pointing to the server2
‹#›
<import resource="file:${jboss.server.base.dir}/configuration/nascarAMQ.xml" />
<camelContext id="DS-HTTP4Consumer" errorHandlerRef="myDeadLetterErrorHandler" useMDCLogging="true“
<propertyPlaceholder id="properties" location="file:/opt/rh/esb/data/property/dataSift.properties, file:/opt/rh/esb/data/property/recordRace.properties"/>
<onException>…</onException>
<route id="HTTP-‐Ingest-‐Route" autoStartup="false" streamCache="true" trace="true"> <from uri="timer://dsHTTP4Timer?repeatCount=0"/> <to uri="http4://api.ingest.com/v1/pull?httpClient.soTimeout=30000&throwExceptionOnFailure=false& id={{streamId}}…"/>
<choice> <when>
<simple>${header.CamelHttpResponseCode} == "200"</simple> <convertBodyTo type="java.lang.String"/> <to uri="seda:sedaToQ"/> <log message = " %%%%%%%%%% HTTP=[${header.CamelHttpResponseCode}] ::: " loggingLevel = "INFO"/> <to uri="log:response200?level=INFO&groupDelay=500&groupSize=100"/> <delay><constant>500</constant></delay>
</when>
Media Ingest
‹#›
<route id="sedaSendToPreQ"> <from uri="seda:sedaToQ"/> <to uri="activemq:queue:preQ?disableReplyTo=true&allowNullBody=false& acceptMessagesWhileStopping=true& disableReplyTo=true"/> <to uri="activemq:queue:{{recordRace.fileName}}Q?disableReplyTo=true& allowNullBody=false&acceptMessagesWhileStopping=true& disableReplyTo=true"/>
</route>
Media Ingest - Seda Queue
‹#›
<import resource="file:${jboss.server.base.dir}/configuration/nascarAMQ.xml" />
<camelContext id="DS-HTTP4Consumer" errorHandlerRef=“deadLetterErrorHandler" useMDCLogging="true“
<propertyPlaceholder id="properties" location="file:/opt/rh/esb/data/property/dataSift.properties, file:/opt/rh/esb/data/property/recordRace.properties"/>
<onException>…</onException>
<route id="HTTP-‐Ingest-‐Route" autoStartup="false" streamCache="true" trace="true"> <from uri="timer://dsHTTP4Timer?repeatCount=0"/> <to uri="http4://api.ingest.com/v1/pull?httpClient.soTimeout=30000&throwExceptionOnFailure=false& id={{streamId}}…"/>
<choice> <when>
<simple>${header.CamelHttpResponseCode} == "200"</simple> <convertBodyTo type="java.lang.String"/> <to uri="seda:sedaToQ"/> <log message = " %%%%%%%%%% HTTP=[${header.CamelHttpResponseCode}] ::: " loggingLevel = "INFO"/> <to uri="log:response200?level=INFO&groupDelay=500&groupSize=100"/> <delay><constant>500</constant></delay>
</when>
Media Ingest - deadLetterErrorHandler
‹#›
NASCAR AMQ Client Configuration - Ingest<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<bean id=“deadLetterErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder"> <property name="deadLetterUri" value="activemq:queue:dlqIngestQ"/> <property name="redeliveryPolicy" ref="myRedeliveryPolicyConfig"/>
</bean>
<bean id="myRedeliveryPolicyConfig" class="org.apache.camel.processor.RedeliveryPolicy"> <property name="maximumRedeliveries" value="0"/> <property name="redeliveryDelay" value="5000"/>
</bean>
‹#›
<import resource="file:${jboss.server.base.dir}/configuration/nascarAMQ.xml" />
<camelContext id="DS-HTTP4Consumer" errorHandlerRef=“deadLetterErrorHandler" useMDCLogging="true“
<propertyPlaceholder id="properties" location="file:/opt/rh/esb/data/property/dataSift.properties, file:/opt/rh/esb/data/property/recordRace.properties"/>
<onException>…</onException>
<route id="HTTP-‐Ingest-‐Route" autoStartup="false" streamCache="true" trace="true"> <from uri="timer://dsHTTP4Timer?repeatCount=0"/> <to uri="http4://api.ingest.com/v1/pull?httpClient.soTimeout=30000&throwExceptionOnFailure=false& id={{streamId}}…"/>
<choice> <when>
<simple>${header.CamelHttpResponseCode} == "200"</simple> <convertBodyTo type="java.lang.String"/> <to uri="seda:sedaToQ"/> <log message = " %%%%%%%%%% HTTP=[${header.CamelHttpResponseCode}] ::: " loggingLevel = "INFO"/> <to uri="log:response200?level=INFO&groupDelay=500&groupSize=100"/> <delay><constant>500</constant></delay>
</when>
Media Ingest - onException
‹#›
<onException> <exception>net.sf.saxon.trans.XPathException</exception> <exception>org.apache.camel.language.bean.RuntimeBeanExpressionException</exception> <exception>org.apache.camel.builder.xml.InvalidXPathExpression</exception> <exception>java.net.SocketTimeoutException</exception> <handled> <constant>true</constant> </handled> <pipeline> <setHeader headerName="goingToJMSQ"> <simple>${routeId}</simple></setHeader> <setHeader headerName="Exception"><simple>${exception.message} </simple></setHeader> <setHeader headerName="Exception.Stacktrace"> <simple>${exception.stacktrace}</simple></setHeader> <log message="exception handled from [${routeId}] with error ${header.message}" loggingLevel="ERROR"/> <to uri="activemq:queue:dlqQ?disableReplyTo=true"/> </pipeline> </onException>
onException- Ingest
‹#›
Audit services
‹#›
Replay Service
‹#›
NASCAR StoryArchitectureHP HAVEnIntegrationQ&A
Agenda
NASCAR StoryArchitectureHP HAVEnIntegrationQ&A
‹#›
Q&A
‹#›
Facts about this projectDuration to Production: < 6 months
Server Types:
JBOSS: EAP6.3, StandAlone
ActiveMQ: AMQ5.9.0, Kahadb
Hawtio: 1.48, The One to Rule Them All
Packaging: War
need the server types
‹#›
Facts about this projectCamel/EIP/Features:
Components: Cache, jdbc, sql, xmlJson, HTTP4, SEDA, JMS, ActiveMQ, Multicast, Enrich, Aggregator, Recipient List, JMSGroupId, JMX, Jsoup, Quartz, log4J, SMTP, RSS/ATOM, Swagger, Saxon, Jackson, Servlet, Xstream,
Languages: Spring DSL, Header Language, File Language, Simple, Groovy, JavaScript, Java Beans, Xquery, regex, Python
Error Handlers: onException, deadLetterErrorHandler
Integration Points: Media Ingest, Vertica, Autonomy, SQL Svr, Hadoop, File, Queues, Media Bin, Streams
need the server types
‹#›
Facts about this projectDuration to Production: < 6 monthsLevel of development expertise at the start:
Camel Architect: Six Months
Spring DSL: Three Months
Integration Architect: 25 Years
SOA Architect: 20 Years
Three Java Developers: 10 years Total
JMS: 20 Years
‹#›
Most irritating moment– Looking online for answers and examples. Invariably you’ll see the dreaded Claus Ibsen quote; “refer to the documentation…”
‹#›
Most irritating moment
- what’s irritating is… he’s usually correct.
‹#›
Facts about this projectMost important moment –
Camel instills a “Route” thought process. It ensures you are thinking of the message at all times.
‹#›
Facts about this projectBest Quote Found while researching.
“The only reason for time is so that everything doesn't happen at once”, Albert Einstein
‹#›46
• Storage o 3PAR 7200 installed and configured o Brocade switches installed and configured in C7000 enclosures o Zoned to all HP server blades o FMEC hosts defined on 3PAR
• Servers o 9 server blades installed and configured in c7000 enclosures o VMware ESXi 5.5 has been installed on all 9 server blades o There are 11 VMs and 9 hosts o 4 RHEL 6.5 physical servers
Facts about this project
