Boost your data analytics with open data and public news content

Boost Your Data Analytics with Open Data and Public News Content

Ontotext Webinar, 24 Mar 2016

Open Data & News Analytics #2

Presentation Outline – PART I

• Quick news-analytics case• Technology approach• FactForge-News: Data architecture• Sample queries on Linked Open Data• News analytics examples• Today’s News Map

Mar 2016

Quick news-analytics case

Mar 2016

• Our Dynamic Semantic Publishing platform already offers linking of text with big open data graphs

• One can get navigate from text to concepts, get trends, related entities and news

• Try it at http://now.ontotext.com

Presentation Outline

Mar 2016

Open Data & News Analytics

Our approach to Big Data

1. Integrate relevant data from many sources− Build a Big Knowledge Graph from proprietary databases and

taxonomies integrated with millions of facts of Linked Data

2. Infer new facts and unveil relationships− Performing reasoning across data from different sources

3. Interlink text and with big data− Using text-mining to automatically discover references to

concepts and entities

4. Use NoSQL graph database for metadata management, querying and search

Mar 2016 #5

NoSQL Graph Database

Mar 2016

myData: Maria

ptop:Agent

ptop:Person

ptop:Woman

ptop:childOf

ptop:parentOf

rdfs:range

owl:inverseO

inferred

myData:Ivan

owl:relativeOf

owl:inverseOfowl:SymmetricProperty

rdfs:subPropertyOf

owl:inverseOf

rdf:type

• The hottest NoSQL trend• W3C standards• Efficient Data Integration

− Using logical inference

− For data integration and BI

Analyzing Text

Mar 2016

• Full spectrum of NLP weaponry

• Semantic indexing− Tag references with entity IDs

− Generate semantic metadata descriptions of documents

− Store metadata in GraphDB

Mar 2016

The Web of Linked Data in 2007

Mar 2016 #9

structured database version of Wikipedia

database of all locations on Earth

product reviews

semantic synonym dictionary

Note: Each bubble represents a dataset. Arrows represent mappings across datasets; e.g. dbpedia:Paris owl:sameAs geo:2988507

The Web of Linked Data is Gaining Mass

Mar 2016 #10

The Web of Data is Gaining Mass (2011)

Mar 2016 #11

The Web of Linked Data is Gaining Mass

Mar 2016 #12

• 2013 stats: 2 289 public datasets− http://stats.lod2.eu/

• Growing exponentially − see the dotted trend line

• Structured markup− Schema.org; semantic SEO

• Enables better semantic tagging!− As there are more concepts and richer

descriptions to refer to

27 43 89 162295

2007 2008 2009 2010 2011 2012 2013

Linked Data Datasets

The FactForge Data

• DBpedia (the English version only): 496M statements

• Geonames: 150M statements− SameAs links between DBpedia and Geonames: 471K statements

• NOW data – metadata about news: 128M statements

• Total size: 938М statements− 656M explicit statements + 281M inferred statements

− RDFRank and geo-spatial indices enabled to allow for ranking and efficient geo-region constraints

Mar 2016

News Metadata

• Metadata from Ontotext’s Dynamic Semantic Publishing platform− Automatically generated as part of the NOW.ontotext.com semantic news showcase

• News corpus from Google since Feb 2015, about 10k news/month

• ~70 tags (annotations) per news article

• Tags link text mentions of concepts to the knowledge graph− Technically these are URIs for entities (people, organizations, locations, etc.) and key phrases

Mar 2016

News Metadata

Mar 2016

News Metadata

Mar 2016

Category Count International 52 074

Science and Technology 23 201Sports 20 714Business 15 155Lifestyle 11 684

122 828

Mentions / entity type Count Keyphrase 2 589 676Organization 1 276 441Location 1 260 972Person 1 248 784Work 309 093Event 258 388RelationPersonRole 236 638Species 180 946

News Geographic Coverage

Mar 2016

• Quite focused on USA!

Class Hierarchy Map (by number of instances)

Mar 2016

Left: The big pictureRight: dbo:Agent class (2.7M organizations and persons)

Mar 2016

Sample queries

• There is a rich set of sample queries that allow exploration of this combination of DBPedia, GeoNames and news metadata

• We will showcase few of those, starting from the simple once

• In bold we marked the “parameters” of the queires

Mar 2016

Query: Big Cities in Eastern Europe# benefits from inference over transitive gn:parentFeature# benefits from owl:sameAs mapping between DBPedia and Geonames

PREFIX dbr: <http://dbpedia.org/resource/>PREFIX onto: <http://www.ontotext.com/>PREFIX gn: <http://www.geonames.org/ontology#>PREFIX dbo: <http://dbpedia.org/ontology/>select *from onto:disable-sameAswhere { ?loc gn:parentFeature dbr:Eastern_Europe ; gn:featureClass gn:P. ?loc dbo:populationTotal ?population ; dbo:country ?country . FILTER(?population > 300000 )} order by ?country

Mar 2016

Query: People and Organizations related to Google# benefits from inference over transitive dbo:parent# RDFRank makes it easy to see the “top suspects” in a list of 93 entities

PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX rank: <http://www.ontotext.com/owlim/RDFRank#>PREFIX dbr: <http://dbpedia.org/resource/> select distinct ?related_entity ?rankwhere { BIND (dbr:Google as ?entity) { ?related_entity a dbo:Person ; ?p ?entity . } UNION { ?related_entity a dbo:Organisation ; dbo:parent ?entity . } ?related_entity rank:hasRDFRank ?rank} order by desc(?rank)

Mar 2016

Query: Airports near London# GraphDB’s geo-spatial plug-in allows efficient evaluation of near-by# RDFRank brings the top 6 passanger airports at the top of a list of 80

PREFIX dbr: <http://dbpedia.org/resource/>PREFIX geo-pos: <http://www.w3.org/2003/01/geo/wgs84_pos#>PREFIX gdb-geo: <http://www.ontotext.com/owlim/geo#>PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX gdb: <http://www.ontotext.com/owlim/>

SELECT distinct ?airport ?rrankWHERE { { SELECT * { dbr:London geo-pos:lat ?lat ; geo-pos:long ?long . } LIMIT 10 } ?airport gdb-geo:nearby(?lat ?long "50mi"); a dbo:Airport ; gdb:hasRDFRank ?rrank .} ORDER BY DESC(?rrank)

Mar 2016

Query: Top-level Industries by number of companies# benefits from mapping and consolidation of industry classifications# and predicates in DBPedia (ff-map)

PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX ff-map: <http://factforge.net/ff2016-mapping/>select distinct ?topIndustry (count(?company) as ?companies)where { ?company dbo:industry ?industry . ?industrySum ff-map:industryVariant ?industry . ?industrySum ff-map:industryCenter ?topIndustry .} group by ?topIndustry order by desc(?companies)

Mar 2016

Semantic Press-Clipping

• We can trace references to a specific company in the news− This is pretty much standard, however we can deal with syntactic variations in the names, because state

of the art Named Entity Recognition technology is used

− What’s more important, we distinguish correctly in which mention “Paris” refers to which of the following: Paris (the capital of France), Paris in Texas, Paris Hilton or to Paris (the Greek hero)

• We can trace and consolidate references to daughter companies

• We have comprehensive industry classification− The one from DBPedia, but refined to accommodate identifier variations and specialization (e.g.

company classified as dbr:Bank will also be considered classified as dbr:FinancialServices)

Mar 2016

Query: News Mentioning an IBM# technical example to demonstrate how news metadata can be accessed

PREFIX pub-old: <http://ontology.ontotext.com/publishing#>PREFIX pub: <http://ontology.ontotext.com/taxonomy/>PREFIX dbr: <http://dbpedia.org/resource/>PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

select distinct ?news ?title ?date ?pub_entity where { ?news pub-old:containsMention / pub-old:hasInstance ?pub_entity . ?pub_entity pub:exactMatch dbr:IBM . ?news pub-old:creationDate ?date; pub-old:title ?title . FILTER ( (?date > "2015-10-01T00:02:00Z"^^xsd:dateTime) &&

(?date < "2015-11-01T00:02:00Z"^^xsd:dateTime))} limit 100

Mar 2016

Query: News Mentioning Gazprom and Its Related Entities

# benefits from inference over transitive dbo:parent relation and mappings to it

select distinct ?news ?title ?date ?related_entity where { { select distinct ?related_entity { BIND (dbr:Gazprom as ?entity) { ?related_entity a dbo:Person ; ?p ?entity .

FILTER NOT EXISTS { ?related_entity dbo:club ?entity } } UNION { ?related_entity a dbo:Organisation ; dbo:parent ?entity . } UNION { BIND(?entity as ?related_entity) } } } ?news pub-old:containsMention / pub-old:hasInstance ?pub_entity . ?pub_entity pub:exactMatch ?related_entity . ?news pub-old:creationDate ?date; pub-old:title ?title .} order by desc(?date) limit 1000

Mar 2016

Query: Most Popular in the News Automotive Companies# benefits from mapping and consolidation of industry classifications

select distinct ?pub_entity (max(?entity_label) as ?label) (count(?news) as ?news_count)

where { ?news pub-old:containsMention / pub-old:hasInstance ?pub_entity . ?pub_entity pub:exactMatch ?entity; pub:preferredLabel ?entity_label. dbr:Automotive ff-map:industryVariant ?industry . ?entity dbo:industry ?industry . ?news pub-old:creationDate ?date .} group by ?pub_entity order by desc(?news_count)

Mar 2016

Query: Most Popular in the News, including children# benefits from mapping and consolidation of industry classifications

select distinct ?parent (count(?news) as ?news_count)where { { select distinct ?parent ?entity { BIND(dbr:Software as ?industry) ?industry ff-map:industryVariant ?industryVar . ?parent dbo:industry ?industryVar . ?parent a dbo:Company . FILTER NOT EXISTS { ?parent dbo:parent / dbo:industry / ff-map:industryVariant ?industry } { ?entity dbo:parent ?parent . } UNION { BIND(?parent as ?entity) } } } ?news pub-old:containsMention / pub-old:hasInstance ?pub_entity . ?pub_entity pub:exactMatch ?entity . ?news pub-old:creationDate ?date .} group by ?parent order by desc(?news_count)

Mar 2016

News Popularity Ranking: Automotive

Mar 2016

Rank Company News # Rank Company incl. mentions of controlled News #1 General Motors 2722 1 General Motors 46202 Tesla Motors 2346 2 Volkswagen Group 39993 Volkswagen 2299 3 Fiat Chrysler Automobiles 26584 Ford Motor Company 1934 4 Tesla Motors 23705 Toyota 1325 5 Ford Motor Company 21256 Chevrolet 1264 6 Toyota 16567 Chrysler 1054 7 Renault-Nissan Alliance 13328 Fiat Chrysler Automobiles 1011 8 Honda 8649 Audi AG 972 9 BMW 715

10 Honda 717 10 Takata Corporation 547

News Popularity: Finance

Mar 2016

Rank Company News # Rank Company incl. mentions of controlled News #1 Bloomberg L.P. 3203 1 China Merchants Bank 409402 Goldman Sachs 1992 2 Alphabet Inc. 242193 JP Morgan Chase 1712 3 Capital Group Companies 43794 Wells Fargo 1688 4 Bloomberg L.P. 38935 Citigroup 1557 5 Exor (company) 27756 HSBC Holdings 1546 6 JP Morgan Chase 27157 Deutsche Bank 1414 7 Nasdaq, Inc. 21788 Bank of America 1335 8 Oaktree Capital Management 17579 Barclays 1260 9 Goldman Sachs 1085

10 UBS 694 10 Sentinel Capital Partners 1064

Note: Including investment funds, stock exchanges, agencies, etc.

News Popularity: Banking

Mar 2016

Rank Company News # Rank Company incl. mentions of controlled News #1 Goldman Sachs 996 1 China Merchants Bank * 382882 JP Morgan Chase 856 2 JP Morgan Chase 19723 HSBC Holdings 773 3 Goldman Sachs 10304 Deutsche Bank 707 4 HSBC 9665 Barclays 630 5 Bank of America 7716 Citigroup 519 6 Deutsche Bank 7427 Bank of America 445 7 Barclays 6818 Wells Fargo 422 8 Citigroup 6309 UBS 347 9 Wells Fargo 428

10 Chase 126 10 UBS 347

Note: including investment funds, stock exchanges, agencies, etc.

Mar 2016

Today’s News Map: Business

Mar 2016

Today’s News Map: International

Mar 2016

Expect in Part II

• Mentions of entity and related by month

• Most relevant co-occurrnig entities

• Most relevant co-occurrnig entities per month

• Related News

• and more

Mar 2016

Thank you!

Experience the technology with NOW: Semantic News Portalhttp://now.ontotext.com

Start using GraphDB and text-mining with S4 in the cloudhttp://s4.ontotext.com

Learn more at our website or simply get in touch info@ontotext.com, @ontotext

Mar 2016

Boost your data analytics with open data and public news content

Data & Analytics

HOW TO USE LINKEDIN ANALYTICS TO BOOST YOUR MARKETING EFFORTS

Boost Your Data Career with Predictive Analytics! Learn How ?

Decoding web analytics to boost conversion

Web Strategies to Boost Donor and Volunteer Engagement Using Google Analytics

Data Analytics | Big Data Analytics Services - Phdassistance.com

Analytics & Data Warehousing Reader Surveydocs.media.bitpipe.com/io_10x/io_102267/item_835376/Analytics Da… · Analytics & Data Warehousing Reader ... IT/BI/analytics/data warehousing/data

How data analytics could boost financial industry · Based on Oracle Financial Services Analytical Applications Data Requirements. ... Optimizing Finance Operations Customer Insight

Data Analytics with MATLAB - itpcas.ac.cnlib.itpcas.ac.cn/documents/18/0/Data+Analytics+MATLAB.pdfMATLAB Analytics work with business and engineering data 1. 6 Data Analytics Workflow

How to Use Social Media Analytics to Boost ROI

DATA ANALYTICS Big Data & Analytics

Video analytics and facial recognition to boost security and congestion management

Embracing Analytics To Boost Profits And Loyalty

Andrey Sverdlov, "How analytics can boost ecommerce"

MSc Data Analytics. - UCLan Cypruscomputing.uclancyprus.ac.cy/factsheets/MScData... · such as Data Analytics Business Consultant, Data Analytics Architect, Data Engineer, Data Solution

Data Analytics€¦ · Data Analytics CS301 Introduction to Data Analytics Week 1: 1st Sept Fall 2020 Oliver BONHAM-CARTER. Data Data Data Data Data Data Data Data Analytics CMPSC*301

Data and Analytics Boost Your Business Performance

Integrating Email and Web Analytics Data to Boost ......Integrating Email and Web Analytics Data to Boost Conversion Rates Matthew Kritzer Director of Ecommerce, L’Occitane en Provence

Idiro Analytics - Analytics & Big Data

Data Analytics and Visualization Workshop … · 09/04/2019 · Data Analytics • Data Visualization • Data Science • Big Data • Predictive Analytics • Machine Learning

How Advanced Analytics Can Boost Your Commercial Profitability€¦ · HOW ADVANCED ANALYTICS CAN BOOST YOUR COMMERCIAL PROFITABILITY FINANCE THE DATA DIFFERENCE To more clearly illustrate