SF Data Science: Developing Data Products

Developing Data ProductsSF Data Science MeetupPete Skomoroch @peteskomorochSeptember 19 2013

Developing Data ProductsExamples, Techniques, & Lessons Learned

Our MissionConnect the world’s professionals to make them

more productive and successful.

Our VisionCreate economic opportunity for every

professional in the world.

Members First!

LinkedIn is the leading professional network site

Worldwide Workforce

3,300M+2

Worldwide Professionals

640M+2

LinkedIn Members238M+

LinkedIn profiles represent our professional identity

238MMembers 238M MemberProfiles

We have a lot of data.

We have a lot of data.And (like everyone else), we store it in Hadoop.

We have a lot of data.And (like everyone else), we store it in Hadoop.And people build awesome things with that data.

What do we mean by data products?

Building products from data at LinkedIn

A few examples:

People You May Know Skills and Endorsements Year in Review Network Updates Digest InMaps Who’s viewed my profile Collaborative Filtering Groups You May Like and more…

Collaborative Filtering: LinkedIn Skill Pages

Classification: giving structure to unstructured data

Extract

Clustering & Disambiguation

De-duplication and Normalization

Network Algorithms: Relevance & Ranking

Prediction: Personalized Skill Recommendations

Skill Endorsements: Over 2 Billion and Growing

Social Proof and the Skill Endorsement Graph

The Economic Graph: Skills, Jobs, People, Locations…

TimeLocation

Lessons learned developing data products

Collect the right data at the right time

Large amounts of data can reveal new patternsP

Time since graduation

Be wary of “black-box” approaches

Look at your data

Aggregate statistics can be misleading

Build a viewer app, “micro-listen”

Algorithmic intuition: include data geeks in design

OODA: Think like a jet fighter

OODA: Observe, Orient, Decide, Act

OODA: The speed you can move determines victory

Red teaming: what can go wrong likely will

Error data is valuable, analyze it and adapt

Conclusion: tips for developing data products

Collect the right data at the right time Large amounts of data can reveal new patterns Be wary of “black box” approaches Look at your raw data Aggregate statistics can be misleading Build and use viewer apps Include data geeks in design process OODA: Think like a jet fighter Red-teaming: anticipate edge cases Find opportunity in your error data

Questions?

@peteskomoroch

SF Data Science: Developing Data Products

Technology

SF Data Description as of January 2014

WE'VE GOT THE SPACE. · 2020. 6. 2. · cort 2,980 sf 4,962 sf 2,24 sf 2,42 sf 2,42 sf 3060 sf 4,72 sf 6,503 sf 4,288 sf 2,035 sf 3209 sf 2,800 sf 5,377 sf 9,20 sf 5,986 sf 3532 sf

4 Developing data elements

Data that persuades: Annotated - from SES SF 2012

Developing Data Products

Sf data mining_meetup

Velox at SF Data Mining Meetup

Executive Education Planner 2020 · DST-SF NICF- Data Storytelling (SF) 3.0 2700.00 29 6 27 1 27 12 16, 21 5 30 FEAIOT NICF- Feature Engineering and Analytics using IOT Data (SF)

Downtown SF Lucene/Solr Meetup: Developing Scalable Search for User Generated Content at PlayStation

9127 SOUTH JAMAICA ST...9127 SOUTH JAMAICA STREET ENGLEWOOD, CO 80112 LISTING DATA AVAILABLE SF: Suite 100 16,143 SF Suite 150 6,669 SF Suite 250 8,332 SF Suite 300 11,696 SF Total

Downtown SF Lucene/Solr Meetup: Developing Scalable User Search for PlayStation 4

Keynote at Big Data Tech Con SF 2014

THE VINEYARDS - LoopNet...993 sf 2,200 sf 1,100 sf 2,199 sf 1,23 sf 3,163 sf 1,760 sf 21,714 sf 22,000 sf 19,000 sf 1,200 sf 3,200 sf 1,600 sf 4,000 sf 2,000 sf 11,500 sf 19,200 sf

BizSpark SF Lightning Talk: "Developing For Azure" by Clayton Peddy

Developing Evidence Gathering Data

Cloud Native Data Pipelines (DataEngConf SF 2017)

adtech SF 2012 Startup spotlight data and targeting magnetic

Big Data and Mobile Analytics - MMA SF Jan13

Sf Europe Outstanding Addendum Data Tables Eur Afme Sifma

TRIANGLE SHOPPING CENTER - LoopNet...,050 100 7,585 SF SF SF SF SF SF SF SF SF SF SF SF SF SF SF SF SF SF 1 1 1 1 1 1 1 1 1 1 ,050 ,695 ,456 ,975 ,930 ,650 ,625 ,500 ,543 ,500 89 2,344