18
Walnut ai What we learnt from processing 100k job postings with NLP and WordEmbedding Methods beta

Walnut AI - Empowering a Smarter Workforce

Embed Size (px)

Citation preview

Page 1: Walnut AI - Empowering a Smarter Workforce

Walnut aiWhat we learnt from processing 100k job postings with NLP and WordEmbedding Methods

beta

Page 2: Walnut AI - Empowering a Smarter Workforce

Evolution

Peak Hanging-in Barely skill relevance of professionals

Page 3: Walnut AI - Empowering a Smarter Workforce

Opportunity

Most people don't know how

they can do better

Page 4: Walnut AI - Empowering a Smarter Workforce

Does this problem exist

ask for advicefriends

mentorsmanagers

groups

Page 5: Walnut AI - Empowering a Smarter Workforce

How do we solve this problem

what are people talking

about

What are companies asking for

What are companies hiring for

Data

CommunitiesJob Postings Resumes

Page 6: Walnut AI - Empowering a Smarter Workforce

Product Manager [Masked Company Name], Inc. - Redwood City, CA 94063

[Masked Company Name] connects people globally around shared video content. Think online living room - watch together, completely in sync, and group video chat at the same time. We’re looking for a Product Manager to help us grow the product, our user base and ultimately, [Masked Company Name].

You will work directly with a team of engineers and designers to improve existing products and develop new ones. You need to be able to think holistically about user challenges across multiple platforms, develop that into an actionable roadmap and execute.

Our lean team is growing quickly, and you’ll have the opportunity to make a huge impact at Rabbit.

Who we’re looking for:

3+ years product management experience in consumer products or applicationsProven analytical skills and the ability to make data-driven decisionsA test-measure-iterate approachA creative problem solver with a “get it done” attitudeExcellent communication and project management skillsBonus points: experience managing web and Android applications, design or marketing experience, an appreciation for puns, growth team experienceJob Type: Full-time

Required experience:

product management: 3 years

Lets look at some data

Page 7: Walnut AI - Empowering a Smarter Workforce

Product Manager [Masked Company Name], Inc. - Redwood City, CA 94063

[Masked Company Name] connects people globally around shared video content. Think online living room - watch together, completely in sync, and group video chat at the same time. We’re looking for a Product Manager to help us grow the product, our user base and ultimately, [Masked Company Name].

You will work directly with a team of engineers and designers to improve existing products and develop new ones. You need to be able to think holistically about user challenges across multiple platforms, develop that into an actionable roadmap and execute.

Our lean team is growing quickly, and you’ll have the opportunity to make a huge impact at Rabbit.

Who we’re looking for:

3+ years product management experience in consumer products or applicationsProven analytical skills and the ability to make data-driven decisionsA test-measure-iterate approachA creative problem solver with a “get it done” attitudeExcellent communication and project management skillsBonus points: experience managing web and Android applications, design or marketing experience, an appreciation for puns, growth team experienceJob Type: Full-time

Required experience:

product management: 3 years

How do we read data

Page 8: Walnut AI - Empowering a Smarter Workforce

How to compute text

the quick brown fox jumped over the lazy dog([the, brown], quick), ([quick, fox], brown), ([brown, jumped], fox), ...

Cat 010000110110000101110100011010110110100101110100011101000110010101101110

Kitten

Text Decimal

CBOWSkipGram

https://www.tensorflow.org/tutorials/word2vec/

Page 9: Walnut AI - Empowering a Smarter Workforce

Data Sources

Cleanse, Remove Stop Words

Generate Bigrams/Trigrams ..

UnSupervised Learning

Plot Phrases

cvUnSupervised Learning

cvUnSupervised Learning

cvUnSupervised Learning

cvUnSupervised Learning

develop new productsdata-driven decision

'to','of','by','for','will','w

0.5474381446838379

Lets crunch thisScrapy, Python, Beautifulsoup

Nltk, Python

Gensim, Nltk, Python

Gensim Word2VecTensorFlow

Matplotlib, Sklearn

Page 10: Walnut AI - Empowering a Smarter Workforce

What the !!!

Page 11: Walnut AI - Empowering a Smarter Workforce

yeah that makes sense

Page 12: Walnut AI - Empowering a Smarter Workforce

CoreProduct mgmt

Skills

Experience Years

Noise

Lets get a pen and paper

Page 13: Walnut AI - Empowering a Smarter Workforce

Refine Keywords

Keyword freq/region

Merge

Publish

cvUnSupervised Learning

cvUnSupervised Learning

cvUnSupervised Learning

Meta data +

Placing this in context

Page 14: Walnut AI - Empowering a Smarter Workforce

Early ResultsProduct Managers

Backend Engineers Software Engineers

Software Engineering Mgrs

Page 15: Walnut AI - Empowering a Smarter Workforce

Make this data work

https://m.me/Walnutbot

Page 16: Walnut AI - Empowering a Smarter Workforce

Summary• What did we learn

• Know what you intend to learn• Know your data sources [Open, Free, Paid]• Unigrams are useless unless they are proper nouns• Reuse learnt n-grams, continuously add to stop words phrases• Ask real people if data makes sense

• Whats next

• More data sources+ Meta data• Multiple Models exist for the same role, discover them• Delta between models, insights lie in delta• Combine insights with Meta data

Page 17: Walnut AI - Empowering a Smarter Workforce

Evolution

skill relevance of professionals

Augmented Intelligence

Stay RelevantProgressTransform

Page 18: Walnut AI - Empowering a Smarter Workforce

Thank You

Chandan Maruthi

https://www.linkedin.com/in/chandanmaruthiWalnutAI