Machine learning by exampleMichał Matłoka @mmatloka

Outline1. What is Machine Learning?2. Learning types3. Use cases4. Example5. What can be improved?

Can machines think?(Alan Turing 1950)

What is ML?● Make machines “think” like

humans

● Learn from data and make

predictions

Learning types

● Supervised learning

○ Classification

○ Regression

● Unsupervised learning

○ Clustering

○ Dimensionality Reduction

● Semi-supervised learning

● Reinforcement learning

○ E.g. AlphaGo

Use cases

● Voice recognition

● Fraud analysis

● Face detection

● Ads click-through rate prediction

● Spam detection

● Shop recommendations

● Photos description

● Self-driving cars

● Healthcare

● ...

Learning Process(Classification)

1. Data gathering

2. Data cleaning & feature

extraction

3. Dataset -> training & test set

4. Learning -> Model

5. Evaluation -> Accuracy

6. New observation -> Prediction

Example

Apache SparkRDD (Resilient Distributed

Dataset)

DAG (Directed acyclic graph)

● RDD - map, filter, count etc

● Spark SQL

● MLib

● GraphX

● Spark Streaming

● API: Scala, Java, Python, R*

Classify conference talk abstracts into tracks

https://github.com/mmatloka/machine-learning-by-example

What can be improved?

● Bigger data set

● Smarter tokenizer

● Stemming & lemmatization

● IDF - Inverse Document

Frequency

● Cross-validation

● Parameters tuning

Articles & references

● https://www.csee.umbc.edu/courses/471/papers/turing.pdf

● http://spark.apache.org/

● https://databricks.com/try-databricks

● https://research.googleblog.com/2016/09/introducing-open-images-dataset.html

● https://www.kaggle.com/

● https://github.com/dylanmei/docker-zeppelin

● https://github.com/databricks/spark-corenlp

Thank you, Q&A?@mmatloka

http://www.slideshare.net/softwaremillhttps://softwaremill.com/blog/