Embed Size (px)
Citation preview
Kirill: This is episode number 99 with AI Learner Tuatini Godard.
(background music plays)
Welcome to the SuperDataScience podcast. My name is Kirill
Eremenko, data science coach and lifestyle entrepreneur.
And each week we bring you inspiring people and ideas to
help you build your successful career in data science.
Thanks for being here today and now let’s make the complex
simple.
(background music plays)
Welcome ladies and gentlemen back to the
SuperDataScience podcast. Today I've got a super inspiring
guest on the show, Tuatini Godard. Tuatini is my friend from
France, and the way we met was when Hadelin and I were
on our European roadtrip. Tuatini came to our catchup
there with students and it was so inspiring to hear him talk.
So one thing you need to know about Tuatini is that he used
to be a software engineer, an Android developer to be exact,
and he actually intentionally quit his job and became
unemployed just to pursue his passion in learning about AI.
How inspiring is that? When I heard that story, I was so
thrilled and so excited for him.
So you will hear him say in the podcast that he's actually
surviving on his unemployment benefits but he's extremely
happy that he has all the time that he needs to pursue his
passion in learning about AI and deep learning. And the
confidence that he has that the opportunities will come out
of it, that he'll just learn what he's passionate about and the
opportunities will come is extremely encouraging and
refreshing as well.
So in this podcast, we'll discuss quite a lot of different
things. We'll talk about how Tuatini structures his day since
he doesn't have to go to work. What does he do, and how
does he make sure that he does follow his passion in
learning and he pursues that. And we'll talk about the
Pomodoro time management method which he uses, and
actually I was using, even though I didn't know about it,
back when I was starting my educational courses.
We'll talk about what he's learning in artificial intelligence,
what the recent developments are, and we'll talk about deep
learning and why he's chosen to go into that path. And we'll
even cover off some of the major recent technological
breakthroughs in terms of GPUs and Tuatini will give us a
quick excurse into that world as well.
So, all in all, a very varied podcast, lots of interesting things
that we're going to discuss, and I can't wait for you to check
it out. And without further ado, I bring to you Tuatini
Godard.
(background music plays)
[Speaking in French]
Kirill: Oh man, that was so cool. Just for everybody, back into
English, we met in Paris, right? Tuatini, when Hadelin and I
were on the roadtrip. It was a pretty cool catch up.
Tuatini: Yeah, it was really amazing to have dinner with you two
guys. Really interesting stories we told each other.
Kirill: Yeah. And for those of you who don't know French, that's
exactly how I felt. Like how you felt just now at the start of
the podcast was exactly how I felt because I know just a
little bit of French, and these guys were just talking French
the whole night long, right Tuatini?
Tuatini: Yeah, yeah, but actually you told me that I articulate well, so
you can understand what I am saying, so I thought it would
be ok!
Kirill: Oh yeah, that's right! That's right, out of all people there, I
could understand you the best. It's interesting, because
everybody was talking at about the same speed, but you just
articulate your words so well in French that I could
understand maybe like 80% of what you were saying. It was
really cool.
Tuatini: Oh, nice. Thank you.
Kirill: Yeah. But anyway, welcome to the podcast. So Tuatini
Godard is an AI learner. So tell us a bit about yourself,
Tuatini. What do you do for your free time, and what do you
do in life right now?
Tuatini: Currently I am unemployed. By choice, rest assured. And I
spend all my days actually learning everything I can around
data science, and especially deep learning. Most of the time,
when I have to get out of my house, I go to meetups and
meet some interesting people, like when I met you and
Hadelin in Paris. So basically, that's it.
Kirill: And this is really interesting, because I just found out just
before the podcast that you actually had a job at a startup
and then you decided to quit it because you wanted to spend
100% of your time on learning. Is that right?
Tuatini: Actually, I left the company almost 1 year ago and that was
a tough choice because when you are in your comfort zone,
it’s really hard to tell yourself that there is a new field which
is not related to what you do currently and you want to go
into that field. Basically, at that time I had multiple choices.
The first one was to stay in the company. And I really liked
the company and the team I had there. At the same time, I
wanted to get involved in AI. So, I went to my employer and I
asked him if he wanted to stop the contract I had with him
and then as of today I am living on my unemployment
benefits and more like surviving on my unemployment
benefits than living with it. But still, as of today, I’m doing
something I really like, even if it’s hard. Because there is a
lot of stuff in data science that I didn’t know, like everything
around mathematics, so that’s really hard for me. Still, I
really like what I do today. Even if I don’t have a good salary
or good co-workers, still that’s the most important to me – to
be able to do what you really want.
Kirill: That’s so cool. Is this something like you’re happy doing
what you’re doing, or are you building up your skills and
knowledge so that later on you can either start your own
business or get a very exciting job, or is this something that
you want to continue doing, just having fun with deep
learning on your own?
Tuatini: Actually, I want to make a living with deep learning. Because
I left my job and I don’t want to spend two years learning
deep learning and then going back to software engineering
and development because this is what I was doing in that
startup. So, yeah, as of today I’m learning everything I can.
And I don’t have any clear goal, but I take opportunities as
they come. I meet people. I try to get involved in open source
projects if I can, and I do what I can with my free time.
Kirill: Okay, got it. Tell us a bit more about what you did before,
because it sounds pretty interesting – software development.
What’s your background? Where did you study and where
did you work after that?
Tuatini: As for my background, I started coding at the age of 13. I got
my Master’s degree at the age of 23. And at the age of 22 I
went into the startup for which I worked for 2 years. And I
was the lead developer for Android over there. It was like
every morning you wake up and you tell yourself, “Okay,
there is this bug to fix or this stuff to add,” but you are still
solving the same problem, you know. So in the end, I was a
bit tired of this and this led me to this decision to quit this
job and go into deep learning. But during the last years, I
was involved in many areas of computer science. I was a
sysadmin, I worked on security systems, I worked as a
backend developer, and as an Android developer of course.
And nowadays I’m doing data science and deep learning.
Kirill: Gotcha. Okay, very interesting. So, tell us a bit about your
day. From my own experience, I know that it’s very hard to
organize yourself when you’re just working or studying from
home. You don’t have to be like, you know, at 9:00 you have
to be there, at 6:00 you come home. When you don’t have
this schedule that’s imposed upon you, when you have to
control your own time, sometimes it can be very hard. So tell
us about your day. What time do you wake up and how do
you go from there?
Tuatini: Yeah, sure. Actually, before leaving my old startup, what I
did was to create a clear roadmap of what I have to learn. My
days actually—what I do of my days is actually using a
Pomodoro Technique. I don’t know if you know this
technique….
Kirill: I’ve heard about it. Tell us a bit more. My brother told me
about it.
Tuatini: Yeah. Actually, it’s a really dumb technique, I would say. It’s
basically you take a timer, you put it for 20 minutes, and
you work during these 20 minutes but you have to work,
really work, like no Facebook, no eating, nothing, just work.
And when these 20 minutes are over, you can start having a
pause for 5 minutes and then you go back to your work for
20 minutes again. And actually, I use this technique every
day to get at least 4 hours of really productive work or not
work, but learning, really productive learning at least per
day so that I know that I was useful, you know, that I didn’t
lose my time spending my time on just social media so that I
can keep up with my goals which are defined on my
roadmap, you know.
Kirill: Yeah, that’s really cool. So, you use this Pomodoro
Technique. For those listening out there, it’s a very
interesting technique. I find it a bit controversial, or not
necessarily controversial, but perpendicular to a lot of the
other techniques that exist out there. For instance, there are
philosophies out there that say that it’s very hard when you
switch over between tasks. Like, you’re doing something for
an hour or two and then you switch over to something else.
It takes you at least 20 to 40 minutes to readjust your brain
to work on something else.
Whereas in the Pomodoro Technique, every 20 minutes you
have to get up and do something. And you can set the stamp
to whatever you want: it can be 10 minutes, 20 minutes, 30
minutes. You get up and you have a break and then you
come back to the technique and so on. You’re constantly
consciously interrupting yourself, which is contrary to the
other methodologies and philosophies where you try to limit
the interruptions. It’s interesting. It’s great to meet a person
like Tuatini, for whom it’s working.
Tuatini: Actually, I don’t respect the technique 100%. (Laughs) When
I’m working on a piece of code and I really want to solve it, if
the timer goes off, I just start it again and I just don’t take
the 5-minute pause between the two 20 minutes. Actually, I
just use this technique to know that at least I commit 4
hours per day of doing productive work. I would say I only
use this for that, but I don’t respect it 100%. As you said, it’s
really hard to shift your focus to something else for 5
minutes and then come back to the same thing you were
doing before. Sometimes it doesn’t work for me either.
Kirill: Yeah, gotcha. I can totally relate to that. You kind of get into
this state of flow, right? You’re coding and you’re into it and
the last thing you want is to get up and have a break. I think
maybe that’s the way it’s designed. If you’re getting up and
having a break, that means you’re not in a state of flow, or
at least you’re kind of keeping control of how you’re
spending time. It’s harder to fall into the pattern of getting
on Facebook or doing something irrelevant to your work that
way, because even if you do that, then you’ll only be limited
to 20 minutes before you snap out of it and you realize,
“Okay, the next 20 minutes I will do well.” But if you do get
into the state of flow, definitely don’t get out of it. Because
when I feel that I’m in flow and I’m getting things done, like
you say, you don’t want to get up and you just restart the
timer.
Actually, I did something similar when I was also quitting
my job and I was working in the evenings and on the
weekends, I was valuing every single minute. Even without
knowing the Pomodoro Technique, what I did was I broke my
hour into four parts, so 15-minute parts, and I set a timer
for 15 minutes. And my goal wasn’t to get up and go have a
break.
My goal was to reset the timer every 15 minutes, but on the
side of my desk I had a paper and a pen, in my notebook I
had a page dedicated to this, and it would be split into two
parts. On the left it would be ‘productive,’ on the right it
would just say ‘non-productive.’ And then whenever the
timer would go off, I would reset it and I would just put
another point or a line into ‘productive’ or ‘non-productive.’
So, I would look back and I would say, “Okay, were my 15
minutes productive or did I just waste them?” And then I
would just put a line—you know how they do, you put like
four lines and then you cross them over and that’s five and
so on.
I put a counter into one of the two sides of the page and that
way, at the end of the day, I could look back and similar to
you I would be like, “Okay, I’m going to count how many 15
minutes I spent wasted and how many 15 minutes I spent
productive,” and based on that, I would come up with it and
I would be like, “Oh, I actually had 4 hours or 8 hours of
actual productive work this day,” and I’d be satisfied with
that. If there was too much unproductive time, I would have
to look back at it and understand what I was doing wrong
and the next day it would become better.
Tuatini: Yeah. Actually, I also used, until a certain time, I used a
Don't Break the Chain Technique. I don’t know if you know
about that.
Kirill: No.
Tuatini: It’s actually also very dumb, but it’s like—for example, if I
tell myself that I have to commit 4 hours a day of really
productive work, I have to keep up with that for the rest of
the days. I can say that on weekends I don’t have to respect
that, but if I have to respect that from Monday to Friday, I
have a calendar and I have to check each day that I didn’t
break the chain, you know, I didn’t commit only 3 hours on
one particular day or things like that. It’s just that, actually.
And I had this calendar where I checked every day for like 3
months, and then I went on a trip and everything was
screwed, you know. (Laughs)
Kirill: Yeah, I can imagine. But that’s good. That’s how you get
stuff done, right? I’m sure you’re learning a lot.
Tuatini: Yeah, sure.
Kirill: We’ll get to that in a second. So you wake up and you start
working right away, or do you go for a walk? Walk us
through your day. What do you usually do first?
Tuatini: Actually, as soon as I wake up, I do my work. And if I see
that I’m reaching my 4 hours on the day, I can try to do
some stuff which are not useful, like browsing the Internet
or—I don’t know, but nothing special about it. Basically all
my days look like that. I wake up, I work, I commit to my 4
hours and then I can play video games or talk to my
girlfriend or—I don’t know.
Kirill: Gotcha, have a social life. But why only 4 hours? Is there a
reason? Why not 7? Why not 8? Why not 10?
Tuatini: Actually, it’s not only 4 hours; it’s at least 4 hours. So, it’s
that I know I was committed to these 4 hours. Yesterday, I
think maybe I spent 8 or 9 hours because I was in a Kaggle
competition and I was like, “Why doesn’t my accuracy go
up?” So, I was on my code and I just couldn’t stop.
Kirill: Gotcha. So, basically it’s better to set a reasonable but not a
very high level that’s easy to achieve and you are always
satisfied with your day and if you do more, you do more.
Tuatini: Yeah, exactly.
Kirill: Gotcha. Okay, really cool. Okay, so that’s how your day is
structured. Now tell us a bit about what is it that you’re
learning. We’ve had a few guests talk about deep learning on
the podcast, but what’s your version of deep learning, your
summary on deep learning in a few sentences? What is it?
Tuatini: Currently I’m learning everything I can around CNN and
computer vision. I just finished the courses on phase.ai, and
I plan to start the new one you released on Kickstarter, you
know, as soon as you release it. I still didn’t finish the one
you released on AI because I was doing other MOOC before
that. But yeah, I think I will probably base my work around
computer vision and reinforcement learning because these
are the two fields which fascinate me the most.
Kirill: That’s really cool. Why do you like computer vision so
much?
Tuatini: Well, compared to NLP, you have this feedback, this visual
feedback, so it’s much more rewarding than NLP. I don’t
know how to—
Kirill: Yeah, yeah, I got you. NLP, for those listening, is natural
language processing. Is that right?
Tuatini: Yeah, sure.
Kirill: No, I totally understand. It’s more gratifying to see your
results. You can actually see green boxes and stuff like that,
what’s going on.
Tuatini: Yeah. And there is also the fact that the field of computer
vision opens so much more opportunities than NLP, you
know. When you are doing NLP, in my opinion at least, I
don’t see that many opportunities compared to what you can
do with computer vision. With computer vision, for example,
you can make robots understand the world, you know. But
with NLP, you try to make them understand our language,
which is not an especially real representation of our world.
It’s a language which was created by humans, whereas
computer vision is a thing which is native. I don’t know how
to say that, but you get me.
Kirill: Yeah, gotcha. It’s just like information that’s out there
regardless of language.
Tuatini: Yeah, exactly. It was not made by humans, if you will.
Kirill: Yeah. Okay, very interesting. And you’ve got a very
interesting project that you’re working on or you’re planning
on working on – StarCraft, right? Tell us a bit about that.
Tuatini: Yeah. I didn’t start yet, but I plan to start hacking around
with StarCraft and really start diving deep into
reinforcement learning because this is actually one of the
fields which is very—not a lot of startups are involved in
reinforcement learning except OpenAI and DeepMind, and I
think that’s because it still has to make its proof. Because as
for today, with CNN and RNN – convolutional neural
networks or computer vision, and RNN, natural language
processing – you can already achieve a lot of real work. But
with reinforcement learning, it’s much more complicated
because you cannot—I don’t know how to say that, but it’s
much harder to see the goal of what you can do with
reinforcement learning.
Kirill: Gotcha. Just so that everybody is up to speed, let’s break it
down because there are people listening to this podcast,
maybe this is their first time encountering reinforcement
learning or CNN and RNN, so I’ll just explain the
abbreviations. CNN is convolutional neural network, it’s a
type of deep neural network that is used to recognize images
and process them and recognize objects and images and so
on. RNN is a recurrent neural network. It’s a deep neural
network that has memory. That’s what’s used for natural
language processing because like words and sentences—you
need to know what words came before the current word to
make sense of the meaning. And reinforcement learning is a
technique which—maybe you can explain reinforcement
learning, Tuatini. What’s your explanation of reinforcement
learning?
Tuatini: Actually, for me, reinforcement learning happens when you
have a goal. For example, let’s say you take a game—for
example, let’s take “Mario” because everyone knows “Mario.”
Your goal in “Mario” is actually to go to the end of the road
and get the flag at the end and you win. So, in reinforcement
learning, you define some specific goals that you want your
algorithm, or as you say in reinforcement learning, your
agent, to reach. But you don’t tell him about how to play
“Mario.” You just show him, for example, people playing
“Mario,” and it just figures out how to play “Mario” based on
how the human players were playing it. Basically that’s it.
But I believe there’s a lot more things you can do with
reinforcement learning, apart from games, but as of today, I
don’t yet have this vision because I didn’t look deeply into
reinforcement learning yet, so I can’t really tell.
Kirill: Yeah, exactly. There’s lots of applications. Reinforcement
learning basically allows computers and robots to learn from
the feedback they’ve been getting from their environment or,
as Tuatini mentioned, from observing humans and others
play games. It’s a way for computers to learn without us
hardcoding the algorithms into them, without saying, “If
this, do that; else, do this.” You know, without the
hardcoded algorithm, they can still learn.
So, the very interesting developments in the world these
days are using reinforcement learning for very complex
tasks. And games are a cool environment to test this stuff
out because games represent a mini real world with their
own rules and so on. So if you can make a robot or an AI
work in a game, that means you’re one step closer to making
it work in the real world. And one of the recent
breakthroughs was—I think it was OpenAI that created an
artificial intelligence that can play the game of DOTA. Did
you hear about that?
Tuatini: Yeah, I saw that. Actually, some people said on Reddit that it
was kind of dumb because it took a path in the game that
the algorithm wasn’t aware of. It was just failing at
completing its tasks. I don’t know, I didn’t look further into
that, but—
Kirill: Okay. It’s definitely getting closer. This game is quite a
complex game where they have annual competitions and the
pool prize is like $2 million or $8 million, it’s growing every
year. This artificial intelligence can already play comparable
to a one-on-one player. The next step is five-on-five and so
on. Those who are interested can check it out online, it’s
called DOTA – I think it stands for “Defence of the Alliance” –
version 2.0 – and artificial intelligence is playing that game.
You can have a look at that. So you’re venturing into an
even more complex territory. You want to create an AI or
take on the challenge of exploring how to create a
reinforcement learning algorithm for StarCraft. StarCraft is a
different type of game. It’s a strategy game, right?
Tuatini: Yeah. Actually, as I understood, the most complicated part
in this game is that when you start the game, you are not
aware of the map of the game. You actually have to do some
discovery by sending what they call in the game a probe, so
you’ll just send the probe to try to ‘map the map,’ you know.
But this is really challenging because when an AI is not
aware of its whole environment, it makes things more
complicated, you know.
Kirill: Yeah. That is going to be very interesting to see, how you
take on that challenge. So, once you dabble on with the
game—you mentioned you don’t have any ideas on how to
apply this in real life, but how are you going to go about
that? I think that’s an interesting question. Let’s say you’re
successful at creating an artificial intelligence to play
StarCraft. What is your next step for turning these skills
that you’ve developed into a business, or into something that
you can bring to the world and add value to the world in the
real world?
Tuatini: To be honest, I don’t know. (Laughs) All I know is that I want
to work on something that I like. And this is the very reason
why I left my old startup, you know. The future will tell me.
I’m the kind of guy who likes to do what he wants to do, but
I do not worry about the future like, “Will I start a startup
with that?” or “Will I work in that field with big companies?”
I’m only worried about doing the stuff I like, you know. And
random events of life will happen. Maybe I will have some
opportunities to meet people who are also interested in that
field and who have ideas to make that into a business. I’m
not close to anything, but as for now, I don’t know. I can’t
tell you how I could apply it to a business.
But as you told me when you came to Paris, there is actually
one real application for reinforcement learning that Google
made, which was to reduce their cost in electricity, and it
was reinforcement learning. So I believe that you can make
very useful stuff, but as of today, I’m not aware of what you
can really do, what is the potential of reinforcement learning.
I cannot tell you that I want to create a business out of that,
you know.
Kirill: No, I totally understand that. And that’s really inspiring,
right? It’s so cool to have the courage to just say, “No, I don’t
care what everybody else thinks. I don’t care that I don’t
have a job. I’m just going to do what I love. And I don’t even
know where it’s going. I don’t know what opportunities I
have in the future, but I know they will come. I know that if
I’m doing what I love, then the world cannot happen in a
way that I will not have those opportunities. They will come
no matter what. I’m just going to keep doing what I’m doing
and when they come, they come.” That is very inspiring.
Tuatini: Yeah. Well, actually this is all I did for my whole life, you
know, because I was born in French Polynesia and I lived on
a very small island. And I really liked computer science so I
was just committed to it. I didn’t have really good grades in
school, but I didn’t care. I was only passionate about what
was useful to me and what I wanted to do. And then when I
came to France, I started applying this knowledge.
So, I don’t know if it’s the randomness of events or if it’s
chance, but I believe that you cannot have guarantees in the
future of what you will do. For example, I cannot tell myself
that as of today I will work on data science for marketing so
that I’m sure that I will get a job in that field. For me, I just
want to work on things I like. And at some point, you will
find something according to that, you know.
Kirill: Yeah. And how good is that, that when you do find
something, because all the time you’ve been doing just the
things that you like, you’re guaranteed that you’re going to
like it because you’ve never even thought of doing anything
that you don’t like?
Tuatini: Yeah, it’s really rewarding when you find something. It’s the
way I live, actually. (Laughs) Actually, I think randomness of
events can also come, because I told you I’m surviving on
unemployment benefits and actually the guy who gives me
these unemployment benefits can just come to my door and
say, “We will cut the grass under your feet.” So from one day
to another, I can find myself not having any income but I
just tell myself that when the time comes I will just figure
out what to do.
Kirill: Yeah. So you’re not stressing out about it? It’s not a problem
now right? “It might happen, it might not happen. If it
happens, then that’s when it will become a problem and
that’s when I’ll sort it out.”
Tuatini: Yeah, exactly.
Kirill: Very interesting. I’m actually reading a book now by Eckhart
Tolle, it’s called “The Power of Now” and there he talks
exactly about that. It’s very interesting how your story
depicts the philosophy that he explained. He says that
“There are no problems in the world. Think about right now.
Right now, do you have a problem? Does anybody have a
problem? Not five minutes from now, not a day from now,
not a year from now, but right now do you have a problem?”
No, all you have is the present moment and you have the
situation that you’re in. So, what we perceive as problems,
somebody else in your situation might say, “Oh, I might lose
unemployment benefits. This is a problem.”
But what we perceive as problems is actually a situation
that might arise in the future. And when the future comes,
when it becomes a present moment, if it arises it’s not a
problem, it’s a situation you’ve got to deal with. “It’s a
situation I’m in. I’m going to deal with it. Okay, done with
the situation. Now I’ll keep going with my life.”
Tuatini: Yeah, exactly.
Kirill: It’s a very healthy mentality, right? It helps you be calm.
Tuatini: Well, I would like to tell you that when this day arises, I
won’t stress a bit, you know. (Laughs) But I think I will
figure out what to do. There is no such thing in life as
insurance. You can’t say that, for example, “Oh, I’m great in
my position at the company where I’m in and I want to work
for the coming 10 years,” because you don’t know what will
happen. Maybe you will never get a salary raise, maybe the
company will fail or maybe you will get hit by a car and die. I
don’t know. So there is no such thing as trying to forecast
your future in terms of insurance, job insurance, for
example.
Kirill: Yeah. I totally agree. You’ve got to be not even prepared, but
accepting of whatever comes. Whatever comes, comes, and
we will deal with it when it comes.
Tuatini: Exactly.
Kirill: Cool. That’s very inspiring. I’m sure a lot of people listening
to this will have something to think about. Tell us, has
somebody inspired you? Has there been like a mentor, a life-
changing event, something in your life that has inspired you
to choose the path that you’ve chosen of software
development and then get into deep learning and so on?
What’s been the most inspirational thing for you?
Tuatini: For software development there were two things. The first
one is that I wanted to get out of my little island. (Laughs)
And the second one is still related to that. My father told me,
“What do you want to do? You want to watch the cows and
do agriculture? Or you want to go abroad and make relevant
things?” So, I quickly made my choice.
As for the influencer, I don’t have any person who influenced
me. Well, not one that I remember. I think it’s a whole, you
know. I saw a lot of people in the field of AI and I was always
passionate about creating AI when I was young, but at that
time I was afraid of math and there wasn’t really a lot of
things happening in that field, [indecipherable], as everyone
called it. And when I saw all these views coming from
TechCrunch and your MOOC, for example, all the resources
online to learn about AI actually, I told myself, “Yeah, let’s go
for it.”
Kirill: So you could see the potential?
Tuatini: Yeah, exactly. But there wasn’t just one thing that led me to
that decision, you know. This is just a whole, I would say.
Kirill: Okay. And was it hard to start? There’s probably a lot of
listeners who are into data science but not yet into the field
of deep learning or artificial intelligence, and they can also
hear about it, hear about the potential and see things, how
it’s changing the world, but it’s very scary. There’s a lot of
programming, there’s a lot of complicated libraries and even
mathematics and so on. So was it hard to start when you
made that decision?
Tuatini: Actually, yes. (Laughs) As I told you, I have a software
engineer background, so it’s really like my comfort zone
when we are dealing with computer science stuff, you know,
whether it be sysadmin or programming or anything. But
when it comes to math, that’s a completely different story.
(Laughs) Like, as of today, when I look at a math formula,
when I look at one on paper, I just want to go on a tutorial,
or find an alternative to implement what I want to
implement.
But this past week, I was trying to implement the Unet
paper. Unet is actually a technique in deep learning to
segment images. So basically, let’s say you have a car in the
middle of the image and you have a background. You just
want to get the car and have a transparent background
behind it, you know. Unet is meant for that. So what I’m
trying to do is to implement the paper from A to Z, as you
used to say, and really look at the math formula and try to
make sense of that. Because I think what is really hard for
me in the math is not the logic, but it’s actually like a foreign
language to me.
Like, if I see some sigma sign, I know it translates into a ‘for’
loop in Python or things like that, but there is a lot of signs
that I don’t know in math. It’s just really foreign to me. But
I’m working towards that goal. And my goal actually is to be
able to take a paper, implement it and ship it into
production. Basically to do the whole workflow. Not just
implement the paper or just ship things into production, but
be able to do the whole thing, you know. Because if someday
I will have to create my business or contribute to something,
I will have to deal with this whole process. You won’t be
constrained just to implementing the paper or just shipping
things into production.
Kirill: Yeah, I totally agree. That’s a very big step, learning math. I
agree, if you want to do the full flow of things, then you’ll
need to know the mathematics. At the same time, for the
benefit of our listeners, I just wanted to say that there’s
ways to get into this field even without delving too deep into
the mathematical aspects of things, but rather just
understanding intuitively how and why things work and
then understanding what goals you want to achieve and
then putting everything together. It’s like building Lego
bricks together with the pre-coded libraries and so on. Like
in PyTorch—well, less on PyTorch and more in the Google
one—
Tuatini: Keras?
Kirill: Yeah, with Keras, if you use Keras there’s a lot of stuff pre-
coded for you, right?
Tuatini: Yeah, exactly.
Kirill: You can just build this AI with a few lines of code. Even if
you don’t know that much coding, you just put in a few lines
of code and there you go – you’ve got a convolutional neural
network going or something like that.
Tuatini: Yeah.
Kirill: Do you think that’s a valid way for people to start?
Tuatini: Yeah, sure. Actually, this is how I started, so yeah, totally. If
you want to start and you don’t have a lot of background
into mathematics or even computer science, it’s better to
work with high level libraries, as we call them, because
actually it obstructs every low level computation you could
do with Flow or TensorFlow or with PyTorch, for example,
because Keras actually is a library on top of TensorFlow so it
extracts a lot of stuff you would spend hours to figure out
how to do.
Kirill: So do you use Keras or do you use PyTorch?
Tuatini: I started with Keras, but currently I’m with PyTorch. The
reason is very simple because with Keras, as it’s on top of
TensorFlow—TensorFlow uses what we call static graph—
Kirill: Yeah, and dynamic graph.
Tuatini: Yeah. So it basically means you have to define all your pre-
processing input. You have to define how you will multiply
this or that. And then when you have defined your graph
basically, you start a session and you run the thing. But the
drawback with that is that when you are used to work with
IDE and debuggers, it’s really hard to check on things. It’s
really hard to look at, for example, “If I multiply this matrix
by this scalar, what is the result of that?” Actually,
TensorFlow has its own debugger, but I didn’t get a chance
to look at it. But the difference between TensorFlow and
Keras and PyTorch is that PyTorch uses dynamic graphs so
that as you go through your code you can put break points
to your code and can just look at the result of your code,
actually. It’s not like blackboxed as you can find in
TensorFlow. That’s the main reason why I moved away from
Keras, but Keras is very great to start with.
Kirill: Yeah, definitely. And it just makes things easier for people
looking at this. It’s a natural progression, right? When a
child is learning how to speak, they don’t start by reading
Shakespeare. They start with the basics and then they
progress. It makes sense. Let’s talk a bit about hardware. I
think that’s a topic close to you because from our catch-up
in France, I remember you were talking quite passionately
about your computer setup and so on. So tell us a bit about
that. For deep learning you obviously need a powerful
computer. How did you go about that?
Tuatini: I started with buying a GTX 1070 because you have to get a
really good GPU, especially if you’re working on a computer
vision program with deep learning. With NLP it’s less
relevant, because when you are inputting images into your
model, images are actually big matrices, so you have to have
a really good GPU to handle this kind of data. Then I bought
myself a GTX 1080 Ti.
Kirill: Sorry, you lost me there. Is that a better system?
Tuatini: Yeah. It’s actually better, yeah. So, what I actually have is I
have two computers: one is in my basement and one at
home. And I have my MacBook also and I work on my
MacBook and I actually execute my code remotely on this
machine and I get the result back onto my MacBook.
Actually, on my blog – I will give you the link – I teach people
how to build their own rig. And I really try to explain
everything, like how do you do that and what does it mean,
not only what are the steps to build your rig, but also what
does this step mean so when you have to fix your rig, if it’s
not working anymore, you know what you are dealing with.
Actually, I have—I don’t know the model of my CPU
anymore, but I have a good CPU and especially a good GPU
and 32 GB of RAM. I think it’s the minimum for a deep
learning project.
Kirill: Gotcha. And tell us what’s the difference of CPU and GPU
and why do you need GPUs for deep learning?
Tuatini: Actually, deep learning deals a lot with matrices and GPUs
are a specialized kind of hardware which is meant to deal
with matrices. This is why you can transfer your data to
your GPU memory and then make the GPU work instead of
your CPU. And the difference between the two is that—I
don’t know how to say that, but GPU is really parallel, it
does a lot of parallel work, it has like 3,000 cores, compared
to your CPU, which has 8 cores or something like that. CPU
can do a lot of stuff, but not really parallel. In contrast, the
GPU is meant to deal with very specific narrow tasks, but it
can parallelize a lot of stuff. So that’s the main difference
between the two.
Kirill: And GPU are actually for graphics, right?
Tuatini: Yeah. Graphics, if you think of it, are actually just matrices,
you know. So yeah, it’s why GPU can also be used to train
deep learning models.
Kirill: Yeah. So it’s just a convenient coincidence that GPUs were
originally developed for graphics, which are matrices, and
“We’re going to run some deep learning now on these GPUs,
only a trillion times faster?”
Tuatini: Yeah.
Kirill: And did you hear about the NVidia Volta chip?
Tuatini: Yeah.
Kirill: Tell us a bit about that, because Ben Taylor told me about it
a couple of times and I’m not really up-to-date with it. What
is this Volta chip and why is it so special?
Tuatini: As for today, there is only one GPU which was released with
this chip, with this new architecture, which is Tesla V100.
Volta is actually an architecture. Basically, you have
different kinds of architectures: there is Maxwell, there is
Pascal, there is Volta. Currently, with the latest models of
GPU you can find on the market, we are with Pascal. And
Volta actually is one step ahead, and it’s GPU which was
made especially for deep learning. It’s an architecture which
was made especially for deep learning because with Pascal,
it was made more for gamers because it uses a special
architecture, which actually has some optimization
especially made for games.
But these are optimizations which aren’t needed for deep
learning models, so the Volta architecture, at least the Tesla
V100—it doesn’t really take away, but kind of takes this
optimization and it also uses the half-precision mode which
is—how could I simply explain that? Basically, you have two
modes, which are FP32 and FP16. FP32 is actually to deal
with numbers with really good precision. And FP16 is
actually what we call half-precision, which is actually what
we need in deep learning because we don’t need to have a lot
of decimals in deep learning.
Kirill: Floating points, how many floating points you have. Gotcha.
Tuatini: Exactly. Volta was made actually to deal with half-precision
in a way better sense than past architecture. As of today, if
you try to put matrices with half-precision on your GPU, you
would get really worse results than if you put FP32 precision
matrices.
Kirill: So how come there’s only one of them in the world then?
Tuatini: You mean one of—
Kirill: There’s only one type, right?
Tuatini: Because it’s fairly new. And as of today, I don’t think it’s yet
released to the public. I think NVidia sends it only to a few
people involved in artificial intelligence, but you cannot buy
this GPU yet. They will release it next year, I think. I’m not
sure about that, but as of today, it’s hard to get your hands
on this GPU.
Kirill: And it says that it’s about $150,000 for one of them. Crazy.
Tuatini: Yeah. (Laughs) It’s not for everyone.
Kirill: Yeah, that’s right. Okay, that’s really cool. Thanks a lot,
Tuatini. I just have one question to wrap up the podcast
here. You obviously have a huge experience in this field and
all this knowledge that you’ve acquired in a short period of
time. From what you see and from what you know about
deep learning and AI or data science, about all these fields,
where do you think this is all going and what should our
listeners look into in order to prepare for the future?
Tuatini: Given the past events, the past news I saw passing by over
the Internet on TechCrunch, Reddit and things like that, it
seems like a lot of VC are interested in investing in startups
which deal with edge computing. Simply put, edge
computing is being able, for example, to learn your deep
learning models not on the cloud, but directly on your
device, for example an IoT device or your smartphone. And I
feel like it’s the next step, that you are able to run directly
your deep learning models onto your device locally.
Because if you take, for example, drones, when they have to
map their environment and avoid obstacles, if they lost their
Internet connection or if there is a lot of latency onto that
connection, they cannot make the decision quickly enough
to not go into the obstacle, you know.
Yeah, I saw a lot of people saying that the next step in deep
learning is to be able to ship model into production, and
actually even François Chollet, who is the author of Keras,
said that the deep learning field in term of research paper is
told and the real problem now is to be able to ship this
model into production, whether it be on the cloud or on edge
computing. But I feel edge computing is the next big thing
for me.
Kirill: Gotcha. Okay, thanks a lot. That’s something to look into.
You normally think that everything is going offline more and
more now, but as you say, some applications have to make
very quick decisions.
Tuatini: Yeah. And if you look at the chipmaker, for example
Qualcomm, for example, or even if you look at the latest
iPhone, iPhone X that Apple released—
Kirill: iPhone 8.
Tuatini: They also released iPhone X, I believe.
Kirill: Oh, okay. I didn’t know that.
Tuatini: Yeah. There’s 8, 8S and X. X is the one without any buttons
on it. And actually you can do face recognition directly,
locally on the phone. And when they spoke about the chip
which is in the iPhone, they talked about a neuro-engine,
you know, and same goes for Qualcomm. They are trying to
make this kind of core processors which are meant to deal
with deep learning models. So, even by looking at the
hardware makers, you can see that they are trying to do this
edge computing scene.
The same thing goes for Movidius. Movidius is the company
which was bought by Intel. They created the neural compute
stick, which you can buy for $70 and then run your deep
learning model on that. But be aware that these kinds of
devices are not meant to train your model, but just to run
inference – in other term, predictions.
Kirill: Yeah. And to your point, if anybody wants a really vivid
example of this, I’m not sure about the latest iPhones, but
on the more older versions – for instance, iPhone 5 or maybe
6 and 7 as well – if you try to do speech-to-text, you know,
you’re trying to type a message and you click that little
microphone button on the left, and I’m pretty sure it’s the
same thing on Android phones like Samsung, you click that
button and it will convert what you’re saying into written
text, into the text message. So you can dictate the text
message. And that’s really cool. That’s actually deep learning
in action. But if you switch off your internet connection, if
you put it into flight mode and you try to do the same thing,
it won’t work because it’s actually going through the cloud.
It’s sending your voice to the cloud, it’s recognizing it there
and then sending the text back to your phone.
So this is what we’re talking about here when Tuatini is
saying “edge computing,” is that those models, and that
hardware and everything, will be integrated into your device.
That’s the challenge. In order to perform that there and then
so you could, for instance, do the speech-to-text even while
you’re offline.
Tuatini: Yeah. And that’s really interesting for people in this field,
because actually, instead of paying cloud instances to run
your deep learning models, they just let the client pay for
their latest iPhone and then they ship their application
which runs into the iPhone. They just cut costs regarding
this.
Kirill: Yeah, gotcha. And cutting costs means competitiveness and
competitive means successful business.
Tuatini: Exactly.
Kirill: Okay. Thanks a lot, Tuatini. Could you give us some details
on how our listeners can find you, follow you or connect with
you if they’d like to learn more about your journey?
Tuatini: Sure. There is my LinkedIn, my e-mail, and I have also a
personal website where I blog from time to time, which is
http://tuatini.me. The next blog post I’m going to do is, as I
told you, about Unet. And actually, what I want to show is
how I went from reading the paper to doing the
implementation and explaining each step of the
mathematical formula and translate it into code. I think that
it may be useful to some people. I know some people found
useful the last article I released on how to build your deep
learning rig, but the next coming article will be on real deep
learning stuff. As for now, there is only computer science
stuff; there is no deep learning stuff.
Kirill: Gotcha. I’m looking at it now. It’s very detailed. The posts
have a lot of information. So if anybody listening wants to
build their own deep learning machine – and I know there’s
a couple of people like that who are following this podcast –
then check it out, it’s tuatini.me. And of course, we will
share these details in the show notes. I’ve got one more
question for you Tuatini: What is the one book that you’d
like to recommend to our listeners?
Tuatini: Regarding the book, there is one which is not really related
nor to AI nor to data science, but it’s a really good book
which opens your mind, I would say. It’s called “Homo
Deus.” Actually, it’s made by a really famous historian that
you can find on TED because he gave some talk there. He
basically created two books: First one is “Homo Sapiens,”
which tells the story of where we came from; and “Homo
Deus” is where we are going. And he’s not only talking about
artificial intelligence and stuff, but everything about our
future. At the same time, he doesn’t put himself as someone
who wants to predict the future. He only opens your mind to
think about what happened in the past and now look at our
present and imagine our future. It’s really, really interesting
and I really recommend this book. I really loved it.
Kirill: Fantastic. I might check it out myself. I just googled it, it’s
called “Homo Deus: A Brief History of Tomorrow” by Yuval
Noah Harari. Very interesting. Once again, thank you so
much, Tuatini, for coming on the show and sharing all the
insights. It’s been a fantastic episode. I really loved to learn
from you.
Tuatini: Thank you.
Kirill: So there you have it. That was Tuatini Godard, AI learner. A
very inspiring episode! We wish Tuatini the best of luck. I’m
sure that he’s going to definitely find the right application of
all of these skills that he’s learning and there’s probably tons
of companies that would love to have a person like Tuatini
on their team, somebody so passionate about what they’re
doing. And as you saw from the podcast, there were lots of
things that were discussed.
My personal favourite out of all of them was just the whole
commitment and dedication to his passion that Tuatini has;
how he quit his job and became unemployed just to follow
his passion. It takes a lot of guts, it takes a lot of courage to
do that, and it’s very inspiring to see that there are people
like that in the world, who can jump straight into it and
know that they will be successful if they follow their passion.
And I wish the same to everybody. I think when you do what
you’re passionate about, when you do what you love, then
eventually you will become successful and it’s just a matter
of time. So make sure to follow Tuatini on his LinkedIn to
see how his career goes. I’m sure there are lots of exciting
and inspiring turns that are coming up for him.
And on that note, you can get all of the show notes for this
episode, the transcript, and a link to Tuatini’s LinkedIn at
www.superdatascience.com/99. If you enjoyed this episode,
we’d really appreciate your review or a rating on iTunes. All
of your reviews definitely help us spread the word about
data science. We’re getting close to our 100th episode, so we
can’t wait to share more exciting and fun guests with the
rest of the world. Thank you so much for being here. I can’t
wait to see you here next time. Until then, happy analysing.
