Embed Size (px)
Citation preview
Kirill: This is episode number 119: Data Science Trends for 2018.
(background music plays)
Welcome to the SuperDataScience podcast. My name is Kirill
Eremenko, data science coach and lifestyle entrepreneur.
And each week we bring you inspiring people and ideas to
help you build your successful career in data science.
Thanks for being here today and now let’s make the complex
simple.
(background music plays)
Happy new year everybody, and welcome to the very first
episode of the SuperDataScience podcast for 2018. Very,
very excited to have you on board. Super pumped about the
beginning of a new year, a new adventure. And we're kicking
off the year strong and on an exciting note. So Hadelin and I
got together to record a webinar a week or so ago, which
some of you actually attended. We were discussing trends.
The trends that are coming up in 2018, what to expect, what
to look into in the space of data science.
So on this webinar, we discussed topics such as AI,
blockchain, data security, self service analytics, digital twin,
and many, many more. So those are just some of the
highlights of what we talked about. Prepare yourself for this
exciting adventure, and also note that this webinar is
available in video version. You can find the video at
www.superdatascience.com/119. And if you have the
opportunity, then it's probably best to watch it that way. But
also you will get exactly the same insights from listening to
the podcast.
And on that note, without further ado, I bring to you the all-
time favourite, the incredible Hadelin de Ponteves and the
trends of 2018.
(background music plays)
So today we're talking about trends, data science trends. It's
sometimes a bit complex to separate tech trends from data
science trends because they come hand in hand. You can't
really imagine, a lot of times, technology without the power
of data science behind it, and also data science usually is
used to empower certain applications that have some
technological aspect to them. We can highlight the ones that
are more data science specific.
Hadelin: That's right. And anyway, today when I talk about data
science, I mostly talk about AI because that is the most
exciting part, first. And second, data science has been kind
of automated by AI actually, and so that's why the demand
for machine learning AI is growing as the demand for
analysts is slightly decreasing its acceleration. So I mostly
talk about AI, and we're going to see that the big trends
coming in 2018 will all be around AI.
Kirill: I'll try to talk less about AI to mix it up, because Hadelin is
super passionate about AI, and he will be doing most of that.
Ok, so thank you very much to Leonid, our resident data
scientist, who has helped us put together a list. We have two
lists, actually. We have a list of trends that are predicted to
be popular in 2018 and to pick up, and we have a list of
trends that were predicted to be popular in 2017. So the way
I think we'll structure it is we'll first go over the 2018 ones,
see what's coming, and so we make sure we cover them. And
if we have time at the end, we'll review what happened in the
previous year and see how correct those predictions were
with what we found there. Sounds good?
Hadelin: Sounds very good.
Kirill: Alright, let's get started. So first trend in 2018: An AI
Foundation. Funnily enough. We're talking about how
businesses are going to start using AI more and more and I'll
probably steal the quote here from Andrew Ng that AI is the
new electricity. So Hadelin, what are your thoughts on that?
How are businesses going to be using AI more and more in
2018, and what are going to be the major developments
there?
Hadelin: That's right. So lots of companies have integrated AI into
their business process. And actually I have some figures. I
think there's around 60% of companies that have already
made the move to integrate AI into their system, into their
company, into their business processes. Automated
processes, reduce costs, or make it an AI-based company.
And only around 30-40% are starting to think about this but
have not made the real step yet. So definitely that's a big
trend of not 2018, but already in 2017. Companies are
adopting AI and try to build some AI teams and make it AI-
optimised. So yes, this is really happening.
Kirill: And we talk about the difference between general artificial
intelligence and narrow artificial intelligence, right? And in
the sense that when we say AI, we don't mean that there's a
robot controlling the whole business or anything. It's like
very narrow applications, in say one specific area, in
marketing, in operations, or in another part of the business,
there is AI.
Hadelin: And also in the decision-making process. They use AI lots for
decision making. It's sometimes the AI that makes the
decision, because they apply the machine learning models
on the data and they get some great [inaudible] tools to help
the decision process, sometimes at the highest level. The
executives use AI to help make their decision.
Kirill: This statistic is very interesting, because I was surprised at
the 40%. I think it sounds really high, that 40% of
businesses would have already in some shape or form
started adopting AI. But I guess what that’s saying is that
it’s not across the board, not across the whole business.
What are your thoughts on that? Do you think more and
more businesses will be adopting AI, not just in one specific
application, because that’s what I think it’s talking about,
that in one area there is some sort of AI that they’ve
introduced and that’s how they get the 40% number?
By the way, guys, if anyone is interested, some of these stats
come from the Gartner reports. We can mention where else
we get them as we go along, but for example, that came from
one of Gartner’s recent reports. So do you think, Hadelin,
that businesses are going to limit themselves to having one
application or two applications of AI across the whole
business? Or do you think they will start doing narrow AI in
many different spaces?
Hadelin: I think they will just leverage AI. Most of them leverage AI to
improve their business, to optimize their business, to reduce
their cost. There are a lot of applications that you can
leverage AI from to improve your business process. So I
think that’s mostly what’s going to happen. Then you have
on the side the real AI-based companies that do general
artificial intelligence, that do artificial intelligence at the core
of their business. But if we’re talking about most companies,
well, they’re definitely starting to leverage AI. And mostly
thinking of the consulting companies, each one of them is
building a team of data scientists and using AI, as I said, for
their decision making process.
Kirill: Okay. All right. Any other comments on AI and how it’s
going to lay foundations in business in the coming year?
Hadelin: Well, it can go really far. AI can be applied in some field that
is not covered yet today. You know, when we talk about AI,
there are actually a lot of branches of AI. You have computer
vision, so computer vision can be used in many businesses
to improve it. You also have deep natural language
processing like what we’re doing with chat bot, and that can
help significantly some companies by bringing some chat bot
systems that can help people in the company and that can
help navigate or whatever. You also have some other
branches like robotics. Robotics can definitely automate the
processes and everything.
And you also have those data robots that can leverage the
data automatically and provide some outputs that will be
insightful for decisions you have to make. So, there are tons
of applications and there are even some applications that we
haven’t thought about. So that’s definitely going to develop
in the coming years.
Kirill: What’s your favourite application? Is there something that
pops to mind that you’ve recently heard of that you’re like,
“Wow, that’s a really cool application of AI?”
Hadelin: Yeah, I have two in mind.
Kirill: Okay, let’s go.
Hadelin: I have augmented reality and blockchain. Blockchain and AI
is going to be the big trend in 2018, the combination of both.
I actually don’t know which one exactly is going to be the
new electricity. Remember, you said AI is going to be the
new electricity, but when I see blockchain developing and
what it’s capable of doing and all the things that are
happening right now, I have some doubt what is going to be
the real new electricity.
Kirill: Nice. And that’s a good segue for us to get into blockchain.
Blockchain is a shared distributed decentralized ledger that
basically takes out the middleman, takes out the bank or
the polling system or something and helps people have
trustworthy transactions with each other, secure
transactions with each other even when they don’t know
each other. I know you’re also very excited about blockchain
and there actually was a TED Talk recently where they were
saying that blockchain, as you just mentioned, is going to be
the technology that’s going to change and shape the world in
coming years, but especially in 2018, we’re going to see
some major shifts because of blockchain, and those shifts
are actually going to be even bigger than the ones that we’ve
seen with AI. So what are your thoughts on blockchain and
why are you so excited about this technology?
Hadelin: Oh, my goodness. It’s so disruptive. For example, it could
build a new Internet because the fact that it is totally
decentralized all over the world makes it extremely powerful
at, for example, compressing data. AI and data science is all
about compressing data so that we can have faster and
faster transfer of data or faster and faster transactions and
even more secure. And blockchain will play a significant role
in that because since everything is decentralized and since
everything is scripted and since everything is well-organized
into flows that are in such a way that you cannot go back in
the flow and modify anything, well that makes it a super
solid, safe, and fast system.
And why did I say that it could build a new Internet? Since
everything is decentralized, we could have the data divided
into very small parts all over the world and that would make
some kind of peer-to-peer compression that would make
everything super powerful, like fast compression, you know,
everything decentralized so that you have some extremely
fast connections around the globe and that would be thanks
to blockchain. So that could go really far. I think maybe two
years or three years from now that that could go really,
really far. And AI, of course, has a part to play in that
because AI automates everything, it automates the processes
so it will optimize the process inside the blockchain and
that’s why the combination of both these technologies will
make something super powerful.
Kirill: And also I wanted to mention that a lot of people, and I used
to do the same, when we hear blockchain we think Bitcoin
and when we hear Bitcoin we think blockchain, but those
are not synonymous. Bitcoin is one of the things that
leverages blockchain, that is built upon blockchain, and at
the same time blockchain can be used for many other
things. I really like the example somebody suggested of
using blockchain to do voting. You know how in the U.S.
there were elections and people were voting and then you go
and you submit your vote? There is always an organization
in the middle, like a big organization that counts the votes,
that makes sure everything is safe, that there’s no cheating
and that everything is accounted for, everything is
trustworthy.
The organization that’s in the middle to ensure trust, that’s
where blockchain comes in. Blockchain can remove that
organization and basically you could do voting from your
computer and the way blockchain works—we are not going
to go into detail for me personally because I’m not an expert
in blockchain, not yet, but I really want to get deep into the
stuff and understand it better because it’s so interesting and
disruptive.
But at the same time, you take out the organization, you put
a blockchain, and what that enables, through the
cryptography it has and through this decentralized and
hyperconnected system, what happens is that now all of a
sudden it’s completely trustworthy. There’s a ledger of
everything that happens, this ledger is decentralized and it’s
distributed to lots of people. You would have to hack
hundreds of thousands of computers at the same time with
the highest level of encryption in order to break into that,
and that’s much harder than to hack into an organization or
something like that – I guess, I’m not a hacker. So, that’s the
power of blockchain. For instance, voting during elections,
that’s a big deal. Like, imagine you wouldn’t have to get out
of your house; you’d just vote on your laptop and do it like
that. There are a couple of other examples. Anything else
pops to mind?
Hadelin: Yeah, that’s right. You actually raised a very important
point, which is actually there will be a massive trend in the
coming years. And that trend is security. Blockchain will
have a huge part to play in security. As you said,
blockchains are safe because you need to hack thousands of
computers to hack the system. And for this reason we’re
going to definitely leverage blockchain to make some more
secure and safe systems like the example that you gave
about voting or any other ones.
Well, the main application of blockchain today is the new
money, Bitcoins, which I actually have doubt it’s going to
last because it has signs of a bubble. But with this
blockchain we can make a totally decentralized and safe
financial system, so that indeed we improve the security.
That’s just an example of the security brought thanks to
blockchain can be applied to many other fields and that’s
definitely going to be a big trend in the coming years.
Blockchain will not only play a part in security, but AI will
have to play a part on itself for security, because AI is
developing pretty fast and at some point we will reach some
powerful artificial intelligence that will go beyond human
capacities and therefore we will have to control AI and that
will be another part of this big security trend.
Kirill. Yeah. I had another thing I heard about blockchain, that it
can be used for distributing music. There is a couple of
people who distribute their music through blockchain and
what that does, if you’re just a user, you can download it for
free and listen to their song and so on, but if you want to
use it for a project, like in a movie or in a trailer or inside
your own YouTube video or something like that, then you
just get it through blockchain and that way the transaction
happens automatically. So, again, it’s this whole trust thing
that nobody is going to get your music on its own.
Hadelin: Yeah. And what’s crazy is that you don’t even have the
music on your phone. That’s because it’s totally
decentralized. You have one part of the music somewhere,
another part of the music somewhere else, sometimes 1,000
kilometres away from each other, and that’s a peer-to-peer
system which makes it fast compression that allows you to
have the music very quickly on your phone without adding it
literally on your phone.
And that’s the same for movie compression or streaming.
You can leverage blockchain by having some parts all
around the decentralized system to get your streams, movies
on your computer, and this will come from all this
decentralized system brought by this blockchain technology.
Again, there’s the safety component that is really improved
by blockchain, but also the speed of the transfer, and also
the data compression.
Kirill: Nice. So, the question I actually have that I was thinking
about was, it’s really cool to know about blockchain and
understand that, “Oh, cool, it goes into the foundation of
Bitcoin or this can go into foundation of the voting system or
content distribution and so on.” But the question is, is it
something that’s completely out of reach, or is it something
that we can create ourselves? Can we just sit down and
program a blockchain in Python or something like that? Is it
possible?
Hadelin: Yes, it is. And we will do it.
Kirill: We will do it?
Hadelin: Yes, we will do it very quickly.
Kirill: (Laughs) Awesome. Okay, so that’s our two cents on the
blockchain. Okay, so the other one is not on our list right
now, it’s on our list of stuff for 2017, but since we mentioned
it I think it’s an important trend to point out: security, so the
whole concept of security of data on the Internet and how
that is important and how that is progressing. I have a few
interesting examples. In 2017 we had some major, major
security breaches in the world and that is a huge indication
that in 2018 security is going to start growing again. Two
major breaches: WannaCry, cyber attack in May 2017,
people have probably heard about that. Microsoft computers
in many countries around the world were locked down
[indecipherable 20:07] companies from FedEx to the
Ministry of Foreign Affairs of Romania were impacted by
that. That was a major thing, there’s a Wikipedia article
about it and so on and that was all over the news. That was
a big one. That was in May 2017.
And just when you think it can’t get any worse, one of the
biggest attacks in the world history also happened last year.
You might know about a company called Equifax, it’s a
credit rating company that has about 800 million customers
around the world, it’s like one of the top three biggest credit
agencies in the world. And 143 million customers were
affected by that attack. That happened last year and it was
announced on the 7th of September that they had a data
breach, but that happened actually ages before that or
months before that. People’s first names, last names,
addresses, Social Security numbers, dates of birth and more
information were stolen.
And if you think about it, 143 million people – there’s like
324 million people in the U.S. on its own. So that’s 143
million in the U.S. alone that were affected. There were
people in Canada, in the U.K. that were affected. So, 143
million out of 324 million, which is the population of the
U.S., that’s almost 50% of the U.S. population was affected
by this attack. How crazy is that? And that’s just in the U.S.
alone. That just stands to show.
And there were tons of other smaller examples, like the Uber
hacking where they paid the hackers not to say anything
and then some executives were fired for that. And other
companies as well that we’ve heard in the news that have
fallen victim to attacks. It’s on the verge of different trends.
Data is becoming more and more popular, more and more all
over the place. And with the proliferation of data, what’s
happening is it’s harder and harder to keep it safe, it’s
harder and harder to keep it secure and then also hackers
have access to much more sophisticated tools, not even
talking about AI and machine learning. Even the algorithms
and ways they can infiltrate systems are much more
sophisticated. There’s a huge thread and I think that
security is going to be a major trend in 2018 and onwards
because companies understand the importance of protecting
their data and their customers’ data. What are your
thoughts, Hadelin?
Hadelin: That’s absolutely right. Customers’ data actually already had
issues in the past. People were complaining that the data
was actually used in some non-ethical ways and that’s
because there were some data breaches, security breaches.
So, yeah, this is another essential part of the security trend
that is coming, we not only the need to protect ourselves
against powerful AI or we not only need to protect ourselves
against hacking, but we also need to protect the data. For
this, again, blockchain will definitely play a part in that,
because since everything is encrypted, that can be a way of
protecting the data in such a way that it would be very
difficult to hack it thanks to the multiple systems
decentralized all around the world.
So, I think we will always have to work on that because it’s
the war of technology. Once you find the technology that can
protect your data, what comes after that is that somebody
who has a better technology and can break your technology
that protects your data. So, we’re going to have some leap
over leap, so the more we will get into some improving
technologies, the more we will have to pay attention to the
fact that it’s going to be difficult to keep up. And we’re going
to have more and more experts in depth of all the
technologies that are protecting the data and therefore less
and less people because we’re going to reach a higher level of
expertise that’s going to be very high.
And that’s another trend that I’m going to, it’s that trend
and possibly a danger that has to do with security, is that
the research on AI and everything is becoming so high-level
that less and less people manage to get into it at the state-
of-the-art. That’s a danger because imagine the state-of-the-
art research falls into the wrong guys’ hands. You know, the
black box will only be understood by the wrong guys. That
would lead the world into some kind of danger. So, that’s
why we not only need to protect the data and develop those
technologies, but we need – and that’s the most important
thing – to educate the world to teach them how this works
and to explain them how the state-of-the-art models work,
so that the black box doesn’t become that black for too many
people.
Kirill: I think that’s a great answer, a great comment on that. I’m
just checking the questions occasionally. We just had Halper
say that “Sorry, guys, this is cyber-security. Can we please
get to things more related to data science?” So, I want to
respond to that because I have seen the world of cyber
security. I’ve been working at Deloitte and I’ve seen what a
market this is. It’s a huge, massive area which is so
underrated by data science practitioners for the reasons that
we mentioned.
One is that any kind of data science work that you do can
fall victim to cyber-crime. Also, because it intersects with
data science. There are so many different data science
applications. Like, the machine learning algorithms that
we’re using, that we’re learning, they can all be applied in
the space of cyber-security and in the space of data security.
There are definitely inherent algorithms that are specific.
Like, what’s that best one called? I forgot, but basically
there’s a mathematical equation that is specifically used in
the space of cyber-security, at least one that I know of.
But all the other algorithms with the increasing rise of
machine learning, AI, deep learning, that is slowly shifting
into the space of cyber security and that’s what we had
witnessed in 2017. We saw the infiltration of deep learning
and machine learning in the space of security to help find
these anomalies, find these possible areas where things can
be breached, and to help mitigate those risks. So, I
personally think that data science trends, we will cover off as
many as we can right now, but one of the biggest trends
overall is the one we just talked about. You can see how it’s
on the intersection of different things like data science, AI
and blockchain.
Okay, moving on. So, next one we have is deep learning
technology is becoming mainstream. We’ve seen a lot of
things happen just recently from image classification,
machine translation, facial recognition, chat bots and other
things that use deep learning insights. They’re starting to
rise now. Hadelin, what do you think is happening now?
Because a lot of these algorithms, tools and technologies
have been around, some since the 80s, but some have been
around since 2012. Why is it all happening now in 2017/18?
Hadelin: Because of the applications that we realize we can do with it.
You know, we can do some crazy applications with deep
learning now and we don’t have to wait for one or two years
to do them. So, they have become extremely popular. We can
see, for example, the GANs that managed to create some
fake images of a real-looking human person, or those
computer vision applications that can detect any object in
videos. You know, these are some very cool applications.
And the other reason why they have become so popular is
that it is being democratized. Thanks to all the open-source
platforms like GitHub, you can get the code, and as we show
in our course, apply them very easily on your videos or on
your applications to do these very exciting applications.
That’s why, it’s mainly thanks to everything becoming open-
source and more and more easy to apply, because as you
know, 4 or 5 years ago only experts could use that, only an
expert had a good understanding of how it works and
everything. You know, it’s like people had trouble in the
beginning getting from old phones to smartphones. Four
years ago people had trouble getting on all these codes and
all these models, but today more and more people, even if
they don’t have any notion of coding, they manage to code
and use the deep learning applications.
However, that being said, I read some statistics about the
models used in data science, in companies, or in general,
and those deep learning models are still at the bottom of the
ranking. The most used models in companies today are still
logistic regression, Random forest, XGBoost, decision trees,
and the deep learning models like CNNs, RNNs or GANs
actually are still at the bottom of the list. It is growing, it is
definitely growing. However, Geoffrey Hinton has just issued
a new paper on capsule networks. If that works, if it can be
implemented easily, and if it works fast enough, that could
mark the end of the actual deep learning model because this
would be revolutionary. But we’re not there yet. The
implementation is very long to implement and execute, so we
still have some to realize that.
Kirill: Okay. I agree with those points. Capsule networks are
definitely some interesting disruption that’s coming our way.
And you can tell about the importance of deep learning just
by the number of students that sign up to our course. When
did we release the deep learning course?
Hadelin: We released the deep learning course last March, actually.
Kirill: March, yeah?
Hadelin: Yeah.
Kirill: So that would be like nine months. And how many students
signed up so far?
Hadelin: We have in total 65,000 students.
Kirill: 65,000 students just in that one course signed up in nine
months. That’s quite insane. That’s like 7,000 people per
month signing up to that course alone, is that right?
Hadelin: Yes, absolutely.
Kirill: That’s hard to believe. That’s showing where the world is
going. We can kind of get a sense for those things. You
know, before, machine learning—and still machine learning
is very powerful, but now we’re slowly going into the space of
deep learning because deep learning can solve any problem
machine learning can solve, but better and more accurate.
Hadelin: Yes. However, for simple applications, models like logistic
regression and XGBoost will still stand above deep learning
because deep learning is not the fastest to execute, you have
to iterate over many epochs, to train and apply forward
propagation and backward propagation many times to train
your data. Whereas XGBoost, you just put the data as your
input, and in a flash it returns the output you want. So, for
simple applications, for simple problems, which can
definitely give you some insights for your company, I think
the logistic regression models will still be among the first. So
it’s not like deep learning is going to erase the other one, it’s
not that it’s going to make the other ones disappear. It’s just
that for the powerful applications that are extremely
demanding, deep learning will become the best models.
Kirill: Gotcha. Very, very good points on that. Okay, moving
further: Persistent growth of the Hadoop market. So, what
we are seeing is that Hadoop and any Big Data systems like
Spark, for example, which is kind of the new thing out
there—remember that we were at the ODSC, I think, and
Spark 2.0 came out, that was in May this year. That was like
a new big thing. So why are these technologies, why is
Hadoop and Spark, why are they becoming more and more
prominent and why are more and more companies going to
them? What do you think?
Hadelin: Well, that’s because the amount of data is constantly
increasing. You need systems like Hadoop and Spark and
Pig and Hive to handle all this data, to handle all the Big
Data systems, because otherwise it would be really slow to
handle them. Those systems are faster and faster to manage
your data and to organize it and to leverage insights from
them. You definitely need those systems. And actually I
heard that—well, actually there’s an important point to say
about data science, it’s that Python and R are still by far the
software that’s the most used in company to do data science
or machine learning or deep learning.
But then there is something growing up, it’s called Scala and
it’s based on those Hadoop systems that handle Big Data.
That is growing because you need more and more powerful
systems to handle bigger and bigger data. That’s why it’s
something to definitely consider. Actually, on LinkedIn I see
a lot of recruiters’ posts and in these posts I see the skill
that are needed, and I see now almost all the time, besides
Python or R, I see Hadoop, Spark, Hive, Pig and Scala.
Among all of them, if I had to choose one, if I had to
recommend one, I would say Scala, because it’s extremely
powerful at handling Big Data.
Kirill: And also I’d like to add that what we discussed before, deep
learning and AI are contributing a lot towards the rise of
Hadoop and Big Data systems. Because to train deep
learning models and AI algorithms, you need a lot of data.
You need that data to be stored somewhere, you need to be
able to access it quickly, so it’s just natural that those two
come hand-in-hand. The more the world turns to AI and
deep learning, the more we’ll see Big Data systems such as
Hadoop, Scala, Spark and so on.
And also a lot of it is going into the cloud. It’s like a trend
that we’ve seen in 2017, that it’s not just Big Data, but it’s
also Big Data in the cloud. And the reason for that is the
cutting of costs, right? If you have servers on your premises
for a large organization, that’s one thing. Then you have
servers and you need to scale them, you need to broaden
them, you need to update them even as new technology
comes out and new hardware comes out. That’s millions and
millions of dollars, tens of millions of dollars depending on
the size of the organization; hundreds of millions of dollars
for some organizations.
Whereas if you have things on the cloud, it’s less likely or it’s
not as trusted yet by executives, especially old school
executives who don’t want to let go of their data, they don’t
want it to be somewhere else, they’re worried about security
and so on. But, it can actually be even more secure, it can
be very easily accessible, and it’s very easily scalable, and all
the things can be updated very quickly so you don’t have to
worry about updating your hardware. You can just click a
button and your hardware gets updated, or the team that is
managing it, because now they have economies of scale, the
company that is managing it, they’re doing it for many
businesses, so it’s easier for them to upgrade their hardware
and also you just click a button and it’s scaled. That’s a
huge thing and that’s why a lot of start-ups that are starting
out, they don’t even consider having their servers on
premises, but straight away in the cloud. It’s harder for
companies that have been around for a while to make that
move, but the ones that are doing it, those are the ones that
are going to be ahead of the curve. My question to you,
Hadelin, is, are we going to make a course on Big Data one
day?
Hadelin: Big Data? Well, we are going to make a course on Big Data
once Scala or any other system stands out. Because right
now it’s not standing out that much. We still have Python
and R, but as soon as one of these Big Data systems is the
most used system in the companies that can have some
tremendous and powerful impacts on the companies, we’ll
definitely make a course on that. And actually, I did a lot of
that when working in Google, I worked on a lot of the Big
Data systems, so I could share with you my experience and
how this works. Yeah, we can definitely do it. What do you
think?
Kirill: I agree with that. Because we’ve been asked by students
quite a lot about this. I think the reason we haven’t yet is
because this industry is still in early stages, it’s very much
forming, it’s very much shaping up. Like, Spark 2.0 came
out this year, at the start of 2017. You constantly have new
technologies, new versions come out and so on. Like, if we
record a course today, two months from now we will need to
re-record it, we will need to update things and so on.
It’s a good point, as soon as there’s a prominent market
leader in that space and we know where this whole thing is
going, then we can give you a course and also understand
for ourselves and help you guys understand where this
whole thing is going and how to keep updated with these
things. It’s not in the pipeline yet, but it’s in our vision to
create this course some time soon.
By the way, Leonid, our resident data scientist, asked me to
make this little plug, so a little bit of advertising here. We
don’t have a course on Big Data, but we do have a series of
tutorials which apparently are amazing on YouTube, which
are about PySpark. So, if you want to learn about PySpark,
no cost involved and you don’t have to purchase anything,
just subscribe to our YouTube channel, make Leonid happy,
give him a Christmas present, because he is in charge of our
YouTube channel, and you will also get updates about the
PySpark series that we are releasing and maybe that will be
something that you’re looking for. And also, of course, there
are other things apart from Big Data that we talk about on
the YouTube channel. Check it out. Okay, any other
comments on Big Data?
Hadelin: Yeah. Well, PySpark is amazing. I really encourage students
to subscribe to that channel. You know, it’s not standing
out. If it was standing out, we would make a course on it,
but it’s definitely useful. So that’s a great thing. That’s
amazing.
Kirill: Awesome. Okay, we’ve already talked about AI. There’s
another side of things which is applied AI, applying it in
different spaces, different areas. I think we’ll skip that for
now in the interest of time. Let’s talk about digital twins –
interesting concept.
Hadelin: Digital twins! Yeah, I hear that more and more. That’s the
Internet of Things, right? It’s like you’re connected to your
objects and you can transfer some information between the
digital twins and yourself, so that you can use them at a
better and better rate. Is that correct?
Kirill: Yeah, yeah. And it’s not just for people. Like, an airplane will
have a digital twin, or an airplane engine will have a digital
twin and there’s like a data connection between them, or like
a whole city could have a digital twin and you can basically
model different scenarios that can happen in the city or in
the turbine of an airplane by analysing the way that the
digital twin is behaving and having the inputs from the
actual object to the digital twin and also adding your own
inputs. Or you could take the inputs from your own object to
your digital twin and then also take inputs from the other
hundred airplane engines that you have in your company
and you compare it to the average, tweak things, and see
model scenarios.
It’s very useful, for instance, in things like airplanes for
preventative maintenance, so you know when the issues
might come up even way before they are going to come up.
And, of course, cities, to understand the behaviour, social
and demographical things, transportation and things like
that. So, you can model traffic jumps. For example, a city is
growing. You constantly feed inputs from all the sensors that
you have in the city into that digital twin and then you’re
like, “Oh, I wonder what will happen during Thanksgiving
when we block off these three roads.”
Because you have a digital twin, which is pretty much the
identical copy of the city, you can actually block off the
roads. This is my understanding of things. And then you see
what happens to the traffic all over the city simply because
you have been inputting those data points. It’s not just like a
model that stores data points, it’s actually a model that
learns how they behave, how they interact and what
dependencies are in there.
Hadelin: That’s right. And there is another term we haven’t spoken
about. At the beginning of this webinar I said, like, “The
biggest trends that I’m most curious of or that I see coming
the most tremendous way,” so we talked about blockchain.
And then I don’t remember if I said it, but the other one was
augmented reality. We didn’t speak about this one, right?
Kirill: No.
Hadelin: Yeah, that’s right. I heard that this could be the first AI
trend that could reach the trillion dollar market, augmented
reality, because it has tons of applications.
Kirill: Like Pokémon Go.
Hadelin: Yeah, but not that kind of applications. Like, long-lasting
applications. And I heard this has huge potential, so that’s
definitely something to follow. What do you think? Do you
think it could be reaching such a huge market?
Kirill: Yeah, definitely. I heard that there’s VR, virtual reality, but
augmented reality actually has the potential to be bigger. We
saw initial attempts at that with the Google Glass, and that
was like ages ago. What was it, 2015 or ’11?
Hadelin: Funny, when I worked at Google, they actually introduced
them and I was there when it happened so it was 2015.
What did you say, 20—?
Kirill: I don’t remember, 20-something. (Laughs)
Hadelin: Quite recent. Yeah, it was definitely 21st century.
Kirill: Yeah. And then you left Google and the whole project fell
apart.
Hadelin: Yeah. (Laughs) Well, that happens.
Kirill: Yeah. Okay, augmented reality is an interesting one. It was
funny to see how Pokémon Go just boomed and then was
gone. I don’t hear about it.
Hadelin: I’m not sure what the reason is exactly. I don’t know, but it’s
crazy how this was like what we call these ‘bubble trends.’
You think it’s a trend, you think it’s growing and at some
point it bursts out and nobody hears about it anymore. Like
Bitcoins, for example. I hear debates on Bitcoin all the time
right now, like every day. The debate is whether it’s a
bubble. “Do you think it’s a bubble? Do you think it’s not a
bubble?” It’s based on blockchain technology, but still it has
all the signs of a bubble, so everybody is talking about this.
What do you think? Do you think it’s a bubble?
Kirill: That’s a good question. It really reminds of how the first time
Bitcoin really spiked, I think it was 2014 or something like
that, and it just went up and then people were like, “No, it’s
going to keep going up forever,” and then – Bam! So, I don’t
think it’s a fad, I don’t think it’s something that will go away.
I think we will be using more and more cryptocurrency,
Bitcoin or others, but I have doubts that it will keep growing
forever like that. A lot of it is fuelled by hype, by media and
stuff like that, and as soon as something else, the next big
thing comes along, I think there will be a correction. This is
not financial advice, by the way, guys watching this webinar.
It’s just our opinions.
Okay, so we talked about digital twins, we talked about
augmented reality. What do you think about self-serve
analytics? Data science is growing. Let’s get down to the
basics. Forget about AI and stuff like that for now. So,
business intelligence, and we’ve got lots of different tools,
lots of different approaches, and the amount of data, the
volume of data, the velocity, variety, veracity, etc. of data is
growing all the time and very quickly. So, with that amount
of data, organizations are slowly starting to realize that it’s
unsustainable to have only data scientists look into that
data and pull the insights out.
It’s still very important to have data scientists, and more and
more organizations are getting into that, but at the same
time, what if everybody in your organization can look into
data and get insights from it at some extent? And that is
self-serve analytics. What are your thoughts on the trends of
self-serve analytics in 2018?
Hadelin: That’s a very good question. Indeed, it’s like what I said
about the black box. You’re absolutely right. Right now, only
data scientists can leverage the data to gain some insights
and help with decision making and everything. At the same
time, we have those automated systems like this company
DataRobot that basically makes what they call ‘data robots’
that take your data as input and will return the output
without needing the work of a data scientist doing all the
process of data analysis.
That’s what I said at the beginning of this webinar. I think
that it has the potential to be automated, like self-managed
data systems, and it’s actually going to come pretty quickly,
but it will not replace data scientists. We will always need
data scientists to improve these systems, check these
systems, control that they give the right insights, check that
that makes sense because sometimes the decisions can only
be good decisions if you include the human factor. So, we’ll
always need some people to have a complementary job on
that, because the machines cannot do everything. So, I think
self-analysing systems, as you call them—
Kirill: Self-serve analytics.
Hadelin: Self-serve analytics will grow, but will never grow to the
point that it will replace the data science jobs.
Kirill: That’s a good point. A little bit of reassurance there. Yeah, I
agree with that. And I also think it’s important for those of
you out there who are data scientists or who are aspiring
data scientists, it’s an important trend to keep in mind. It’s
been around for a while now, but it’s going to be picking up
more and more that people in organizations, regardless of
their level, they are going to need to have some sort of data
literacy. And it’s your job, or you can make it your job as a
data scientist, to spread that, to create data advocates and
to create people who are excited and inspired by data.
It’s going to make your job easier because that way, the
people you talk to in the organization, they know about what
you’re doing, they know the value and the importance of
data and data science, and that’s cool. But also, it’s going to
help the organization to grow into that right direction. If you
really care about the organization, and I really hope you do
in the sense that you’re working in the company that you
love and that you believe in their mission. If you do, then
that will be your contribution into putting them onto that
right pathway where not only you are doing the data science
work, but everybody in the organization is contributing,
some people can do a simple regression, some people are
better at understanding the different types of data, or some
people have access to BI dashboards that you’ve created and
now instead of you redoing it every time, you’ve created them
in an interactive way so that everybody can get their own
insights.
It’s an important trend for data scientists to consider,
because one thing is just doing data science on your own
and being the rock star that’s cool; another whole big thing
is about educating others in the space of data science. At the
end of the day, you will help them out not just in their roles,
but also in personal growth because that’s where the world
is going. You have to be data literate to be up-to-date with
everything that’s happening and have other opportunities,
you know, have a broad spectrum of ways that you can
develop your career.
And speaking of not doing data by yourself, we had an
interesting trend that we haven’t talked about yet, and that
other trend is that companies are going to look more into not
just hiring data science geniuses or wizards standalone, but
actually building out data science teams. So, a slight
difference there, but a very important one at that. What do
you think, Hadelin? Why do you think companies are going
to be steered away a little bit just from one super genius
data scientist? That’s cool, but how about we build a team of
five or ten that work together very well?
Hadelin: Because the goal in the end is to get as much people as
possible on data and getting the skills to manage the data.
It’s what you said a couple of minutes ago. Most people
should be able to leverage the data to gain some insights as
everybody is using a smartphone today. Everybody knows
how to use a smartphone. We need everybody to know how
to leverage data to gain some insights. That’s why I think
they are making the teams. They don’t want to leave that to
the experts because this is not democratization. If we leave
that to the experts, we will miss out a lot on other
capabilities because data science is not that hard.
Everybody can do it. Everybody can apply the models. You
just need to understand the intuition. And sometimes you
don’t even need to understand the intuition, you just need to
understand how you have to get your inputs into the system
and apply the models and gain your insight.
The data is becoming so abundant. Data is everywhere.
There is more and more data that of course we need more
and more people, and the only way to get more and more
people is, instead of leaving that to the experts, building
teams of many data scientists or many people that can at
least do the basic stuff in data science to gain some powerful
insights.
Kirill: I agree with that. Like, when you have a team of people, you
have one expert that’s awesome, but you’re dependant on
them. Like if they leave, or if they decide to do certain things
in a certain way rather than exploring other possibilities,
other tools, you will be very dependent on that kind of stuff.
I think your opinion here—people watching this or listening
to this, you guys really should listen to Hadelin on this
because you’ve worked in data science teams, right? You’ve
been in Google and your other jobs that you’ve been in—I
have been in that situation where I was the one data
scientist and I was doing all the things.
From that I can totally speak to, yes, I tried to do my best in
good faith and do really amazing work as much as I could,
but at the same time it was very highly dependent on my
subjective opinions, on my subjective ways I think the
company should go and do things. You know, that might be
wrong, it might be right, but you don’t want a large
organization depending entirely on the opinion of one
person. So I think what you said is valid here.
And the other thing that pops to mind is executives, so let’s
talk about executives for a bit. There’s two sub-trends in the
trend for executives that I see. We have more and more
organizations hiring CDOs, Chief Data Officers, and the
other one is that more and more executives, like Chief
Executive Officers, the guys that are directors and heads of
the companies, they are looking to get educated in the space
of data science. Like, it’s not their jobs to be data scientists,
but they want to find out more about algorithms, about
applications, about AI, about deep learning, about all these
different things data science-related to not become
technological or data science dinosaurs so that they can see
what this is all about. What are your thoughts on that,
Hadelin? Why do you think more and more executives are
jumping on board with this trend, and do you think it’s
necessary?
Hadelin: Of course that’s 100% necessary, and a simple reason for
this is that executives are the one who makes the decision.
They are the ones who decide the next move in the company.
And since data science is so powerful at leveraging the data
to get the right insights that will help in a significant way to
take the right decision, well, executives definitely need to be
connected to data science; not necessarily be experts, but be
connected to data science to understand and be convinced
how data science can help them make the right decisions.
And I say that not only from a logical point of view, I also say
that based on experiments. I had on the phone a lot of
executives that asked me for some advice on how they would
leverage data science to take decisions. They said mostly
that the problem was that there’s a huge pyramid between
them and the data scientists, so they are far from the data
science teams and therefore they need some better data
visualization tools to understand how the data is leveraged
and the insights are extracted to help them take the right
decisions. So the executives want to get more and more into
data science and they actually need it for the simple reason
that they’re the ones making decisions and data science is
so powerful at helping them to take the right decisions.
Kirill: Interesting. So, let’s talk about strategy because decisions,
they link up into strategy. What are your thoughts on data
strategy for large organizations? Is that a thing? Is it
important for an organization not just to think through their
marketing strategy or let’s say operation strategy, growth,
expansion and so on? Do you think that executives should
be thinking about data strategy? And what does that mean,
what does it mean to think about data strategy?
Hadelin: If you talk about strategy, I think strategy has a lot to do
with intuition as well. It has a lot to do with intuition,
experience and not only data. Data can help in the strategy
because in the strategy you have to take some decisions and
data helps in taking the right decision, but there is so much
more than decisions in strategy. It’s a combination of things.
It’s pretty complex, by the way, but you also need intuition a
lot, and I think the intuition is the opposite of data. That’s
why data will never replace everything because you always
need intuition, and you mostly need intuition and strategy.
So that’s a very interesting question, actually, which I think
the answer is that data is not everything for strategy.
Kirill: I agree, but what I’m referring to is—data is not everything
for strategy, I totally agree, but in the sense that let’s say we
have strategy overall, but inside strategy we have everything
to do with data, like the tools that we’re going to use. Are we
going to install Hadoop or are we not going to install it? Are
we going to go to the cloud or are we not going to go to the
cloud? Do we add more data points? Do we have enough
data points about our customers? Do we need more inputs?
Do we need more unstructured data? Do we need to handle
unstructured data? What insights can we gather from our
data, or what is our current data saying about where our
organization is going and how can we leverage that more,
how can we implement deep learning or AI algorithms and
so on? That’s the stuff I mean for the strategy around data.
I think it’s quite important for organizations to start keeping
that in mind. I don’t know if it’s just going to happen on its
own, the way it happens, and that might be a bit more
reactive than proactive. Data strategy helps you be proactive
in the sense—it’s really hard to be proactive in the first place
because there are so many technologies that are coming out
that you don’t even know about and that’s going to come out
next year or a few months down the track, but at least you
put in effort to be on top of your organization. You know
your pitfalls and you know where you need to patch things
up, you know where you’re not keeping up to speed with
everything that’s going on in your organization in the sense
of data. But if you don’t even think through data strategy,
that leaves you way behind everybody else and I think that
takes away a huge competitive advantage for companies.
Hadelin: Yes. And I’m reading something interesting here, so I’m
going to read that to you. According to Gartner, 59% of
organizations are still building their enterprise AI strategies
while the remaining 41% of the organizations have already
made the plunge. So, yeah, there is definitely something
happening with the AI strategies for companies right now.
59% is a lot.
Kirill: Yeah, so they’re at least thinking through how they’re going
to—
Hadelin: Yes, leverage AI for strategy.
Kirill: Gotcha. Okay. Yeah, very cool stuff. What else? Do you have
anything else that we have missed?
Hadelin: No, I have mentioned all the trends I wanted to speak about.
The ones that I’m very curious about and I will be following
very closely for the next year, in 2018, will be blockchain
and maybe augmented reality.
Kirill: Nice. And for me probably blockchain, I definitely want to get
deep into that topic and understand a bit more about
blockchain, what’s going on there, and how we can apply it
in the world, how it’s going to be transformative. And I think
AI, I will be interested to see how that goes. I’d say more
deep learning, less AI for me. It’s kind of more basic than AI,
but I like the concept of narrow applications. So something
like, “Okay, there is a problem. Let’s apply deep learning and
solve it.” That’s pretty cool.
Okay, we’re kind of running out of time, so I think that’s all
the trends that we’ve covered. I think that was pretty cool.
Thanks a lot, guys, for coming on the webinar.
Hadelin: Thanks so much, guys. That was my first webinar and I
really enjoyed it.
Kirill: Yeah. All right, take care guys, and hopefully we’ll have more
of these, we’ll see these coming up more. And good luck in
2018. Let’s stay in touch.
Hadelin: Yes, keep up the good work.
Kirill: All right. See you, man.
Hadelin: See you.
Kirill: There we go. Those were the trends for 2018 that we were
able to identify. Of course, some of them will happen, some
of them will happen less, but overall those are the most
exciting things to look out for in this coming year. Which
was your favourite trend? Which is the one that you’re most
excited about, the one that you’re looking into the most?
Personally for me, I like the concept of anything to do with AI
and digital twins and security as well, but the one I’m most
curious about is blockchain. I have this new project of my
own that is going on that I’m learning about blockchain and
I want to learn more and more about blockchain, I want to
find out how it works, what exactly goes into it, what the
security, encryption and other implications are, and what
are the use cases and so on. So definitely that’s the one for
me. But again, yours might be a bit different.
In any case, I hope you enjoyed these trends and now you
know what to look out for in 2018. If you know somebody in
the space of data science that could benefit from this
episode, then forward it to them and help them also get
prepared and maybe you’ll have something to discuss and
debate after they listen or watch, because this episode is
available in video mode, and you’ll have something to
discuss with them. Plus you can get all the links from this
episode and the show notes at
www.superdatascience.com/119. There you can also find
the video recording. And on that note, thank you so much
for being here. I can’t wait to see you back here again soon.
Until then, happy analysing.
