Up next in 10
Session info:
In this session we will investigate a special branch of Machine Learning called deep learning by tackling a simple computer vision problem: distinguishing between hand gestures. We will build upon some machine learning concepts introduced earlier by focusing on a popular deep learning framework called PyTorch. We will first look at handling image data and then move on to model building and optimization. Finally we will spend some time using a model in code.
About the speaker:
Seth Juarez received a Bachelors Degree in Computer Science at UNLV with a Minor in Mathematics. He also completed a Masters Degree at the University of Utah in the field of Computer Science. Seth is currently interested in Artificial Intelligence specifically in the realm of Machine Learning. He is currently working on a .NET library meant to simplify the usage of the common machine learning algorithms.
Seth currently live near Redmond, Washington and work for Microsoft.
Show More Show Less View Video Transcript
0:00
Hi, good morning
0:29
Good afternoon and good evening, everyone, depending on where you join us from today
0:34
I'm so happy to welcome all of you today again with Seth Juarez, who is going to bring us
0:39
an awesome topic again, getting started with deep learning. I'm very excited
0:45
How are you guys feeling? I'm excited. I got a lot of stuff
0:49
Hopefully, we have enough time. I might go super fast. And if I'm going too fast, people will tell me to slow down, okay
0:56
Yes, we'll let you know. And you guys don't remember that we had a session two weeks ago with Seth already that was about getting started with machine learning, which was also really amazing
1:09
So if you have missed that, you might want to check that out later on
1:13
And with that, I think it's a good time to give you a quick introduction about what is AI42, and then we switch back to you, Seth
1:21
All right? Great. See you later. Hi, welcome back everyone to another AI42 session
1:41
Håkan, could you tell a bit about what is the motivation of AI42
1:46
Yeah, sure, Yves. So the main idea here is that we would like to give everyone a chance to get into this
1:52
interesting field of data science and machine learning. So in order to do this, what we've done is we've invited industry experts and recognized speakers
2:01
So they would come to our sessions and we will stream the sessions two times a month on Wednesdays at five o'clock Central European time
2:09
And the way that we've done this is we've started out with mathematics, with statistics and with probability theory
2:16
And then we've moved into different types of languages like Python and with R and SQL
2:22
And then we will go more into tools like Databricks and Power BI
2:27
And you will also learn how can you set up your own machine learning pipeline
2:32
And then we'll also go into more advanced topics like reinforcement learning and explainable AI
2:38
And in addition to these sessions, we will also have some more practical workshops
2:43
where you can actually put the theory into practice. into practice. Yes, and you will be able to connect with the best in class experts from all around
2:56
the world, and you will be able to join and collaborate with the community in the field
3:02
of AI and data science. And for that, you only need to follow us on Instagram, Twitter, and Facebook
3:09
And if you would like to watch our previous lectures, you can go to YouTube if you follow
3:14
the link that you can see on the screen, or you can see our upcoming sessions if you go on the
3:19
meetup link. Yes, and we would also like to take the moment here to thank our sponsors
3:27
sponsored by Microsoft, and Miles, thank you so much for sponsoring us
3:33
And we also would like to say a great thank you to our supporters like Global AI Community and
3:39
C-sharp corner. And we also thank you a lot for our awesome intro music that you could hear during
3:45
the stream that is done by Mina Marie and our amazing graphic designer, Levento Ponger, who made
3:53
all our graphics for the speaker cards, for the stream and so on. So thank you everyone a lot
4:00
Yep. So shall we get back to Seth? I think it's time to do so
4:09
Hi, Seth. Welcome back. Howdy doodle. Got to get the smoke out of here
4:19
There was a lot of smoke everywhere. Yeah, we really like it. I think it feels like a bit shady, right
4:26
It's like, hmm. What's going to happen now? Yeah, so what is going to actually happen today
4:33
So today we're going to be talking about deep learning. I am going to review some of this
4:37
Well, last time we did a talk about machine learning in general, and I motivated how it works with something called the Perceptron
4:43
Today, we're going to take that a little bit further. So there should be a little part review that I'll intersperse, and I'll show people how to use PyTorch to solve three different problems
4:53
We might not get to the last problem, depending on the time, but for sure, we'll get to the first two
4:59
Yes, and I'm really excited to hear more from you. And before we give you the stage, just a quick practical information that feel free to post your questions in the chat because Seth is going to answer them either during the session or after the session
5:16
Correct. And I think with that, we bring back a little smoke
5:20
Yes. So everyone can get ready for what's coming next. And then the stage is yours
5:32
Got to get the smoke out of here. Hello, my friends, and welcome to Getting Started with Deep Learning
5:40
My name is Seth Juarez. I work at Microsoft as a developer advocate
5:45
Good morning, Gabriel, Ashish. It's good to see you. I'm looking at the chat
5:49
If you have questions, tell me to slow down. Hurry up. Whatever you want, let me know. This is for you all
5:53
Let's start with the agenda. Today, we're going to look at machine learning is a little bit different
5:58
I'll go through this part. It's similar to what we did two weeks ago, but I'm going to go through a little bit faster
6:03
And then we're going to start talking about how data is the key. We'll talk about tensors, datasets, and data loaders in PyTorch
6:09
We'll talk about how to actually craft machine learning models using PyTorch
6:13
And then we'll talk about how to make the models better using these concepts called loss and cost
6:19
loss or cost and optimization. And then we'll talk about using a model to save, load, and use models
6:26
Remember, the thing about it is that machine learning is a bit different
6:30
Remember, programming is like this. When we have a problem, we think of a series of steps to solve the problem
6:35
We give the input to those series of steps, and then out comes an answer
6:39
Machine learning is different. Machine learning flips those things around. Instead of actually having an algorithm, what we do is we give the answers and the input to this machine learning process
6:52
and out comes an algorithm or this thing we call a model
6:57
These are the, this is the nomenclature for the things that we had before
7:01
You know, we had this, which we understand from programming parlance, but this is what we call it in Machine Learning
7:08
In comes the data, out comes the model. And once you have the model, that's great
7:14
The question is what happens in the box? Last time we actually motivated it
7:18
from first principles and built the thing. This time we're going to use PyTorch
7:22
And once you have that, then it back to the same thing we had before in comes the data into the PyTorch Out comes a model slash algorithm We use that model slash algorithm with input some code in there to run the model and then out comes the
7:35
answers. It's pretty cool stuff. But the reality is that data is the key, and I am a computer
7:41
vision person. Oh, I did not do these right. So we're going to do these three examples as fast as
7:48
we can. We did this one a little bit. Let's see if my pencil, my pen, we did this one a little bit
7:55
two weeks ago. We're going to do it again, but in PyTorch. So you can see what it looks like
8:00
Last time when we did it, we had to write everything ourselves. You're going to see that PyTorch makes it a little bit easier. Then we're going to go to this one, digits. I'm going
8:09
to make a model. It's not a perfect model, but you'll see what it's doing. And then I'll just
8:14
show you an example of this one in a bigger framework. So we're going to do these two
8:20
These two we're going to do in PyTorch, right? And then this one, I'm going to use something
8:25
called PyTorch Lightning, just to show you. My writing is terrible. Terrible. All right
8:33
so let's get to it. The question is, anytime we look at these things, this thing, right? This thing
8:40
this is a function, right? Think about it like there's a function h
8:46
and you pass in some variable x and then out comes the answer, right
8:51
This is what this model is. So we're gonna call this model h of x. Moving forward, I did this
8:57
So this is h of x. So the question might be, well, what does the x look like
9:02
So we'll start with the first problem because it's the easiest to visualize. The nine square problem is basically this one
9:08
You have pixels like this one. One pixel, another pixel, and they're grayscale
9:13
So it's a number between 0 and 255. So this is what the X is going to look like
9:19
This is what the Y is going to look like, right? So this tells me that it's darker at the top versus darker at the bottom, right
9:25
That's a machine learning problem we're trying to solve. Is it darker at the top or darker at the bottom
9:30
It's a really dumb problem, but it allows us to actually visualize what's going on
9:36
Again, so the problem we're solving, the machine learning problem today is
9:41
is it darker at the top or darker at the bottom? The X looks like this
9:49
This is the input and this is the answer, Y. Let's see how PyTorch deals with these things
9:56
We're going to look at three things, tensors, datasets, and data loaders
10:04
Let's go over to my handy-dandy here. This is how we describe things in PyTorch
10:13
Man, I don't know what's going on with me today. This is PyTorch
10:18
Notice that it's connecting to the kernel. By the way, this is Jupyter. This is Azure Notebooks inside Visual Studio Code
10:26
It's loading up this package. It looks like it's using PyTorch 1.8 with a GPU
10:32
And this is how you define tensors. Or remember, because before when we had this thing
10:42
we wanted to figure out how to describe these things and what data structures to store them
10:47
And these things are called tensors. And this is how you use them inside PyTorch
10:55
So for example, I'm making a random integers, three by three. Notice that I basically made random nine square things
11:04
And remember, it's just a square with a pixel here. And that's what these numbers are
11:09
Notice I can just make them at will. I can add numbers to them
11:14
Do you see that? There's X and Y. And now I can just add them together
11:18
It makes a new one. If I want to add in place because these tensors are super big
11:23
notice that I can add with an underscore. Let's make some more nine
11:27
Let's make our nine squares here. Notice that in this case, I'm making a huge tensor that has 400 three by three things between 0 and 255
11:41
So notice that this tensor basically holds 409 squares. And you can see that once you start to use these data structures, you can basically build a ton of things
11:52
Let's go down to the next one. You can actually reshape the way these things look. right? So here's the nine square, but flattened. There's 400 of them. Then we can do some
12:01
we can do some like multiplication, for example. Here is the X matrix multiply times this thing
12:09
But now this, this, what this does is this adds the top, the adds, the middle adds the bottom
12:13
We did this a little bit last time, you remember, but this is a way to do it in tensors. You can
12:18
also get the argmax. So for example, this one tells you it's darker in at, at the bottom
12:23
darker at the top, top, top, bottom, middle. And then you can do a ton of things
12:30
like multiply things out. And then just to show you that this is a real thing
12:36
these nine squares, the X and the Ys, they basically look like this. And now I think I
12:41
mistakenly told you top and bottom, but we're building on from the last time, we're doing top
12:47
middle, and bottom. So this one tells you it's darkest at the bottom, right? You can see all of
12:52
that goodness there. So these are tensors. Let's talk about the next thing
12:59
which is a data set. This is a cool thing inside of PyTorch
13:06
that lets you group these things together. So if you're not used to Python, that's okay
13:14
Notice that what this is doing, this is doing a data set of square items
13:19
Now, a data set is a class that lets you pull squares out
13:25
So for example, this one, there's the class. I'm going to make 256 squares
13:32
And notice that for each one, I can print square 34, square 254, and square 25
13:40
And notice that not only does each one give you the nine square flatten
13:45
but it also gives you the top, middle, or bottom thing. And you can see that this is actually really cool because ignoring this thing
13:54
this is me forcing the answer because I know the right answers. But basically, it will return the X and the Y and then the size
14:03
And these are the only two things that you really need for a PyTorch data set
14:07
which is super cool. All a PyTorch data set does is it allows you to aggregate the data
14:13
and then pull data out one by one, which is super, super cool
14:21
So that's the first thing. So we did that. The data loader is interesting
14:25
because the data loader allows us to batch things together. So let's go
14:32
I want to draw this so you know what I'm talking about. So a tensor is this thing right here
14:40
This is a way to hold numbers. Hold numbers. Right? It's just a multi-dimensional array
14:48
Think about it. The data set aggregates units of the things we trying to learn about For us it the squares Right But notice that it has tensors inside of it The data loader allows us to batch squares together
15:09
So what do I mean by that? So let's just say I want a batch of 10 squares at a time
15:14
What this will do is it will use the data set and give you 10 squares at a time
15:19
and help you load them up. Because sometimes, for example, these squares are ginormous pictures
15:23
and you have to load them by batches. And so that's what that is doing
15:28
So let's go back to this here. You can see here, I can get this notion of a square
15:34
which is two tensors. And then in the data loader, I'm able to get a batch of five
15:40
one, two, three, four, five. There's the five X's and there's the five Y's
15:45
Okay, so we did that. Fantastic. Tensors, datasets, and data loaders are PyTorch constructions to be able to get the data into a format that then PyTorch can optimize
16:00
Awesome. Let me make sure I want to get the questions here. Make sure there's no questions
16:05
Oh, okay, good. Everyone's good so far. If you're not good, let me know. Okay, so we did this
16:11
You notice that just a little reminder in a different color of what we actually did
16:15
we looked at a square data set that held all of the X's and all of the Y's
16:22
You can see here are, oops, I drew that too fast. Here is the X's
16:28
Remember, before we were able to make these things at random. And then here is the Y
16:35
And this Y basically gives me the answer from here. Okay, fantastic
16:43
And then we looked at the data loader. you recall is this was able to give us batches of five, right? You can also do some other things
16:51
with data loaders to make it more efficient to load data. All right. So creating a model. Remember
17:00
this is my favorite quote, machine learning is a lazy way of writing functions, but with data
17:06
That was Dr. Internet that said this very famous Norwegian professor. her so we did the x we know what that is what does the h look like remember we want to know top
17:20
middle or bottom we're going to top or bottom this time and just a reminder that we decided
17:24
on a construct that looked like this such that if we gave this thing uh i think it's ones here
17:32
zeros here and minus ones this will if we multiply the x with this number multiply the x with this
17:38
number all the way over, we decided that the output is either going to be negative or positive
17:45
right? And if it's positive, we guess the top. If it's negative, we guess the bottom
17:50
And this actually works, right? So if you multiply, basically, we're adding these three
17:55
These ones were zeroing out. These ones were adding and then multiplying by a minus one
18:01
And then this is a plus one. And then these sides like fight each other to the death as we add them
18:06
together and if the number is positive we guess top if the number is negative we guess bottom and
18:12
you notice that that actually corresponds with the actual thing this thing but this bias term you
18:19
recall is let's just say we want this to be like within 10 percent of the bottom what we can do is
18:26
we can add 10 percent of the bias on a positive scale and that will bias it towards answering the
18:32
top fantastic this thing that we just invented uh this w and this b is the actual model
18:40
and these are the predictions the question then becomes what about top middle or bottom
18:46
or we just make three of them uh three of them right and then it starts to look something like
18:54
this right where the w is not just a set of numbers like remember because remember four
19:01
is one one one zero zero zero minus one now it's a matrix or tensor of numbers now you're starting
19:07
to see why we do stuff with tensors and then we take the dot product between this and this the
19:12
inner dimensions need to match you remember uh uh for it to work and basically a dot product what
19:20
it does is if you take i'm gonna i'm gonna imagine if you're grabbing that that long vector and
19:25
you're turning it on its head and then you're doing nom nom nom for each of the three it's like
19:30
you're doing a dot product with each of the rows of the matrix. And basically, if you think about it
19:35
when you multiply these by one and Adam, you get this. When you multiply these by one, see
19:43
and then Adam, you get this and so on. And now you're just looking at the arg max for the right
19:49
answer. If it's dark at the top, middle or bottom, this also becomes a model like a so
19:57
So I went really fast. This is because we kind of went over this last time
20:04
I'd love to hear your questions about that, if you have any. We've basically constructed a model out of two tensors
20:11
a W tensor, that's a matrix, or a dimension two tensor, and then a bias, which is a dimension one, yeah, one-dimensional tensor
20:27
These are not scalars. Okay, so how do you build these in PyTorch
20:33
How do you actually do that? Remember before, we actually had to have W transpose X plus B
20:40
Remember, we actually had to literally write that out. We did like X dot W plus B
20:48
We had to write this in NumPy. How do you do this in PyTorch so that it's a little bit smarter
20:55
So let's take a look at that. inside of this one. Now you're probably wondering
21:01
well, when are we going to get to the deep learning? Well, you got to do with the shallow learning first
21:07
So you remember that now we have our square data set. We have our data loader
21:14
And yeah, so what we're going to do is we're going to get this going
21:19
It's connecting to the kernel. Give it a hot second. All right, so there it is
21:24
now we're literally going to define this model. That's what we're doing
21:31
All models in PyTorch inherit from nn.module. You can make as many modules as you want
21:37
The cool thing about this is because it's class-based, you can make little complex
21:42
little simple mini structures, and then you can aggregate them together in a super module, which is really cool
21:49
So there's two things that you have to do that you need to think about. The first is what are the data structures
21:54
that we want to have. Remember, we need a W and a B. It turns out that this is such a well-known problem
22:02
but this is basically called a linear model, right? And so this defines the thing
22:08
And here is where our W and B lives. Forward basically says, how do we execute H of X
22:15
And you can see here that basically I call the first layer
22:19
and then I get X out. This is how you do that W transpose X plus B
22:24
So let print out some squares We made 60 squares because we can Okay so this is how you do it I guess we get to a little bit more of what this looks like later on
22:38
But basically, to construct a model, you basically say linear model bracket, and then that's your model
22:46
I don't know why I don't just write it. write it so let's just say model equals uh a linear linear model
22:54
uh oh yeah i i gotta give it i gotta give it an x and a y i oh i have those wonderful
23:04
so let's do this um how do i move this down no that's not what i wanted to do there's a way to move these down but i oh dirf you just move it
23:20
down so notice that now what i can do is i can take square data set so for example i can take
23:28
squares i so we'll do squares uh zero right and then we'll take the zeroest part because that's
23:35
the X and then I can do squares, zero and then get the Y
23:42
I think this will work. It may be, maybe not. But you'll see it a little bit
23:49
I don't want to spend too much time here because I actually run these in a little bit
23:56
You can see, oh my gosh, I'm so dumb. I literally make it right here
24:02
there there's the linear model right it's i have to tell it the shape so it's it's nine by three
24:09
right uh there is the x uh there is the y uh and then notice that here in the in the model
24:19
there it is you can see the weight and the bias which is the w and the b okay i spent an extra
24:26
five seconds doing something that's already did because i was smarter than i thought uh there you
24:31
go so that's what this is doing notice that this linear model has inside of it the w here this is
24:39
the w and then it has the b and then you're probably wondering well where is it getting
24:44
these numbers from it invents them because we haven't really done anything with them quite yet
24:50
all right there you go notice that uh we instantiated the model by the way this was
25:00
really dumb of me because like all all this is this number is this says how many this is our nine
25:07
squares we have oh let me delete that because i want to be clear uh this linear model says that
25:12
the size of our x is nine so it's nine right so remember this this is our nine square and the size
25:21
of our answer is three for top middle and bottom okay so that's what that what's that is all right
25:29
So I saw some questions. Oh, good. Very good. Just people hanging out
25:36
I love it. I love it. Okay. Back to how we're doing it
25:42
Now the next time is, well, how do we make this model better
25:46
Because you remember, it just invented the weights and the biases. Well, how do we make the model better
25:52
How do we get the W's and B's? And we're going to do this really fast because this is something we went over last time
25:59
a little bit slower. How do we get these W's and B's in a nice way? Well, what we want to do is we
26:05
want to minimize the amount of mistakes we make whenever we use the model. And so I'm going to
26:12
invent invent is really invented by other people. I'm going to show you a formula that I want you to
26:20
look at and think about. Remember, this is the function that predicts, right? This is the actual
26:27
answer. For us, right, the x is that thing that's the nine square, right, that has the 255
26:38
138, right, this is a nine square. The y is a vector of size three that has a one, a zero
26:46
zero for top, you know, zero one. The reality is that mathematically, if I make it a minus one
26:51
it actually all the numbers come out nicer, but I didn't do that, so forgive me for that. So that's
26:55
what the y looks like. The prediction that it produces h of x, this whole thing will produce
27:04
a vector that says there's a number here, I don't know, 300, five, you know, two. And so what we have
27:13
to do is we have to compare them. So from this, we get an argmax that produces a vector that says
27:19
one zero zero, because that's what our function does. Now, here's the beauty of this. When you
27:25
subtract, when you subtract two vectors that are exactly the same, right, this minus this will yield
27:37
a zero. So this function here, right, if it's right, and we subtract off the right answer will
27:45
always yield a zero. Anytime we get it wrong, it will not be a zero. It'll be something greater than
27:52
zero, right? And then we square it. This is called the sucky function. This says how bad we are at it
28:01
with our model. And what we want to do is we want to minimize the sucky function. To be clear
28:05
remember, this is W transpose X plus B. This is the function that we're making. So how do we
28:12
minimize the amount of errors that we make such that these parameters minimize this particular
28:21
function. By the way, for those that are watching, this is the MSC function or the mean squared error
28:26
function. So just a little reminder, this kind of looks like if we jumble this all together
28:31
I probably should have used some other value other than x. Let's just use the value d, right? Let's
28:37
pretend that all of this is just D. D squared looks something like this. Now, if we're a blind
28:43
2D Mario and we want to push him down, how do we do that? Well, it turns out that Mario can do
28:51
something fun, like put his toe behind him and put his toe in front of him and then look at what
28:59
this slope is. So, for example, he can go a little bit in the direction of the Y. So, a little bit in
29:06
the direction of the y he can go in a little bit of the direction of the x and then he can divide
29:12
them dy by dx and then what he can do is he can make uh these little uh i should have said deltas
29:19
but these little deltas he can force to go to infinitely small and then this basically is a
29:25
derivative which is calculus right for measuring snow slope at every point along a function
29:33
that's infinitely like this dy and dx goes to infinity small. So what this does is this tells us
29:42
what direction we can actually go to because we want to walk in the direction
29:48
of the negative derivative is basically what's happening. All right, making the model better
29:56
So to sum up, the model is the function shape we construct. The cost or the loss function is a function that tells us how bad we are
30:06
at predicting, at predicting, which is awesome. So the reality is that we want to somehow
30:14
get the gradient, which is the same thing as the derivative, but in vector space
30:19
in tensor space, with respect to the parameters w and b of this function, right
30:26
and that tells us how to change the values of w and b in a for loop basically the the optimizer
30:35
what it does is a method for reducing how bad we are at predicting now here's the deal uh calculating
30:45
gradients of w transpose x plus b is easy but let's just do let's just do a two-layer
30:55
neural network because you're probably starting to realize that this little thing that we build
30:59
is like the beginning of a neural network hopefully you're all like yeah that's it so now
31:04
if i were to make another if i were to make w transpose uh times let's just say uh w uh z
31:12
transpose x plus some bias uh c plus b now now oh well this is this isn't even the neural network so
31:23
So let me draw the math out completely so you can see
31:26
And then I'll show you this, why this is this way in another second
31:30
So we have W transpose times some function, right? Times some function
31:35
And then we'll do, I don't know, Z transpose X plus C plus B
31:41
This is now a neural network, a two-layer neural network that uses this same thing
31:48
And if it's confusing, I'll get to it in a second. and we'll motivate this again
31:55
Taking the gradient of this is much harder. And now think of arbitrarily long structures
32:03
How do we take gradients of arbitrarily long structures? Well, it turns out PyTorch tensors are supes special
32:12
Special how? I'll show you. Okay, so there is our thing. Here is our tensors, right
32:21
I'm dividing them by 255, you know, just to get a scale between 0 and 1
32:27
Now what we're going to do is we're going to do our W. We're going to do this all by hand
32:32
So there's our W. There's our B. Notice that we're doing our H function
32:39
You can see it right here. X matrix multiplied W. This is the same thing as W transpose X plus B
32:45
The loss function is W transpose X plus B minus Y quantity squared mean
32:51
not like angry, but like, you know, 1 over N kind of thing
32:55
Let me show you something crazy about PyTorch tensors. PyTorch tensors inherently know how they are generated
33:07
How, say you? I'll show you. So I'm taking this loss function, which has a tensor
33:14
and tensors inside of them, inside of them. Here, let me show you
33:20
H. inside of them know the function that generated them. They know the function that generated them
33:35
Sorry, I have to quit Teams or you're going to hear it. Bling, bling, bling
33:40
And that's the cool thing about it, right? That these tensors know what generated them
33:46
So this grad function tells it the thing that generated it. And I can actually build a tree, right
33:53
This is not me like saying, this is me. I'll show you. Let me, I don't know
33:58
Let me, let me add like, let me add 50 here. I don't need to do that, but okay
34:04
Look at this tree all of a sudden. You see this? Now it's different because it knows like these are constants, right
34:12
These are the actual variables that we're interested in. This is W and B because those are the ones that we are interested in
34:17
These are constants. And while you're saying, well, why is the X a constant? Because it is, it's constant
34:23
And you can see that this knows about the structure of the tensors, which is awesome
34:31
Because the thing about derivatives, and you all might know this, is that derivatives of a complex function
34:39
So let's just say a gradient with respect to some parameters P of a complex function F of G of X
34:49
is the derivative of the outer one. So F of G of X times the derivative of what's on the inside
35:01
Now, if your function is like F of G of H of X
35:06
which is what these functions start to become, then this becomes the derivative of the outside
35:12
F of G of H of X times the derivative of what's on the inside
35:18
g prime of h of x times the derivative of the one that's on the inside of that
35:26
Notice that all of a sudden, the gradients of these things are chained based upon the operations
35:34
And that's why PyTorch is so fantastic, because it remembers, it remembers how those things were
35:42
generated so you can literally say loss backwards and then you can get the gradients with respect to
35:48
these variables boom huh that's why pi torch is so cool you know what i'm saying because it actually
35:58
understands not just the functions that generate not just the tensors but it understands inherently
36:06
the gradients with their spec. So this makes it so that you can create
36:11
what are called dynamic function graphs that can change with if statements
36:18
and still maintain how these tensors were generated to create, excuse me, their gradients
36:25
All right. All right, let's see. Questions here. It says, I thought we will, this is a sheesh
36:33
It says, I thought we will get model from data, but we already have the model here if i'm understanding fine uh great question ashish
36:40
it turns out that the dirty secret is you have to pick a model shape and then see if that model
36:45
shape performs well with your data there is no you have to pick the shape uh there there's no
36:53
magic there it's like it's like saying let me try this one and then if it works it's great if it
36:58
doesn't work then it's not great you try a different model shape uh in scikit-learn for
37:03
example, there's model shapes like logistic regression, decision trees, SVMs. Those are
37:09
like stamped out shapes that you do either use or you don't. For deep learning or for neural
37:17
networks or these kinds of models, you can actually construct them, which is what we're doing
37:24
Okay fantastic Now let take a look at our linear model here You remember we built this linear model We shift enter here Now just to show you this sum H of X minus Y quantity squared one over N is so common
37:46
It's mean squared error that you can see that it's already built in the PyTorch. So now notice that to run the model, right
37:54
I give it an X. I calculate the cost. And notice that I can draw this function, right
38:05
In PyTorch, you can see that there's something called the mean squared error, right
38:11
You can see that there is this thing called this I added a softmax, the cost, add matrix, multiply
38:16
All of that is in there, right? And it's built up. So anytime I want to construct a gradient of this linear model
38:24
it's built in its bananas, B-A-N-A-N-A-S. Tensors are special. It's true
38:34
Notice we have that here. I showed you that one. I also remember I actually also changed this part right here
38:43
And then this thing like got bigger because, I mean, that's how math works
38:48
it's fantastic um all right so let's put it all together now uh let's do the loop here uh i think
38:56
um let me put this down i think i have that one where do i have that one uh
39:03
the loop no where did i put the loop okay um i think it's simple
39:11
uh no this is the one we did last time squares yes here it is uh so um
39:22
uh we did this i don't think we need this one i can take this out so now uh there is the uh why
39:32
is this not happy and then it is defined stop it uh you can see that i'm doing a cost function
39:39
Let's draw the thing out. This is a little bit better, right? Because this is the straight MSE laws
39:45
the cost of the model, which is the linear model. You can see that they were super clever
39:51
because a W transpose X plus B happens so often, they made it into one single operation
39:57
So this is the constant. This is X. I'm sorry. This is B
40:03
This is the constant X. and this is W transpose X times
40:11
whoa, yeah, yeah. And then the mean squared error, where is this constant coming from
40:19
That I'm not sure about. Okay, so let's get into the actual loop
40:27
I don't know why it doesn't, it thinks it's not defined. in case I'll redefine it all right so here we go uh what we have is we're picking the device
40:40
if it's CUDA that's great this is a linear model we're moving the model to the right device
40:44
we're using the cost function and then here here is the optimization method that remember that
40:53
graph thing that we drew where we step Mario down this is called gradient descent uh stochastic
41:01
gradient descent at batching, which is what the data loader does. And that's why we have it
41:06
And so there it is. We did that. And now all we do is we go through the loop
41:14
Right? We get the X and the Y. We zero out the gradients
41:18
We predict the model. We get the loss. We do a backward. We do a step and notice
41:23
This should be going down. You can see that it's going down super slow
41:29
and now we print out the model parameters, which is the W and the B
41:34
and you're like, what? Remember, before we got ones and minus ones
41:40
let me do this. Let me be clever and let me force the issue
41:45
Let me go to the square data set, util.square. Let me force the issue here
41:54
Self.index times two minus one. Boop. Okay, let's go back to the squares
42:02
Let's clear output to restart. Okay, let's do this again. Everyone's like, what the heck are you doing
42:14
Question from Ashish, what is the CUDA device? The CUDA device is a GPU
42:20
The reason why we like using GPUs is because matrix multiplications in GPUs are what make it happy
42:28
Let's do this. Okay, there we go. That's what it is. The clear device says use the GPU
42:35
All right. So notice this loss function should be going to zero
42:41
That's the goal. Notice it's getting smaller, but not by much, unfortunately
42:47
I guess it can't go any smaller. Notice that it gets optimized pretty quickly
42:52
Loss, by the way, is our sucky function. We want that thing to go to zero
42:56
So, wonderful. How many epochs did I tell it to do it
43:01
20. Okay, so. By the way, this is the loop. It's going through the batch
43:06
The batch gets, I don't know, how many squares are we getting at a time
43:10
I don't know where the batch is. Oh, we're getting 128, nine squares at a time
43:15
And we're zeroing the grad, predicting, seeing how bad we are at it
43:19
with respect to the actual answer. We're doing backward. And the optimizer knows about the model parameters
43:26
And then it subtracts the gradients off. All right, looks like we're done
43:31
And now let's go to this. What? Now you're seeing these are close to what we thought
43:38
The ones, the zeros, right? And then the zeros, the ones, right
43:44
And so notice that it basically has reverse engineered the right answers
43:51
This tensor bias is not really needed. So these numbers are going to be mostly the same
43:58
Okay. That's cool, right? So basically in PyTorch, we've re-engineered the thing we've done before
44:07
but notice that this loop is much tighter, much tighter. We don't have to calculate by hand
44:16
what the function is for the gradient. It just knows because of the magic tensors
44:20
We don't have to keep track of the Ws and the Bs
44:26
because those are tracked inside of the model, right? And it's just a really nice way of abstracting things
44:33
Okay, so let me go back here. We did the loop, thank goodness
44:40
We did this gloriousness. It was fantastic. We also, I went in there and re-hacked it a little bit
44:47
because I wanted to show that it got ones and zeros-ish. but it's pretty cool
44:53
All right. So let's use the model here. Now that we have our model let just use it So here how you do it I making a new image right
45:06
Here are the labels, top, middle, or bottom. Notice that TorchNograd says don't keep track of the gradients, so it doesn't, right
45:14
Now there's O. Let's print O out. There it is. Top, middle, bottom
45:21
and then the answer is bottom for this one, right? Because this is the argmax
45:27
I wonder why their numbers are so negative. It doesn't matter because basically it's the bottom
45:33
and you can see that that is actually the case. So let's do this
45:38
Let's try it again. If it doesn't work, this is making me nervous. I wonder why it's not doing the right thing
45:43
So 255 should be dark at the top and indeed it is
45:49
right can you see that now let's go let's go let's do 120 let's do 12s here
45:56
and then let's 255s the middle this is how we use the model
46:04
and that's it i mean i'm there you go okay life is good so this is cool right i mean we basically
46:15
figured out how to use the model and that's basically it right it's quick review machine
46:25
learning is different data is the key tensors data sets and data loaders we crafted a model
46:29
we did a lost cost optimization save load and use digits though we promised to do some digits
46:34
well notice that now it's just like this and the vectors are just much bigger right and so instead
46:41
of this, we have something that's like this. It's bigger. Top, middle, bottom. Remember how we had
46:47
three of them? Now, if you think about it, it's like doing this, right? Top, middle, bottom
46:55
except there's only three of these. For digits, we're going to need 10 of these. But what if
47:03
what if we stacked them? What if we added another layer and then we stacked them again
47:15
Remember how we were doing that before? We had like a W transpose
47:19
times Z transpose. I'll do this wrong so you can see it
47:26
X plus C plus B. That's what this kind of thing is doing
47:30
But the thing about this is that a linear combination, This isn't a neural network yet
47:35
The linear combination of a linear combination is still a linear combination
47:40
So you get no extra power. So what they do is they add functions in between
47:47
These are called activation functions. Activation functions. By the way, I'm motivating neural networks now
47:56
So now we have W transpose times some kind of activation function
48:01
Z transpose X plus bias in there plus B, right? This is a two-layer neural network
48:08
with an activation function. You can see right there. And so now these things start to go like this, right
48:15
This is what we're going to actually build. We have the numbers coming in
48:20
and the numbers are, I think it's a 28 by 28 pixels
48:27
And then so it's 784. We go through this. We do an activation function
48:32
We do it again, activation function. And notice that as we go over
48:36
the matrices are getting smaller. And then here's another activation function. This activation function is nice
48:43
because it forces all these to sum to one, which makes it look like a probability
48:48
But it's not. What does that look like in PyTorch? It looks like this
48:55
Remember how we built the thing before? We had a linear layer. so now we have two linear three linear layers but this time when we do the forward right we
49:04
we use the activation function here and then we output the soft max over the first dimension what
49:12
let's go back to this other goodness so you can see it oh don't this time
49:19
neural network here it is sorry my kids are vacuuming uh here's what this looks like boop mean squared error loss on this
49:30
one yeah notice that we don't have to keep track of the gradients for this one because it does it
49:35
for us there's an add matrix multiply add matrix multiply add matrix multiply soft max mse boom
49:41
mean squared error has the one over n const that's the thing i'm trying to remember that's the thing
49:46
I'm remembering now. There you go. There's the activation function. I'm sorry
49:53
There's the whole model function with the loss in there. Isn't that glorious
50:01
So let's talk about, though, why this is kind of a dumb idea for pictures
50:06
So notice that the way we're doing it with the linear model is we're taking all these pixels
50:11
and we're stacking them side by side. Right? Because it's one long vector
50:17
That's what our linear model does. But the reality is that we're losing this spatial locality
50:25
across the Y direction. So there's some cleverness afoot. There's some things called filters and pooling
50:33
So take a look at this. This is a filter. So we're going to divide this up into this square
50:39
What we're going to do is we're going to multiply all these numbers together and all these numbers together
50:44
And then we're going to put it in here. then we're going to do the same thing again right and then we're going to put it in here
50:50
and then we're going to do it again and put it in here and what the heck does that do
50:54
other than make the image smaller i don't know but let's make let's make it so that keeps the
50:59
image the same size so we pad this and now when we do this boop boop right with this the image is
51:08
the same size same size with padding now what if we want to what if we want to do another operation
51:14
called pooling you're probably looking at me thinking what is he talking about what this does
51:18
is this sums these all or gets the max or the average or whatever and it puts it into here
51:25
here's the next one put it into here this is called pooling now you're probably thinking
51:31
what the heck does that do to a picture well thankfully i've actually set it up so i can show
51:41
yeah so this just does uh open and resize an image that's all it does this is a filter a
51:49
convolution it's also called a convolution it's a 2d convolution i just wrote out that operation
51:55
that i showed you um uh which is quite cool right the operation that i showed you before
52:01
um where it goes and does that thing right and then i i implemented pooling i wrote it out by
52:07
hand because I've got all sorts of time. Then remember, this is the filter we're going to use
52:12
111, 0, 0, 0, minus 1, minus 1, minus 1. Hold on
52:18
Let me see. One second All right Minus 1 minus one minus one
52:35
And notice basically I made this new W. Okay. All right
52:44
So what I'm going to do is I'm going to open an image here. It's an important image of the wedding, obviously
52:53
and I'm going to run. Let me just show the image, obviously, because it's an important image
52:58
Hold on one second. Sorry, I'm trying to get my kid to stop vacuuming
53:12
All right, told her to stop. She's still vacuuming. She's not listening to me
53:17
So here's a picture of the wedding. Obviously, soup's important. and what I'm going to do is I'm going to take this image
53:23
and do a convolution over it 2D and then show you what it does
53:27
notice that that little operation basically basically found all the edges
53:37
let me show you what a pooling does it enhances them yes
53:43
crazy let's do another one one of the kids because the kids are
53:47
super important does the same thing. Now, let me show you one that's actually really quite good, a fence
53:59
Notice that it's finding all of the edges. But basically, these convolution operators and these pooling operators
54:06
when you think about it, what they're doing is they're basically generating
54:10
new images that it can use to learn from. we're going to do this twice and then pass it into this regular linear thing and this is what
54:22
it starts to look like so if we go back to the graph here here's a convolutional neural network
54:30
for the mnist data look at this that's cool right so cool wait is this yeah this is did i do the cnn yeah cnn convolutional
54:45
You can see what makes it so cool. You see MKLDNN? It's basically now a different kind of operation
54:55
but everything else still works. It's fantastic. All right, so let's do a full run here
55:03
and let's do the digits. Let's see if I have it open
55:11
Very good. notice that this is all here let me do a thing here
55:18
because I made these graphs look really nice how did I make these graphs
55:22
look so nice oh I know where it is it's in the kernels sorry I have
55:30
too many things open save so let's go to kernels here and let me take this because I
55:36
made a cool style so that it would look nice and you can see we can run through the whole convolutional neural network i'm kind of running
55:47
out of time i feel bad so basically this draws all the digits you can see all of them in there
55:52
you can see you can see the same thing we did before the loss function i'm using is different
55:58
i'm using binary cross entropy instead of mean squared error because it's a little bit better
56:01
and then you can see it's the same exact thing we had before
56:06
And then it just runs through all of it here. And then does it work
56:13
Let's try it. I actually put this model inside of a website
56:18
Yarn dev. Here we go. By the way, let's get some questions as it's going here
56:25
So I want to make sure. Good. Everything's good so far. So let's go into this
56:34
So nice. Let's go over here. And let me draw a digit
56:46
We'll draw the digit three. What? By the way, down here, sneakily, you see this
56:55
This is a debug to see what it actually looks like. clear. Let's do, I don't know, zero
57:04
But it's not very good because I didn't train it very long. So watch, I'll draw a seven and
57:08
I'll think it's a two or let me draw a nine, right? I think it's a seven. And so that's the
57:15
reality with these models, right? That you got to work on them. By the way, if you're interested
57:18
in what this looks like or how I did it, if I go to github.com, I think it's Onyx Web
57:26
you can totally look at a sample. It's all there. So there's the index page
57:33
and then the components. I have the draw thing. You can see it's all there
57:38
You can take a look at it. Okay. All right. So we did machine learning is different
57:44
Data is the key. Tensors, data sets, data loaders, models, blah, blah, blah
57:49
Now let's do this final one that I promised that I would show you
57:52
but I'll show it to you in iTorch Lightning in the final two minutes
57:59
iTorch Lightning makes things a little bit more, how shall I say
58:08
did I do this right? I wonder why this is, hold on
58:13
Let me open this correctly. So Rochamp, there you go. That's how I wanted to open it as a workspace
58:19
Okay, so here's the AI for Rochambeau. Notice that PyTorch Lightning puts everything into a special place
58:28
So we have the model. There is the model. Oh, wow, this isn't even using a convolutional neural network, but that's fine
58:36
So here is the model. Oh, no, no, this one, I see what it's using. It's using a transfer learning
58:42
Here's the forward, and here's the compute, and then here's a training step, validation step
58:47
and a way to configure the optimizers. This is how we get the data
58:51
By the way, the data are pictures of me doing either nothing, paper, rock, or scissors, right
59:03
And basically, when you run it, let's, I have too many things open
59:09
You go to this trainer and you run it. So let me show you what that looks like
59:15
I can basically, you know, do all of this stuff. So notice it's in the right thing
59:21
And I think I just have to hit F5 to run this
59:25
Or I can just say, yeah, I think it's just F5 to run it. Let's try it
59:33
Oh, yeah, I just changed something on here. I'm sorry about that
59:40
Now I can do it. there it is it's running and basically this thing will just train it
59:51
and it will learn a difference between me doing rock, paper, inches
59:55
alright so my time is up, I'll let it run any questions
1:00:00
Eve, as we go through. And apologies, this, I'll get the, this is
1:00:08
you can totally look this up, by the way. It's on, I do it on my AI show
1:00:13
every other week. And so AI advocates, Rochambo, you can find all the code in there
1:00:18
I recently made a change that broke it and I'd have to go in there
1:00:23
and fix it and I just don't have time to do that. But basically, it's really easy to run
1:00:27
I didn't put it in the reading, but it's basically Python trainer.py
1:00:35
backslash backslash config. All right. Thank you for that. I am out of time
1:00:42
but let me answer one question. Apart from image classification, how is PyTorch being used in applications
1:00:47
What can be found in real business scenarios? PyTorch can do anything from text
1:00:53
language, speech. It's just a question of creating the model and putting the input data into tensors
1:00:59
Great question. All right. I cede my time back to you, Eva
1:01:17
Wow Seth thank you That was really exciting again I really found it really really I mean it was awesome
1:01:24
I don't even find words. I wish I had more time to go over the other one
1:01:28
It's just I want people to understand PyTorch and deep learning. It's pretty cool stuff
1:01:33
Yes, and I think they got a really good start to get into it
1:01:38
If you would be so nice to share some links with us there, they can go and look for more information
1:01:44
That would be nice. Oh, yeah, absolutely. All those notebooks I showed you are all on GitHub
1:01:51
Just Seth Juarez, github.com slash Juarez, deep learning with PyTorch. And even the slides, except I changed them
1:01:58
so they match what I said, deep learning with PyTorch. Yeah, it's all there
1:02:05
I'll commit the changes that I've made recently so you can all go and look at those notebooks
1:02:09
that I showed you. But even the slides are there too. Thank you a lot
1:02:13
That's really cool. And while we're waiting for more questions, I want to say a great thank you for your session, Seth
1:02:20
It was really exciting. And we would like you guys, the audience
1:02:24
to give us your feedback about what you think about the sessions of AI42
1:02:29
which for that survey we will share a link in the chat And if you have further questions and while you watching the recording you welcome to reach out to us on Twitter or Facebook
1:02:43
to AI42 and you can find us and we'll get you the answers
1:02:49
That's right. And sorry if you heard a vacuum cleaner. I told them to stop vacuuming
1:02:54
We actually couldn't hear it, so it was fine. Oh, good. It's my microphone
1:02:59
It's real good. It's directional. And Yves, isn't there some more interesting stuff going on
1:03:09
Oh, that's right. Don't forget about the thing. Oh, yeah. Let's not forget about that
1:03:13
Yes, some exciting stuff comes up tonight in Oslo time zone at 7 p.m
1:03:20
We're going to get started with our area at Global AI on Virtual Tour
1:03:25
So Global AI Community is going to come to Oslo tonight. and I'm going to be the host there
1:03:31
where I bring four amazing speakers talking about some great solutions in the field of AI
1:03:37
You can join us with the link that we're also going to share in the chat
1:03:44
And I think with that, we say a big thank you for everyone for joining us
1:03:48
I think it was really amazing today as well So thank you Seth for talking about these amazing topics and i hope we will see you back at some point i hope so too thanks for having me
1:04:00
and actually we will see you back quite soon actually because on in two weeks time on june
1:04:05
30 we will have a panel discussion where the topic for this is how you can turn your passion
1:04:11
in AI into your career. So Seth will be one of our panel participants and we also have JP
1:04:21
JP Vogt and Veimeng Li and Heine Ilmerinen. So we welcome all of you
1:04:29
Yes, we basically bring back all our experts of AI42 and we will talk about how can you also
1:04:35
be an expert in the field. That'll be cool. Well, I'm excited to be here. Thanks for inviting me
1:04:41
and if anyone has any questions feel free to reach out to me on social media or set.warasitmikestuff.com happy to help out
1:04:47
thank you a lot yeah thanks so much take care everyone you
#Arts & Entertainment
#Computer Science
#Machine Learning & Artificial Intelligence