Up next in 10
Build Real Time Face Detection With JavaScript
10K views
Feb 7, 2024
Embark on a cutting-edge exploration of computer vision with this tutorial on "Building Real-Time Face Detection with JavaScript." In this video, you'll delve into the exciting world of facial recognition technology, learning how to leverage JavaScript and advanced libraries like face-api.js to create a real-time face detection system. From setting up the HTML and JavaScript environment to implementing face detection algorithms, this tutorial caters to developers eager to delve into the realm of computer vision. Explore the intricacies of detecting and tracking faces in live video streams, offering practical applications for various projects. Whether you're a JavaScript enthusiast or a developer looking to incorporate facial recognition into your applications, this guide provides a clear and comprehensive step-by-step walkthrough. Join this tutorial to bring the future of technology into your projects, showcasing your ability to build real-time face detection systems with the power of JavaScript.
View Video Transcript
0:00
Do you like robots? Because I sure do. So in today's video I'm going to be using artificial
0:04
intelligence and face recognition to determine my emotions through my webcam in real time
0:09
Also, if you enjoy this video make sure to let me know down below so that I can create a part 2 to this video where I use face recognition in order to determine who is in a picture
0:17
and display their name next to their face. Let's get started now
0:24
In order to accomplish this face recognition we're going to be using a library called
0:28
faceapi.js you can see over here on the right and this is a wrapper around tensorflow which
0:32
is one of the most popular machine learning libraries out there and this is going to allow
0:36
us to do real time face detection in the browser. And it's really easy to get set up with. The
0:41
first thing that you need to do is download this library. There's going to be a link in the description below where you can download this library from. Also, you're going to need
0:48
to download the models that you'll be using for your face detection. When we get to the
0:52
model section I'm going to list off all the models that we use so that you can download them. Also, all of the models are going to be available on my github in the source code
0:59
for this video so you can get them there as well. To get started we just want to create
1:04
a blank html page here and inside of this we want to put a video element because this
1:08
is going to be where we render our webcam where we do our real time face detection
1:12
And this is just going to have an id of video so we can easily access it in the javascript
1:17
We're just going to give it a width here. You can use whatever width and height that you want but in our case I'm going to use 720 and I'm going to use a height of 560
1:25
And you want to make sure you just set this to be autoplay and muted because we don't actually want any sound. And you have to make sure you specify a width and a height on here
1:32
otherwise the face detection will not be able to draw properly onto your webcam. Now once
1:37
we have that done I'm just going to add a little bit of basic styling to our sheet
1:41
So we can come in here and add a style tag. And really all I'm going to do is just style
1:45
the body of this so I'm going to give it no margin. Make that zero. Same thing with padding
1:51
here we're going to do zero. And essentially the reason I'm doing this styling is just
1:55
to get the webcam centered in the screen. So I'm going to set the width to 100 view
1:59
widths. The height is going to be 100 view heights. And I'm going to change this to be
2:04
display flex just to find the content in the center and align the items. Items in the center
2:12
And this is just going to put the webcam in the very center of the screen. So we can just
2:16
open this up with live server just to see what we have working with. And here we go
2:19
it's loading up and obviously nothing's going to render for our video because we haven't actually hooked up our webcam into our video yet. So let's go into our script here and
2:27
work on doing that. First we need to get that video element. So we're going to create an element here called video. And we can say document dot get element by ID. And we gave
2:35
an ID of video and this is going to be our video tag. Then we can create a function called
2:39
start video. We're going to use this to hook up our webcam to our video element. So in
2:44
order to get the webcam we need to use navigator dot get user media. And this is going to take
2:49
an object as the first parameter. And this just says what we want to get. And we want
2:52
to get the video element. So we're just going to say video is the key and an empty object as the parameter. Then we're going to have a method here which is going to be stream
3:00
This is essentially what's coming from our webcam. So what's coming from our webcam we
3:04
want to set as the source of our video. So we'll say video source object is going to
3:09
be equal to that stream. And then lastly we have an error function that we can call here
3:14
So if we do get an error we just want to log that. So we'll say console dot log error
3:18
Oops error just like that. And let's make sure it's an error log instead of a normal
3:23
log. And now if we call that function we can just say start video and go into our HTML
3:28
Make sure to include that. So up here we're going to include our script tag. Make sure
3:32
we set it to defer. And we want the source of that to be our script dot JS. Now we save
3:37
that. You should see that it's going to access our video. It'll load up right here. And this is just a live preview of my webcam. It may be slightly delayed but that's just because
3:45
it's going through the browser as opposed to going directly into my recording software
3:49
So now once we have that done let's also include our face API library while we're at it. So
3:53
we can do a script here. We want to defer this one as well. And we want the face API
3:57
And we want to make sure that this is defined above our normal script so that it gets loaded
4:01
before we actually run our script. And now we can work on actually setting up our script
4:05
in order to detect my face as opposed to just render my video. In order to do that we need
4:09
to load all of the different models. So let's do that at the very top. And this is all done
4:12
asynchronously. So we want to use promise dot all which is going to run all of these
4:16
asynchronous calls in parallel which will make it much quicker to execute. And in here
4:20
we just pass an array of all of our promises. And what you do is you want to call face API
4:25
dot nets. And then this is going to be where you call all the different models you want
4:29
In our case we're using the tiny face detector. This is just like any normal face detector
4:33
but it's going to be smaller and quicker. So it'll run in real time in the browser as opposed to being very slow. And we want to say load from, whoops if I spell it properly
4:42
load from URI. And in here we have a models folder with all of our different models. So
4:46
we're going to just pass that in. We'll just say slash models. We wanted to do this a couple
4:50
times for all of our different models. So let's copy that down four times. And we have
4:54
our tiny face detector. The next one that we have is going to be our face landmark
5:00
whoops, landmark 68 net. And this is going to be able to register the different parts
5:06
of my face as opposed to my mouth, my eyes, my nose, etc. Next thing that we're going to have
5:10
here is face recognition. I can spell that recognition net. And this is just going to be
5:17
able to allow the API to recognize where my face is the box around it. And then lastly
5:22
what we're going to have is face expression net and whoops expression just like that. And what
5:27
this is going to allow it to do is to recognize when I'm smiling, frowning, happy, sad, etc
5:32
So what we want to do now is promised at all is going to take a dot then and after we're done
5:37
with this, we want to call our start video. So we can remove this here. And now once we're done
5:41
loading all of our models, it's going to start the video over here on the side. And it may take
5:45
a little bit longer because loading these models does take a little bit of time, but it's fairly
5:49
quick. Then we can actually set up an event listener. So we can say video dot add event
5:54
listener. And what we want to do is we want to add event listener for when the video starts playing
5:58
So then when the video starts playing, we're going to have our code down here, which we can say to
6:02
recognize our face. For now, let's just do a simple console dot log. And we just log whatever
6:08
come over here and inspect that. And as soon as the video starts, you see we get that log down
6:12
here at the bottom, which means we know that everything's working so far. Now we can work on
6:16
actually setting up the face detection. And this is actually incredibly straightforward to do
6:19
what we want to do is we want to do a set interval so that we can run the code inside of this
6:23
multiple times in a row, we want to make sure it's an asynchronous function, because this is going to
6:27
be an asynchronous library. And all we need to do inside of here is get our detections. So we're
6:32
going to just say detections is going to be equal to awaiting our face API, we'll just say face API
6:38
dot detect all faces. And this is going to get all the faces inside of the webcam image every single
6:44
time that this gets called. And we're going to do this, for example, every 100 milliseconds
6:49
And what we want to do instead of here is we want to pass the element, which in our case is a video
6:52
element, as well as what type of library we're going to use to detect the faces. In our case
6:56
we're using the tiny face API. So we'll say face API dot tiny face, whoops, face detector
7:04
options. And this is just going to be empty, because we don't want to pass any custom options
7:09
because the default is going to work perfectly fine for our scenario. Then we also say what we
7:13
want to detect these faces with. So we're just going to say with face landmarks. And this is
7:18
going to be for when we actually draw the face on the screen, we're going to be able to see these
7:22
different sections. So the face landmarks is going to be the different dots that you'll see on my
7:26
face. And then we can say dot with face expressions. And this is going to be able to determine whether
7:32
I'm happy, sad, angry, upset, whatever, based on just the image that it gets of my face. So let's
7:39
make sure I spell that properly. There we go. Now we can just log out these detections. So we're
7:43
going to say whoops, console dot log detections, just so we can see that if this is working
7:50
and we can go inspect over here. And you can see that we're getting error immediately. And it's
7:53
saying tiny face detector options is not a constructor. And that's super easy to fix. This
7:58
just needs to be a capital T over here, we can save this. And now we should be able to actually
8:02
get our detection showing up over here. So let's make sure there we go. And you can see we have a
8:06
bunch of objects in here. And we just have one element in the ray because there's only one face
8:10
currently. And this is all the different detection information, expressions, etc. And what we want to
8:15
do is actually display this on the screen. And to do that, we're going to be using a canvas element
8:19
So inside of our index dot HTML here, we just want to style this canvas, it has to be positioned
8:24
absolute so that it positions directly over top of our video element. And we don't actually need
8:28
to put the canvas element inside of our HTML, because we can do that in our JavaScript. So
8:32
let's do that. Now we can just say canvas is going to be equal to face API dot create canvas
8:39
from media, media, and we want to create it from our video element. Then what we want to do is just
8:45
add that canvas to the screen. So we're going to say document dot body dot append. And this is
8:50
just going to put this at the very end of our page. And since it's positioned, absolutely, it doesn't really matter where it goes. And then what we want to do is get the display size of our
8:58
current video. And this will be so that our canvas can be sized perfectly over a video. So this is
9:03
just going to be an object with a width, which is just video dot width. And it's going to have a
9:08
height property, which is video dot height. Now that we have that out of the way, we can actually
9:12
work on displaying our elements inside the canvas. So we want to take our detections, which
9:17
we have right here, create a new variable, which is going to be called resized detections. And we
9:22
just want to set this to our face API dot resize results. And we want to pass in here the detections
9:29
that we have, as well as the display size. And this is just going to make it so that the boxes
9:33
that show up around my face are properly sized for the video element that we're using over here
9:37
as well as for our canvas. Let's make this a little wider, so it's easier to see. And now
9:41
all we need to do is actually just draw this. We can say face API dot draw dot draw detections
9:48
whoops, detections. And what we do is we pass in the canvas we want to draw onto, as well as our
9:53
resize detections. And make sure I spell face API correctly over here. Let's save that and see how
9:58
this works. It's going to load in my face over here. And as you can see, it's already got a
10:02
problem because our canvas isn't being cleared. And we have our video element being shown up over
10:07
top, which is definitely not what we want. The first thing we can do to fix this is to actually
10:10
clear our canvas before we draw it. So let's remove this console dot log. And instead of here
10:16
right after we get our detections, right after we resize everything, and right before we draw
10:19
we want to take our canvas, and we want to get the context, the 2d context, this is just a two
10:24
dimensional canvas, we just want to clear it. So we'll say clear rect, and we want to clear it from
10:29
00. And the width is going to be just the canvas dot width, and the canvas dot height for the
10:35
height, this is going to clear the entire canvas. Also, we want to make sure we match our canvas to
10:40
this display size. So we can say face API dot match dimensions, oops, dot match dimensions
10:49
And we put it in our canvas, as well as our display size, make sure we spell dimensions
10:54
properly. And now if we save that, we should get much better detection over on the side
10:59
And if we just give it a second, you can see that it's detecting my face. And as I move
11:03
it's following me around. It also has a number which is how percent sure it is. So it's about
11:07
90% sure that this is a face, which is perfect. So now we can actually start drawing even more
11:11
details, we can go into the face API dot draw again. And this time, we can draw the landmarks
11:16
So we can save your face landmarks, this is going to take the canvas as well as the resized results
11:22
And if we save that, whoops, resized results, save that again, let it refresh over here. And
11:29
now you'll see that it'll actually draw some lines and dots on my face based on where the landmarks
11:32
in my face are. And you'll notice that's actually not working. And that's just because this should
11:36
be called resized detections, not resized results. So let's save that, refresh it again, and let it
11:41
do its work. And if we wait a second, you'll see that it now has all the different face detection
11:45
it knows where my eyes are my eyebrows, my actual face shape, my mouse nose, etc. And lastly, if we
11:51
want to determine if I'm happy, sad or whatnot, based on this image, we can go into the face API
11:56
here again, dot draw. And this time, we want to draw the face expressions. And this is going to
12:03
take a canvas element, as well as it's going to take our resized detections again. Now if we save
12:09
that, it'll be able to determine my emotion based on just my image alone. So you can see that it's
12:13
about 100% sure that I'm neutral. But if I make a surprised face, for example, it says it's 100
12:19
sure I'm surprised. And if I look angry, it'll say I'm angry, and so on, which is really impressive
12:24
And that's all it takes to create this simple face detection algorithm. If you want more artificial
12:28
intelligence and face detection, let me know down in the comments below. And I'll definitely make videos like that. Also, check out some of my other project based videos linked over here and
12:36
subscribe to the channel for more videos just like this. Thank you very much for watching and have a good day
#Intelligent Personal Assistants
#Machine Learning & Artificial Intelligence
#Programming