1 00:00:00,560 --> 00:00:08,550 Hey welcome to this new lecture! And here you'll learn how to detect faces and we'll 2 00:00:08,550 --> 00:00:15,630 be using OpenCV with Python to detect one or more faces from an image. 3 00:00:15,630 --> 00:00:20,850 So basically how does face detection works anyway? Well the idea is that 4 00:00:20,850 --> 00:00:28,560 someone has created some Cascades which are basically some XML files such as 5 00:00:28,560 --> 00:00:38,040 this one, and this XML file contains the information about the features that an 6 00:00:38,040 --> 00:00:43,980 image of a face contains, so we're talking about a ratio of the shadows 7 00:00:43,980 --> 00:00:48,899 of the eyes, and nose, and lips, and all these features, these pixel intensity 8 00:00:48,899 --> 00:00:54,899 numbers are stored are recorded in this XML file which has been created by using 9 00:00:54,899 --> 00:01:00,030 some images with faces as training samples. So basically you tell your 10 00:01:00,030 --> 00:01:05,549 computer's ok these are, all these are faces and then you use software like 11 00:01:05,549 --> 00:01:12,780 open CV to create such XML files. So these are called haarcascades and this 12 00:01:12,780 --> 00:01:18,900 is the haarcascade for a frontal face object, and if you want other objects you 13 00:01:18,900 --> 00:01:27,000 can find them in this link here so you've got full body and left eye 14 00:01:27,000 --> 00:01:32,640 and lower body and so on. Alternatively you can use the resource 15 00:01:32,640 --> 00:01:38,030 section of this lecture and you can download a zip file with all the XML 16 00:01:38,030 --> 00:01:44,670 haarcascades there, but in this lecture we'll be focusing on frontal face, so 17 00:01:44,670 --> 00:01:50,329 we'll be using this cascade to detect faces. The way it works now is that 18 00:01:50,329 --> 00:01:57,299 we'll load the image in Python and then we will tell Python that this is a model 19 00:01:57,299 --> 00:02:01,229 that you want to look for in the image and you want to find this model, so the 20 00:02:01,229 --> 00:02:06,930 xml model in the image and what Python will do with it with the help of OpenCV, 21 00:02:06,930 --> 00:02:13,530 it will start to search all the image using a window and then it will resize the image, 22 00:02:13,530 --> 00:02:19,020 so it will decrease the image size and using the same window to detect for 23 00:02:19,020 --> 00:02:24,959 for smaller faces and so on. We will go through that step by step. So let's write 24 00:02:24,959 --> 00:02:30,150 the script that detects faces. Import cv2 is the first thing 25 00:02:30,150 --> 00:02:33,630 you want to do, and then the next thing is you want to read 26 00:02:33,630 --> 00:02:41,790 the cascade in Python, so let's call the variable face cascade cv2 and we have a 27 00:02:41,790 --> 00:02:50,010 method called Cascade Classifier, so this will create a face cascade object 28 00:02:50,010 --> 00:03:00,480 in Python and all we have to pass here is the path or the haarcascade. That's it. 29 00:03:00,480 --> 00:03:08,850 And that will create a cascade classifier object, and now you can use 30 00:03:08,850 --> 00:03:15,060 this cascade classifier object of the face feature to search for a face in your 31 00:03:15,060 --> 00:03:20,700 image, so the next thing you might want to do is to load the image in Python. 32 00:03:20,700 --> 00:03:26,820 The image that you want to search for a face. Let's say image equals, so you know that 33 00:03:26,820 --> 00:03:32,790 we can load images in Python via the image read method of the OpenCV, and 34 00:03:32,790 --> 00:03:39,630 you can just pass the photo dot jpg name here, so I'm passing the file path of 35 00:03:39,630 --> 00:03:44,730 this image, so I'm not passing any second parameter here. That means I'm reading 36 00:03:44,730 --> 00:03:49,709 the image as a color image, however a good idea is to use grayscale images 37 00:03:49,709 --> 00:03:55,230 when searching for a face so I loaded the image here, but I'll be using the 38 00:03:55,230 --> 00:04:00,660 grayscale version of the image when searching for a face in that image. That is 39 00:04:00,660 --> 00:04:05,430 thought to produce high accuracy when were searching for faces because you know you 40 00:04:05,430 --> 00:04:09,799 may notice that when you have very busy images with lots of features 41 00:04:09,799 --> 00:04:15,209 OpenCV will not be 100 percent accurate so you may get faces that 42 00:04:15,209 --> 00:04:20,489 OpenCV will miss out or you may get features that will be classified as faces. 43 00:04:20,489 --> 00:04:25,300 Using the greyscale image though increases accuracy. 44 00:04:25,300 --> 00:04:30,280 I could go ahead and pass a zero flag here so that I could read this image as 45 00:04:30,280 --> 00:04:34,240 a grayscale, but I'd like to actually keep the original image as a color 46 00:04:34,240 --> 00:04:39,790 version because I want to show the color version at the end but use the gray 47 00:04:39,790 --> 00:04:45,940 image to pass it to the methods that will be searching for the face so I'll 48 00:04:45,940 --> 00:04:51,160 create here a great image variable where I'll store the grayscale version of the 49 00:04:51,160 --> 00:04:57,340 image so cv2 and we have a method called cvtColor and that takes this 50 00:04:57,340 --> 00:05:04,540 argument, the original image of course and a flag, so an argument. 51 00:05:04,540 --> 00:05:14,410 Call cv2 color BGR2GRAY, so what this means is that this will convert the BGR image, 52 00:05:14,410 --> 00:05:22,600 so blue green red bands, it will convert it to a grayscale image. That's it, so if 53 00:05:22,600 --> 00:05:27,280 you want to quickly show this gray image if you like cv2 54 00:05:27,280 --> 00:05:38,440 imshow gray image there. We give a name for the window, gray and the image you 55 00:05:38,440 --> 00:05:46,030 want to show. Then you need to pass the wait key parameter here. Let's say zero, 56 00:05:46,030 --> 00:05:56,760 so we'll press any key and it will close the window cv2 destroy all windows. 57 00:05:56,850 --> 00:06:01,050 So you learnt these methods in the previous lectures. 58 00:06:01,050 --> 00:06:04,800 Let me execute this. So this is a grayscale version of the image. 59 00:06:13,389 --> 00:06:20,149 And now we will use a method called detect multiscale and what this method 60 00:06:20,149 --> 00:06:26,179 will do it will search for the cascade classifier so it will search for this 61 00:06:26,179 --> 00:06:34,099 frontal face XML file in our image and it will return the coordinates of the 62 00:06:34,099 --> 00:06:43,669 face in the image so for instance this is the image and what this method will 63 00:06:43,669 --> 00:06:49,909 return is it will find the face and it'll say, so it will give you the number 64 00:06:49,909 --> 00:06:56,719 of the row and the column of the upper left point of the face, so it will start 65 00:06:56,719 --> 00:07:03,229 here and it will also give you the height of the face and the width of the face, 66 00:07:03,229 --> 00:07:10,189 so we'd get a rectangle and then we would draw that rectangle in the image so 67 00:07:10,189 --> 00:07:16,399 that's basically it, but you'll understand it better, and so we need to create let's 68 00:07:16,399 --> 00:07:24,459 say a face is object variable where we'll store this X Y width and height values 69 00:07:24,459 --> 00:07:32,599 so let's quickly grab that so we need to refer to the cascade classifier object 70 00:07:32,599 --> 00:07:38,149 which is this one here, and what we want to do this with this object 71 00:07:38,149 --> 00:07:40,749 is detected multiscale and you want to detect the 72 00:07:45,199 --> 00:07:51,259 grey image and then you want to pass scale factor in there. You know that 73 00:07:51,259 --> 00:07:56,719 when you have lots of parameters and they are too many to fit in a line, after 74 00:07:56,719 --> 00:08:01,879 the coma you can just press enter and Python will read your new line as it was 75 00:08:01,879 --> 00:08:08,959 one single line, so but be sure to just press Enter after the coma, 76 00:08:08,959 --> 00:08:13,860 and so we have scale factor, and a good value to give for this 77 00:08:13,860 --> 00:08:24,800 would be 1.05. Now what does this mean anyway? Well, let's consider this image. 78 00:08:24,800 --> 00:08:30,600 What Python will do is that it will start from the original size of the 79 00:08:30,600 --> 00:08:35,849 image and it will search it will create a window that will search for faces in 80 00:08:35,849 --> 00:08:41,340 the image, so searching this area in this area in this area, and so once it does 81 00:08:41,340 --> 00:08:47,370 that, then by giving a scale factor of one point zero five you're telling 82 00:08:47,370 --> 00:08:53,010 Python to decrease the scale by five percent for the next face search. 83 00:08:53,010 --> 00:08:58,590 So what Python will do is it will down scale the image by five percent and it 84 00:08:58,590 --> 00:09:03,870 will search for bigger faces in the image so search again, search again and 85 00:09:03,870 --> 00:09:09,210 then decrease it by five percent again the image and search for bigger faces 86 00:09:09,210 --> 00:09:18,120 and so on until it goes to a final size. So that means a smaller value means 87 00:09:18,120 --> 00:09:24,750 higher accuracy. If you give for example point five Python would decrease the 88 00:09:24,750 --> 00:09:30,180 scale by 50 percent, so it will start from the original size and then it will 89 00:09:30,180 --> 00:09:36,870 go 50 percent higher, and so you don't get much accuracy with that. The profit 90 00:09:36,870 --> 00:09:42,540 with this number is the script will run quicker, so you'll have less passes to 91 00:09:42,540 --> 00:09:50,040 the image for searching for a face. Zero point five is good, okay, and then you 92 00:09:50,040 --> 00:09:56,580 have another parameter called minimum neighbors and that is usually set to 93 00:09:56,580 --> 00:10:03,600 five and what this basically is, is that this tells Python how many neighbors to 94 00:10:03,600 --> 00:10:09,300 search around the window, so you may want to experiment with these numbers 95 00:10:09,300 --> 00:10:15,180 a little bit and see which gets the better result, but these two 96 00:10:15,180 --> 00:10:22,579 are well accepted numbers. So let's do something now. Let's print 97 00:10:22,579 --> 00:10:29,410 out faces and see what this is about, so what kinds of what kind of object this is. 98 00:10:29,410 --> 00:10:40,879 And I could also print the type for faces, just like that. So I'll run the script 99 00:10:40,879 --> 00:10:45,739 now and what the script will do is it will it will read this XML file and it will 100 00:10:45,739 --> 00:10:49,600 load the image it will make the grayscale version of the of image and 101 00:10:49,600 --> 00:10:55,279 then it will detect the coordinates of upper left corner of the face in the 102 00:10:55,279 --> 00:11:01,569 image and the width and the height of the rectangle defining the face in the image. 103 00:11:01,569 --> 00:11:07,249 And then it will print out the type of the faces and it would print the faces, 104 00:11:07,249 --> 00:11:16,220 the actual faces object. So okay, press any key to exit this and so faces 105 00:11:16,220 --> 00:11:22,790 is a Numpy array, a N dimensional array object and it is an array with four 106 00:11:22,790 --> 00:11:29,509 values, so we have detected our face and these are the values basically defining 107 00:11:29,509 --> 00:11:38,149 the face in in the image, so basically what we have here is, so this is 155 108 00:11:38,149 --> 00:11:47,269 which would be the 155th column so this is the X, so the rectangle 109 00:11:47,269 --> 00:11:57,439 should start somewhere here in the forehead and this should be 83, so row 83, 110 00:11:57,439 --> 00:12:06,079 column 155, and then we have the width which is 382 and the height which is the same. 111 00:12:06,079 --> 00:12:12,470 And so we have a rectangle in face. Now let's go ahead and draw that 112 00:12:12,470 --> 00:12:20,869 rectangle in face, in the image, so we create the faces array and then what you 113 00:12:20,869 --> 00:12:26,269 want to do is access all these values of this array. To do that we can use a 114 00:12:26,269 --> 00:12:36,160 for loop, so for X Y width and height in faces. 115 00:12:36,160 --> 00:12:43,400 Well, image so we are updating the image object that we have ready here, we have 116 00:12:43,400 --> 00:12:51,250 loaded here, so we will update it by drawing a rectangle in the image and 117 00:12:51,250 --> 00:12:57,860 again you need to pass the image object in here, and then you need to pass four more 118 00:12:57,860 --> 00:13:03,860 arguments. The first argument would be the starting point of the rectangle so 119 00:13:03,860 --> 00:13:13,550 that'll be x and y, so these coordinates of the face array. Great, and then the 120 00:13:13,550 --> 00:13:18,440 next parameter is another tuple defining the coordinates of the other corner of 121 00:13:18,440 --> 00:13:24,980 the image, so we've got the top left corner in here and we got the lowest 122 00:13:24,980 --> 00:13:36,650 right corner in down here, so what that would be is X plus the width, and Y 123 00:13:36,650 --> 00:13:43,910 plus the height. That's it. Yet another parameter this would be the color 124 00:13:43,910 --> 00:13:52,190 that you want to give to this rectangle. So that comes as a BGR format, so you 125 00:13:52,190 --> 00:13:57,590 want to pass value for the blue color, so let's say 0 for blue and let's have a 126 00:13:57,590 --> 00:14:07,670 green rectangle so 255 for green, so we'll have a full green there and then 0 127 00:14:07,670 --> 00:14:12,530 for red, and you can also pass another parameter which would be the width of 128 00:14:12,530 --> 00:14:21,950 the rectangle let's say 3. Okay, and once you do that, so once you have updated 129 00:14:21,950 --> 00:14:27,680 your image you may want to show your image window on the screen. So we use the 130 00:14:27,680 --> 00:14:32,510 image show method but here we have a gray image so we want to pass the updated 131 00:14:32,510 --> 00:14:42,230 image object, and we should be good to go. Save the script, go and try to run it. 132 00:14:42,230 --> 00:14:47,240 And my system is not being able to find the face detector dot py. Yeah, I have 133 00:14:47,240 --> 00:14:55,340 messed up something with the name here so let me change this. Dot py enter, so 134 00:14:55,340 --> 00:15:02,930 nothing to do with the script and let me try again. Great, so I hope this is 135 00:15:02,930 --> 00:15:08,210 what you were expecting, so the rectangle starts here and it has a width 136 00:15:08,210 --> 00:15:14,240 and a height and it ends up in here. Sometimes though as you may already now 137 00:15:14,240 --> 00:15:19,610 you may have images that are bigger than your screen resolution. In that case your 138 00:15:19,610 --> 00:15:24,020 image would not fit in your screen, on your screen, so what you could do is 139 00:15:24,020 --> 00:15:32,090 resize the image before showing it. So let me create a resized variable here, 140 00:15:32,090 --> 00:15:40,760 and that would be equal to cv2 dot resize. And you want to pass the image, so which 141 00:15:40,760 --> 00:15:46,340 contains the rectangle there and you want to resize the image to these two 142 00:15:46,340 --> 00:15:53,330 values, so you need to set the resolution of the image here, and as you know you 143 00:15:53,330 --> 00:15:57,830 can just put some values here. Let's say 500 by 500, but these values might 144 00:15:57,830 --> 00:16:03,350 stretch out your image so you, a better solution would be to access the shape of 145 00:16:03,350 --> 00:16:09,500 the image, so the resolution of the image. And this should be the width of your 146 00:16:09,500 --> 00:16:14,210 image, so you already know this, I explained this so I'm just going through this 147 00:16:14,210 --> 00:16:20,510 quickly and to make sure you get a good size you could divide it by three 148 00:16:20,510 --> 00:16:26,600 and then again imgshape. You get the height which is the first value of the 149 00:16:26,600 --> 00:16:32,930 tuple, and divided by three again and you also want to convert these to 150 00:16:32,930 --> 00:16:41,060 integers because you might get float numbers and Python you will say that it 151 00:16:41,060 --> 00:16:46,940 cannot resize an image to float numbers so that should be okay and then you want 152 00:16:46,940 --> 00:16:51,370 to show the resized image. Quickly testing it, and I'm missing a 153 00:16:57,100 --> 00:17:02,140 a bracket here, so this should be a tuple all these, but here I'm not starting 154 00:17:02,140 --> 00:17:10,930 it -- a tuple. So bracket here and closes in here. 155 00:17:10,930 --> 00:17:18,550 That should work now. Yeah, so this was my picture, but let's try a more challenging 156 00:17:18,550 --> 00:17:29,380 picture now. This one here. So we've got two faces which are not very clear. This 157 00:17:29,380 --> 00:17:35,320 is not a real frontal face and this guy here is he has his eyes closed and his 158 00:17:35,320 --> 00:17:40,090 chin is not visible. And we've also got these two faces here in 159 00:17:40,090 --> 00:17:46,150 the newspapers which, let me open the original image from here, which have 160 00:17:46,150 --> 00:17:51,490 basically very low resolution and I think Python will not be able to detect 161 00:17:51,490 --> 00:17:57,600 these two faces, so let me check this. Here's our face detector, and we want to 162 00:18:02,440 --> 00:18:15,130 pass the name of the new image file. Save the script and go ahead and run it. 163 00:18:15,130 --> 00:18:21,340 So we've got some back to reality here, and as you see this time Python and OpenCV 164 00:18:21,340 --> 00:18:26,880 were able to detect this face, even though it's not in a very frontal position and 165 00:18:26,880 --> 00:18:34,630 obviously it was also able to detect the hand of this guy, so Python reads it as 166 00:18:34,630 --> 00:18:41,140 a face and what you can do in this case is to trick these values here, so the scale 167 00:18:41,140 --> 00:18:47,860 factor and the minimum neighbors. Something you may also be interested to 168 00:18:47,860 --> 00:18:53,350 know is the faces array here has two lists in this case, so this is the first 169 00:18:53,350 --> 00:18:59,680 face the coordinates of the first face and then the second face which in this case 170 00:18:59,680 --> 00:19:04,840 happens to be a hand, but anyway you get the idea. 171 00:19:04,840 --> 00:19:13,340 So probably by maybe using a 1.1 scale you may be able to get rid of that hand. 172 00:19:13,340 --> 00:19:21,200 So yeah, that's it. You can also try out to detect the face of this guy, but I don't 173 00:19:21,200 --> 00:19:26,360 think you'll be able to do it, so just that you know that these techniques have 174 00:19:26,360 --> 00:19:31,790 limitations, so it's a computer is not a human being, so it will always have some 175 00:19:31,790 --> 00:19:35,580 some downsides in the accuracy. I hope you found this useful though, 176 00:19:35,580 --> 00:19:37,990 and I'll see you in the next lecture.