1 00:00:00,170 --> 00:00:06,629 Hey welcome to this new lecture and in this lecture we will be using open CV to 2 00:00:06,629 --> 00:00:12,480 capture video from the computer webcam. So you'll need OpenCV and you will 3 00:00:12,480 --> 00:00:17,580 import it as cv2. As you already know OpenCV is what we use for image 4 00:00:17,580 --> 00:00:21,420 processing, and of course video processing because video is made of 5 00:00:21,420 --> 00:00:28,710 images, so we have also of images and they are referred to as frames and these 6 00:00:28,710 --> 00:00:33,809 frames they, these images are showing one after the other in a high speed and then 7 00:00:33,809 --> 00:00:39,450 we can see those images as a video. And you'll either hate this lecture 8 00:00:39,450 --> 00:00:43,620 or love it. What we'll be doing is that we'll go in 9 00:00:43,620 --> 00:00:49,260 a really low level of image processing here, of video processing. We will be reading 10 00:00:49,260 --> 00:00:54,600 frames one by one, so images one by one and you'll see that how we use loops 11 00:00:54,600 --> 00:00:59,879 in Python to show those images and to actually build a window where images are 12 00:00:59,879 --> 00:01:03,840 showing fast and you can see it as a video, so if you have a laptop you 13 00:01:03,840 --> 00:01:08,520 probably have a built in camera. If you have a desktop without a camera you'll 14 00:01:08,520 --> 00:01:14,250 need an external camera to follow me in this lecture. And now the first thing 15 00:01:14,250 --> 00:01:19,470 you may want to do is to read a video and then once you read the video, so you 16 00:01:19,470 --> 00:01:25,530 go read a video either from the webcam or from a video file, and once you read 17 00:01:25,530 --> 00:01:30,960 the video then you can apply other operations such as graying out the video, 18 00:01:30,960 --> 00:01:36,930 or adding texture to the video, or detecting objects in the video etc. 19 00:01:36,930 --> 00:01:42,150 So the first thing you want to do is to create an object, let's say video and 20 00:01:42,150 --> 00:01:50,600 that would be equal to cv2.VideoCapture. So that's the method that 21 00:01:50,600 --> 00:01:57,689 triggers a video capture object and that gets an argument here which can be 22 00:01:57,689 --> 00:02:06,600 either a number such as 0, 1, 2, 3, or the video file path, so when you put a number 23 00:02:06,600 --> 00:02:12,460 that implies that you're capturing a video from the camera. So let's say I have 24 00:02:12,460 --> 00:02:18,640 a built in camera in my computer, but also I have an external camera. In that case one 25 00:02:18,640 --> 00:02:24,100 of my cameras will have an index of 0 and the other camera will have an index of 1, 26 00:02:24,100 --> 00:02:29,770 and if I have a third camera it will have an index of 2 and so on. So that's it. 27 00:02:29,770 --> 00:02:36,760 If you had a video file you could just say movie.mp4 or something like that. 28 00:02:36,760 --> 00:02:42,820 It depends on your video file name, and you'd have to have this file 29 00:02:42,820 --> 00:02:47,230 somewhere in your computer, but in this lecture we are talking about capturing 30 00:02:47,230 --> 00:02:52,270 video from the webcam so I have only one webcam in my computer so I'll pass 0 here. 31 00:02:52,270 --> 00:03:00,280 That's it. Once you do that you want to make sure that you will release 32 00:03:00,280 --> 00:03:07,450 the camera, so you want to access your object video dot release is the method, and then 33 00:03:07,450 --> 00:03:12,970 you don't need to pass any argument there. So let's see what we have if we 34 00:03:12,970 --> 00:03:20,590 execute this Python file, so Python capture.py is 35 00:03:20,590 --> 00:03:26,560 the name of my file. And obviously nothing happens, but on my 36 00:03:26,560 --> 00:03:33,700 side I notice that my camera was turned on for a second or for a fraction 37 00:03:33,700 --> 00:03:37,360 of a second, so the light of the camera was turned on and then it was 38 00:03:37,360 --> 00:03:43,450 closed, it was turned off immediately. So what happens is that we actually triggered 39 00:03:43,450 --> 00:03:47,860 the camera here, the webcam but then we immediately release the camera. 40 00:03:47,860 --> 00:03:53,590 So what Python will do is it will open the camera and in a millisecond or so it 41 00:03:53,590 --> 00:03:58,560 will release the camera. Let me try it again. 42 00:03:59,310 --> 00:04:05,230 Sometimes it might happen that you don't even see a light at all, so you don't see 43 00:04:05,230 --> 00:04:09,520 it because the process happens very fast. So probably your computer needs some 44 00:04:09,520 --> 00:04:14,800 seconds to actually turn on the light of your camera, and we can give it some 45 00:04:14,800 --> 00:04:21,940 seconds so. To do that you want to import the built-in time library. The time 46 00:04:21,940 --> 00:04:25,350 library provides some operations related to time. 47 00:04:25,350 --> 00:04:35,030 For example in our case we will use it to hold the script for three seconds. 48 00:04:35,030 --> 00:04:41,550 So what this will do is that Python will try to execute the script line by line, so it 49 00:04:41,550 --> 00:04:46,620 will import cv2 and time, and then it will trigger the camera, and then before 50 00:04:46,620 --> 00:04:52,650 releasing the camera we will have the script wait for three seconds. So this 51 00:04:52,650 --> 00:04:58,650 will be executed, wait for three seconds and then execute the other method. That's it. 52 00:04:58,650 --> 00:05:06,630 Now let me execute this again. Okay. The light is on one two three off, yep. 53 00:05:06,630 --> 00:05:11,640 And the video is released. Now maybe you're thinking why don't we see your 54 00:05:11,640 --> 00:05:16,950 face there showing in the video camera, in the window? Well the reason is 55 00:05:16,950 --> 00:05:23,250 that do you see any line in here that I'm showing a window? No. So why do you expect 56 00:05:23,250 --> 00:05:31,590 my face to show there? And the way we do that is by first creating a frame object 57 00:05:31,590 --> 00:05:38,430 which will read the images of this video capture object, and let me do that first 58 00:05:38,430 --> 00:05:45,840 before going into more explanations. So let me say check frame. That will be 59 00:05:45,840 --> 00:05:56,070 equal to video dot read. Just like that. So what we are reading here is we have a 60 00:05:56,070 --> 00:06:02,760 boolean data type here and a Numpy array, so let me print this out so that 61 00:06:02,760 --> 00:06:13,100 you understand better, so check first and then print frame. Okay, execute the script. 62 00:06:13,100 --> 00:06:24,740 Wait for 3 seconds and close it. So what we got here is, I'll expand this. 63 00:06:24,740 --> 00:06:33,420 So we got true4.check variable and we got the Numpy array which represent 64 00:06:33,420 --> 00:06:39,870 the image. This image is actually the first image that the video captures. 65 00:06:39,870 --> 00:06:44,610 This here, the boolean we can use it for various operations such as checking 66 00:06:44,610 --> 00:06:49,830 whether the video is running or not. So you may have to check that while doing 67 00:06:49,830 --> 00:06:54,240 your programs, and the frame is the most important one because we will use these 68 00:06:54,240 --> 00:06:59,010 frame objects, so we'll use this Numpy array which is a three-dimensional 69 00:06:59,010 --> 00:07:04,770 array because it's a color image, so it has three bands, and you know this from 70 00:07:04,770 --> 00:07:10,980 a previous lecture where we did image processing. Okay, we will loop 71 00:07:10,980 --> 00:07:17,250 through this frame and show it using the image show method, so imshow, 72 00:07:17,250 --> 00:07:23,190 image show method of the cv2 library. So we will recursively show each frame of 73 00:07:23,190 --> 00:07:29,420 video being captured, so I hope that makes sense. 74 00:07:29,940 --> 00:07:39,900 So before we do that recursively I would like to show a window down here so 75 00:07:39,900 --> 00:07:48,930 cv2.imshow, and let's put a name for the window capturing, pass the frame object 76 00:07:48,930 --> 00:07:55,260 in there, and also remember that when you pass an image show method, so when 77 00:07:55,260 --> 00:07:58,650 you're showing a window you want to make sure that the window is closed and you 78 00:07:58,650 --> 00:08:07,950 do that using the cv2.waitKey method and you can pass zero to allow 79 00:08:07,950 --> 00:08:12,450 you to press any button and close that window, and what you want to do is cv2 80 00:08:12,450 --> 00:08:19,470 destroy all windows, so when this button is pressed you destroy all 81 00:08:19,470 --> 00:08:29,310 windows. I actually want to put this up here so the video is not released before 82 00:08:29,310 --> 00:08:34,770 you press this key. So what happens is that Python executes these lines and then 83 00:08:34,770 --> 00:08:38,520 it waits for you to press a key, and then it releases the video, and then it 84 00:08:38,520 --> 00:08:44,190 it destroys the actual window that you're seeing on the screen. Okay, let's 85 00:08:44,190 --> 00:08:59,070 see what will happen now. Surprise I've got only two teeth, you do know that. Okay 86 00:08:59,070 --> 00:09:06,720 I'm joking, I have more but so what Python had did here is that it triggered 87 00:09:06,720 --> 00:09:13,740 the camera so it triggered the camera here and then it read the first frame of 88 00:09:13,740 --> 00:09:19,380 the video, so the very first frame as soon as the camera triggers it reads the 89 00:09:19,380 --> 00:09:25,080 first frame. Then we printed out that frame and it's here somewhere in the 90 00:09:25,080 --> 00:09:31,110 command line. Then the script slept for three seconds, so it didn't do anything. 91 00:09:31,110 --> 00:09:38,070 It waited for three seconds and then cv2 with the imshow method it created 92 00:09:38,070 --> 00:09:43,410 a window where it showed the first frame of the video, so the picture that you 93 00:09:43,410 --> 00:09:50,910 just saw is the first frame of the video. And then, the actual frame, the window is 94 00:09:50,910 --> 00:09:55,800 still open so it's waiting for a key for us to press, so if I press a key now 95 00:09:55,800 --> 00:10:02,040 this last two lines will be executed. So that means that the video will be 96 00:10:02,040 --> 00:10:08,970 released and this window will destroy, so press key, and yep, that 97 00:10:08,970 --> 00:10:14,939 happened. So python is processing the video frame just as simple images. That 98 00:10:14,939 --> 00:10:23,129 means you can also apply methods such as cv2.cvtColor and you can pass 99 00:10:23,129 --> 00:10:34,259 that frame in here, and then cv2COLOR BGR2GRAY, save it again in grey variable. 100 00:10:34,259 --> 00:10:38,810 So what I'm doing is that I am converting the frame, so the color frame 101 00:10:38,810 --> 00:10:50,509 into a gray image, and actually I have to do this up here, so and then I can show 102 00:10:50,509 --> 00:10:57,649 the grayscale version of the image here. Okay, let me try it again. 103 00:10:57,649 --> 00:11:04,800 So that's the grayscale version but it's showing using imshow method, so that the 104 00:11:04,800 --> 00:11:11,029 idea. Now how about showing an actual video, not a single image there. 105 00:11:11,029 --> 00:11:18,300 Well the answer is that we need to use a while loop here and the reason we need to 106 00:11:18,300 --> 00:11:23,550 do that is that a while loop what it does is that it executes some statements 107 00:11:23,550 --> 00:11:28,529 infinitely until we break the loop at a certain point using statements such as 108 00:11:28,529 --> 00:11:37,589 break for example so the idea is that we need to put all this code up to here so 109 00:11:37,589 --> 00:11:44,339 we need to put it in a while loop. I'll do just that, so you can either use a shortcut 110 00:11:44,339 --> 00:11:49,980 from your keyboard depending on your keyboard, or go to 111 00:11:49,980 --> 00:11:53,170 edit lines and then indent to indent these lines. 112 00:11:53,170 --> 00:11:58,380 So here is a shortcut, but anyway then you want to write while 113 00:11:58,380 --> 00:12:06,520 True and then everything goes inside the while loop, as easy as that. But we 114 00:12:06,520 --> 00:12:11,110 haven't finished yet. We have some more things to do here, but first of all I 115 00:12:11,110 --> 00:12:15,340 would like to actually make sure you understand the while loop, so let me 116 00:12:15,340 --> 00:12:20,980 trigger a platform console here quickly. So what one loop does is that let's say 117 00:12:20,980 --> 00:12:32,800 while True print let's say 1. If you execute that now, 1 will be printed 118 00:12:32,800 --> 00:12:41,200 infinitely, so my concert is getting very busy here and I can interrupt this using 119 00:12:41,200 --> 00:12:47,080 a keyboard interruption operation so ctrl C or control Z if that doesn't work. 120 00:12:47,080 --> 00:12:51,460 So the while loop is, while True which will be always true 121 00:12:51,460 --> 00:12:58,450 execute this lines, but if I try to run this now what will happen is that 122 00:12:58,450 --> 00:13:04,060 Python will trigger the capture, video capture and it will show the 123 00:13:04,060 --> 00:13:09,910 window, but it's just one window only so just an image, is the first image of 124 00:13:09,910 --> 00:13:16,540 the video because what what's happening is that the while loop starts, but it 125 00:13:16,540 --> 00:13:21,340 goes, it creates the first frame of the video, so the current frame which is 126 00:13:21,340 --> 00:13:27,730 the very first frame when the video starts, and then it converts the frame to grayscale 127 00:13:27,730 --> 00:13:37,240 image, and then it sleeps for 3 seconds, ok. Then it shows the image, and then what 128 00:13:37,240 --> 00:13:43,030 the while loop does here it waits for you to press a key, and then if you press 129 00:13:43,030 --> 00:13:53,230 a key the process stops. So what you can do in this case is to enter another, let 130 00:13:53,230 --> 00:13:59,230 me make some space there, so you need to alter these values of the arguments. When we 131 00:13:59,230 --> 00:14:04,170 enter 0 it means that any key that you press on the keyboard it will close the 132 00:14:04,170 --> 00:14:09,720 the window but if you enter let's say 2000, that means the script will wait for 133 00:14:09,720 --> 00:14:23,220 2000 milliseconds which is 2 seconds and let's see what will happen here, so it 134 00:14:23,220 --> 00:14:28,890 shows my face first and then it waits for 3 seconds here, then it waits for 135 00:14:28,890 --> 00:14:34,860 another 2 seconds and then it shows the other frame so it's very slow, just like 136 00:14:34,860 --> 00:14:44,580 that. Ok, now if you press any key nothing will happen because you removed 137 00:14:44,580 --> 00:14:49,080 the zero from here. So in that case I'll forcely stop 138 00:14:49,080 --> 00:14:58,230 the script here. Control C. Ok, we don't need this time sleep anymore and we also 139 00:14:58,230 --> 00:15:04,920 need a conditional here, and before doing that we will store this action in a key 140 00:15:04,920 --> 00:15:14,610 variable, and then we say if key is equal to ord method, and then you pass a key 141 00:15:14,610 --> 00:15:26,220 from the keyboard so if this equals to q, then break the while loop and let's put 142 00:15:26,220 --> 00:15:33,660 it 1000 this time. Okay, so now what this will do is that it will capture the 143 00:15:33,660 --> 00:15:39,300 video, it will start the video and then it will iterate through all these lines 144 00:15:39,300 --> 00:15:46,020 so it will create a frame converted to gray, show the frame here, and then it will 145 00:15:46,020 --> 00:15:52,950 wait for one second and if you press the q key from keyboard it will break the 146 00:15:52,950 --> 00:15:57,570 while loop, and then if you if it breaks the video dot release it will be 147 00:15:57,570 --> 00:16:01,560 executed so the video will stop and the window will be destroyed. 148 00:16:01,560 --> 00:16:05,550 But if you don't press any key this will wait for one second, and then it will 149 00:16:05,550 --> 00:16:08,940 show the next frame, and then after one second it will show the next frames, 150 00:16:08,940 --> 00:16:16,760 and so on. So save the script, execute. So the video is improving. 151 00:16:21,949 --> 00:16:26,800 Yeah, looks good now, okay. If you press q, it will stop it. 152 00:16:26,800 --> 00:16:39,889 Okay, let's improve it more. Why not for one millisecond? Hey, hi you're doing? 153 00:16:39,889 --> 00:16:49,910 So it's working good, and you can also see here that the check booleon, the Numpy 154 00:16:49,910 --> 00:16:54,889 array is being printed at every iteration, and we can stop this now we if 155 00:16:54,889 --> 00:17:01,130 we press q. Just like that. Now if you want to know how many frames 156 00:17:01,130 --> 00:17:07,640 are being generated there, there is a nice trick we could do. We could create 157 00:17:07,640 --> 00:17:14,079 a variable, let's say a equals to 1 outside the while loop and then here you say 158 00:17:14,079 --> 00:17:25,880 a equals to a plus 1, and then after the loop breaks here you want to print a. 159 00:17:25,880 --> 00:17:31,030 So what this will do is when you run the script, it gets a value of 1 and then 160 00:17:31,030 --> 00:17:36,909 when the while loop starts running, so it could go through the first iteration and 161 00:17:36,909 --> 00:17:45,679 it will increase the a value by 1 so a equals to 1 plus 1 so it will get 2, so it 162 00:17:45,679 --> 00:17:51,470 will be updated to 2, and then it will go up to the end, and then it will start up 163 00:17:51,470 --> 00:17:58,370 again, so this line will be a again equals to 2 plus 1 so a will be 3 and 164 00:17:58,370 --> 00:18:05,059 the second iteration, and so on. Or you can put this to 1 actually, so if we have 165 00:18:05,059 --> 00:18:09,919 2 iterations that means a will be 2 and if we have 3 it will be 3 and so on. 166 00:18:09,919 --> 00:18:15,620 And then we print a here outside the loop. Ok, let's see what we have. I'll keep the 167 00:18:15,620 --> 00:18:21,220 video running for 3 seconds and we will see how many iterations we will have. 168 00:18:21,220 --> 00:18:30,590 So capture one, two, three and stop. So the last value that was printed out 169 00:18:30,590 --> 00:18:38,299 here is a value. That means we had 51 iterations in about three seconds which 170 00:18:38,299 --> 00:18:46,220 is let's say around 20 frames per second. So our camera, my camera is capturing 20 171 00:18:46,220 --> 00:18:53,630 frames per second, and let me execute this one more time, the last time, so if 172 00:18:53,630 --> 00:18:58,610 I put my finger here in camera so I blacked out the image and you'll see 173 00:18:58,610 --> 00:19:04,850 that the Numpy array a is now 000 because the frames of the image all 174 00:19:04,850 --> 00:19:12,799 it has is zero values, so there's zero intensity in the image. Okay that's 175 00:19:12,799 --> 00:19:17,360 what I wanted to show you in this lecture, so as I said once you have these 176 00:19:17,360 --> 00:19:22,429 frames you can apply operations to them, so you can draw rectangles there, 177 00:19:22,429 --> 00:19:27,379 detect images and so on. And we will do that, so we will build our real world 178 00:19:27,379 --> 00:19:34,009 application by implementing some motion detection in our frames and then we will 179 00:19:34,009 --> 00:19:39,470 capture the times that an object entered our frame, our video, and we'll have a list 180 00:19:39,470 --> 00:19:44,000 of those times. So I hope you loved this lecture, not hated it, 181 00:19:44,000 --> 00:19:44,999 and I'll see you in the next one.