1 00:00:00,060 --> 00:00:07,230 Alright, so I've got my motion detection program running here and as 2 00:00:07,230 --> 00:00:15,630 you see this is capturing me moving at the moment. if I move away you can only 3 00:00:15,630 --> 00:00:23,699 see some shadows there and so this is great but as I said you may want to 4 00:00:23,699 --> 00:00:28,619 store the times when an object enters the frame so that you end up with 5 00:00:28,619 --> 00:00:31,250 something like this. So this is the first entrance 6 00:00:34,160 --> 00:00:40,100 as the start time and here is where the object exited the frame and then the second 7 00:00:40,100 --> 00:00:48,210 entrance of the object and so on. So in this lecture I'll show you how to build to 8 00:00:48,210 --> 00:00:55,620 that table using this script so we will add some more code here. The way to start 9 00:00:55,620 --> 00:01:01,199 doing this is to first figure out the point, the line in the script where the 10 00:01:01,199 --> 00:01:09,000 status is changing from let's say motion to no motion has, so we have no 11 00:01:09,000 --> 00:01:18,750 motion in here so as soon as webcam triggers let's say the status is zero so 12 00:01:18,750 --> 00:01:23,430 just a similar time choosing to denote that there's no motion in the current 13 00:01:23,430 --> 00:01:29,729 frame, so then you get that first frame which has a status of zero and then you 14 00:01:29,729 --> 00:01:34,740 grade out and then you check if the first frame is None, and then you 15 00:01:34,740 --> 00:01:40,950 continue the loop again, and then you grab the second frame so the status is 16 00:01:40,950 --> 00:01:50,310 still zero when you apply the difference, the threshold, and the dilate, and then iterate 17 00:01:50,310 --> 00:01:57,750 through contours. So if you find an area that is less than, I change this earlier I put it 18 00:01:57,750 --> 00:02:03,270 10,000 so I wanted to detect bigger objects because I was closer to the 19 00:02:03,270 --> 00:02:08,069 camera so I wanted to adjust it. So depending on what objects 20 00:02:08,069 --> 00:02:13,140 you want to capture you want to adjust these pixels here. 21 00:02:13,140 --> 00:02:21,570 So 10 000 is something like 100 by 100 pixels window so we are checking if you 22 00:02:21,570 --> 00:02:26,340 don't find a window that is bigger than 10 000 then you continue to the 23 00:02:26,340 --> 00:02:31,140 beginning of the loop until you find a contour with this size 24 00:02:31,140 --> 00:02:37,860 bigger than this. When python finds that contour with that size we want to change 25 00:02:37,860 --> 00:02:52,740 the status to 1. Okay, and let us now print out that status variable, so what 26 00:02:52,740 --> 00:03:02,850 you want to do is print it at the end of your loop, so status. Let me delete these 27 00:03:02,850 --> 00:03:11,240 Numpy arrays which are being printed. We don't need those and make some room here. 28 00:03:13,790 --> 00:03:21,060 So this is inside the loop and I now expect to see the status of 1 or 0 29 00:03:21,060 --> 00:03:27,990 printed out depending on whether there is an object in the frame or not, so if 30 00:03:27,990 --> 00:03:34,020 this is working that means we now have the actual status of the frame and we can 31 00:03:34,020 --> 00:03:43,260 use that not 1 or 0 for other things, so let's see. Okay, there's nothing at the 32 00:03:43,260 --> 00:03:56,160 moment. Let me minimize this so we can see the console. This is well, great. 33 00:03:56,160 --> 00:04:02,390 So 0 is being printed out and if something shows up now you see ones. 34 00:04:03,620 --> 00:04:16,650 Again 1, 0, 1, 1, 0, great. We can now simply apply a date-time 35 00:04:16,650 --> 00:04:26,199 method, you know such as datetime.now. That would record the time of each frame 36 00:04:26,199 --> 00:04:32,620 so we don't want that exactly, we want to use the datetime method, but not 37 00:04:32,620 --> 00:04:38,800 like that because that would record times for every frame, so what we 38 00:04:38,800 --> 00:04:45,849 want instead is we want to know when the status changes from 0 to 1 which 39 00:04:45,849 --> 00:04:50,139 indicates that there is an object entering the frame, so we want to record 40 00:04:50,139 --> 00:04:55,660 that time and we also want to record the other time when the status changes from 41 00:04:55,660 --> 00:05:03,009 1 to 0, so the object exits the frame. Let's say status list that will be equal 42 00:05:03,009 --> 00:05:10,930 to empty brackets, so an empty list and then I want to append the status of the 43 00:05:10,930 --> 00:05:18,250 object to that list. So after this loop finishes which is here, 44 00:05:18,250 --> 00:05:24,760 so I want to be here the same level with the loop, so I want to apply something 45 00:05:24,760 --> 00:05:28,930 outside the loop. So status list. 46 00:05:28,930 --> 00:05:37,030 I want to append the current status of the list here, in the status list, great. 47 00:05:37,030 --> 00:05:43,599 And to illustrate to show you what this status list is a about, I would like to 48 00:05:43,599 --> 00:05:52,479 print this out actually. And I would want to print out a list outside the while 49 00:05:52,479 --> 00:06:01,389 loop so and don't want to print out every time the loop iterates. 50 00:06:01,389 --> 00:06:11,620 So no movement, movement here, and quit. So this are the statuses for each of 51 00:06:11,620 --> 00:06:18,490 the frames so 0, 0, 0 no movement, and then something entered the frame and that's 52 00:06:18,490 --> 00:06:26,110 something entered like let's say 30 frames, and then the object exits the frame 53 00:06:26,110 --> 00:06:33,670 frame, and that's it. So what we want to do now is we want to capture… 54 00:06:33,670 --> 00:06:36,790 So what's next? So the next thing is we want to capture 55 00:06:36,790 --> 00:06:45,700 the time that we switch from 0 to 1 and also from 1 to 0, so basically what we're 56 00:06:45,700 --> 00:06:53,950 interested about is to capture the lost two items of the status list, so let's 57 00:06:53,950 --> 00:06:59,740 say we have status 0, so we start with an empty list. The first item that will 58 00:06:59,740 --> 00:07:06,970 be appended to the status list is 0 and then a second item let's say will be 0, 59 00:07:06,970 --> 00:07:13,630 or let's say we'll do 1 and so we want to apply a conditional to check whether 60 00:07:13,630 --> 00:07:19,840 the last two items of release are changing. So I know this may be confusing 61 00:07:19,840 --> 00:07:25,180 but once I ride this four lines of code here you'll understand it and it will 62 00:07:25,180 --> 00:07:33,520 make a lot of sense. So we want to check if status list, so the last item of the 63 00:07:33,520 --> 00:07:40,330 status list which has an index of minus 1 remember list indexing, if that is 64 00:07:40,330 --> 00:07:53,470 equal to 1 and status_list with index of minus 2, so the item that was added first 65 00:07:53,470 --> 00:08:03,220 the item that was added a second is equal to 0, we want to record the 66 00:08:03,220 --> 00:08:10,900 datetime of this event in the list. So let's create a list here. Say times equals 67 00:08:10,900 --> 00:08:23,080 empty and then we go here, here and say times dot append datetime.now. 68 00:08:23,080 --> 00:08:28,659 You know about this function already. So we want to append the current timestamp 69 00:08:28,659 --> 00:08:41,039 when this change occurred, so that means we need to import. From datetime 70 00:08:41,039 --> 00:08:50,070 import datetime, so datetime class from the datetime library. 71 00:08:50,190 --> 00:08:56,980 So we're basically recording the time when the status changed to 1 and we also 72 00:08:56,980 --> 00:09:03,550 want to record the time when the status changes from 1 to 0, so we would have 73 00:09:03,550 --> 00:09:18,880 a similar conditional, but this would be 0. And this would be 1, okay. So what's 74 00:09:18,880 --> 00:09:26,829 happening here is let's go back to the start of the loop, so python captured the 75 00:09:26,829 --> 00:09:32,140 first frame and it converts it to gray, and blurs it, 76 00:09:32,140 --> 00:09:37,720 and applies a Gaussian blur there to the first frame and then we store the first 77 00:09:37,720 --> 00:09:42,130 frame in the first frame variable, and then we go ahead and calculate the 78 00:09:42,130 --> 00:09:48,060 difference between the first frame and the current frame. Find contours and then 79 00:09:48,060 --> 00:09:57,220 if this loop finds contours greater than 10 000, the statu variable changes from 0, 80 00:09:57,220 --> 00:10:03,579 so it started at 0 so it changes at 1 here. And that means this empty status 81 00:10:03,579 --> 00:10:10,120 list will get another value, but the first value is always 0 so the list will 82 00:10:10,120 --> 00:10:18,790 start with 0, you know, like here so the very first value is 0 and so on. Let's 83 00:10:18,790 --> 00:10:25,779 say for the next 10 frames it will be 0, 0, 0, 0, and so on. And then if an object 84 00:10:25,779 --> 00:10:33,010 enters the frame and then status changes to 1 so we'll have 0, 0, 0, 0, and then 1, 85 00:10:33,010 --> 00:10:39,979 and then what we're doing here we're comparing the last two items of 86 00:10:39,979 --> 00:10:48,379 the list, okay. And here I would like to print out the times list so that you see 87 00:10:48,379 --> 00:10:54,559 what's going on. And I expect to get an error at the first execution here 88 00:10:54,559 --> 00:11:03,919 because we have something more to do, but let's see. So yeah it says list index is 89 00:11:03,919 --> 00:11:10,970 out of range, so that means Python is having trouble accessing the second 90 00:11:10,970 --> 00:11:17,689 item of the list because the list has no items here, so it has no items, and then 91 00:11:17,689 --> 00:11:24,139 we add an item, so we add status, and then Python tries to find the second item, but 92 00:11:24,139 --> 00:11:31,819 there is no second item. So we need to do a trick here. We need to add two items, okay. 93 00:11:31,819 --> 00:11:42,679 Let's say none and none. Now let's execute the script again, so there's no object there 94 00:11:42,679 --> 00:11:53,809 at the moment. Now an object appears. It goes out again. Two times, and one more 95 00:11:53,809 --> 00:12:03,049 time, so three entrances, and I quit it. So let me go through this list now, this 96 00:12:03,049 --> 00:12:11,989 is our times list, and so this is the time, so the year, the month, the day, 97 00:12:11,989 --> 00:12:17,899 and hour, minutes, and seconds, and then microseconds in here. So this is the 98 00:12:17,899 --> 00:12:23,480 start time when an object entered the frame for the first time, and then this 99 00:12:23,480 --> 00:12:28,699 is the time when the object exits in frame, so I stayed there for two seconds, 100 00:12:28,699 --> 00:12:39,470 so 51 up to 53, and then the next object enter the frame, so from these seconds to 101 00:12:39,470 --> 00:12:48,049 these, so for two other seconds, and then another third time 102 00:12:48,049 --> 00:12:52,809 from zero to one for one second. So we've got three entrances in the frame. 103 00:12:52,809 --> 00:12:59,379 So I hope that makes sense! This is good, but sometimes see it can happen 104 00:12:59,379 --> 00:13:08,160 that when you're running the script so you have background, then object shows, 105 00:13:08,160 --> 00:13:15,399 and the object disappears, then shows again. And if you quit the program at this 106 00:13:15,399 --> 00:13:22,420 point, so let me quit it, you'll see that you have the start time here for the 107 00:13:22,420 --> 00:13:29,170 first entrance, then the end time, and then you have another start time here for the 108 00:13:29,170 --> 00:13:34,619 second entrance, but then you don't have an exit time, so you may want to add the 109 00:13:34,619 --> 00:13:40,089 time of the exit of your last object which happens to be when 110 00:13:40,089 --> 00:13:47,139 that window was quit. So you want to go here at the end, and the script is waiting for 111 00:13:47,139 --> 00:13:54,160 a key so if you press quit here the while loop will break, but before that you may 112 00:13:54,160 --> 00:13:58,959 want to check the status. So if status is one, and in that case you 113 00:13:58,959 --> 00:14:05,259 would end up with this scenario here, so if status is one when you press the Q 114 00:14:05,259 --> 00:14:14,170 key from the keyboard, you want to add another item to your time list. So times append 115 00:14:14,170 --> 00:14:25,420 datetime.now. Great, so that should work differently now, so let's 116 00:14:25,420 --> 00:14:35,860 execute the script. No object, object, no object, object in the frame, quit, and now we 117 00:14:35,860 --> 00:14:43,629 should have four datetimes here. Four timestamps, 1, 2, 3, and 4. 118 00:14:43,629 --> 00:14:51,440 So this here was recorded in this expression here, great. 119 00:14:51,440 --> 00:14:57,379 And the next thing we want to do now is to throw this list into a Pandas data 120 00:14:57,379 --> 00:15:02,480 frame and then into a csv file. So we'll have a start and an end column 121 00:15:02,480 --> 00:15:06,560 and in the start column we would have for instance this value, 122 00:15:06,560 --> 00:15:12,139 and in the end column we would have the other value, so this one here and the 123 00:15:12,139 --> 00:15:19,579 last one, this. Okay so first of all we need an empty data frame, the Pandas 124 00:15:19,579 --> 00:15:26,259 data frame, so we'll be using Pandas to create data frames of course, so let me 125 00:15:26,259 --> 00:15:37,959 import Pandas here and let's say data frame df equals to Pandas dot data frame 126 00:15:38,350 --> 00:15:49,310 with columns "Start" and "End", as easy as that, so that will create a data frame 127 00:15:49,310 --> 00:15:56,870 structure for us which has no values, but it has two columns and then the next 128 00:15:56,870 --> 00:16:03,439 thing you want to do is outside the loop, so once you have created, once you have 129 00:16:03,439 --> 00:16:08,930 generated your times list you want to iterate through that list and then 130 00:16:08,930 --> 00:16:15,439 append those values into the pandas dataframe, so you need a loop. Let's say 131 00:16:15,439 --> 00:16:25,540 for i in range, and so here you'd have to iterate with a step of two and 132 00:16:25,540 --> 00:16:32,600 you'd want to iterate as many times as there are values in the list so you 133 00:16:32,600 --> 00:16:39,620 start from zero and then you'd end up at how many values you have in the list. Well you 134 00:16:39,620 --> 00:16:47,209 can find that using the length method, so length times with a step of 2. 135 00:16:47,209 --> 00:16:52,819 Great, so and that would be, so for each iteration df, so the data frame will be 136 00:16:52,819 --> 00:17:00,259 equal to itself dot append, so you are updating the data frame and here you'd 137 00:17:00,259 --> 00:17:05,030 need to pass a dictionary, so for the start column, 138 00:17:05,030 --> 00:17:18,110 you'd want to enter times I, and for the end column, so the keys of the dictionary 139 00:17:18,110 --> 00:17:26,689 are stored, and end and you'd want to append times i plus one, and you want to 140 00:17:26,689 --> 00:17:38,600 ignore index, so equal to true. Great, so what this is doing is let's say for i in 141 00:17:38,600 --> 00:17:44,780 range this would be, let's say we have a list with six items so we have three 142 00:17:44,780 --> 00:17:54,440 start times and three end times. So that'll mean a range 0 to 6 with a step of 2, so for 0 we 143 00:17:54,440 --> 00:18:01,340 would do this, so we'd start from 0 in the loop and we would access the item with the 144 00:18:01,340 --> 00:18:08,020 zero index in times which happens to be this item for instance, so the first item. 145 00:18:08,020 --> 00:18:15,680 And then we append that item to the first row of the start column, and then 146 00:18:15,680 --> 00:18:21,650 we access the item 0 + 1 which happens to be item with index 1 which should be 147 00:18:21,650 --> 00:18:26,690 this item, so the second item, and we append that item to the end column for 148 00:18:26,690 --> 00:18:31,940 the same row, so that's it. Then we go to the next item. 149 00:18:31,940 --> 00:18:42,530 So for, so this time we would start with index 3, so 3 here add it to the start 150 00:18:42,530 --> 00:18:47,840 column and then 4 add it to the start column, to the end column, sorry, and so on. 151 00:18:47,840 --> 00:18:54,860 And lastly we would have a data frame and what you might want to do then is 152 00:18:54,860 --> 00:19:04,940 df.tocsv. Let's say times.csv, so we are exporting the data frame to a CSV 153 00:19:04,940 --> 00:19:10,540 file, great, so let's see how this will go. It's getting quite dark here, okay. And one 154 00:19:21,799 --> 00:19:35,899 more time, three, quit, and as you see times.csv was created in my file system, so these 155 00:19:35,899 --> 00:19:47,629 are the times. I can also open it using Excel, so these are the start and end times. 156 00:19:47,629 --> 00:19:55,129 If you want you can format them. Just format cells here and you can see 157 00:19:55,129 --> 00:20:07,279 the date now, great. Great, so that ends this lecture. I hope you enjoyed it. 158 00:20:07,279 --> 00:20:13,249 We have quite a good script now, and to make it more beautiful we would want to 159 00:20:13,249 --> 00:20:19,190 display these times in a graph, so that's how you turn your data into real 160 00:20:19,190 --> 00:20:24,200 information, and we'll do that through the next lectures, but before I'd like to 161 00:20:24,200 --> 00:20:29,119 introduce you to the Bokeh library which is a library for interactive 162 00:20:29,119 --> 00:20:33,739 visualizations in the browser, so please follow the lectures in the correct order 163 00:20:33,739 --> 00:20:38,000 and I'm sure you will learn a lot. Talk to you later!