1 00:00:05,390 --> 00:00:11,480 So in the last few videos, we looked at timing code snippets using the timeit module. Now interpreting the results 2 00:00:11,480 --> 00:00:13,140 reliably isn't easy. 3 00:00:13,280 --> 00:00:16,260 And we've tried to point out some of the pitfalls you may encounter. 4 00:00:16,580 --> 00:00:21,080 Now we also saw how to enable garbage collection, if that's likely to be an important factor in your 5 00:00:21,080 --> 00:00:22,020 timings. 6 00:00:22,040 --> 00:00:26,810 Now it is important to pay a lot of attention to what you're timing, and to the conditions under which 7 00:00:26,810 --> 00:00:32,150 the code's running. Now, our example in the previous video didn't come up with any conclusive results, 8 00:00:32,450 --> 00:00:36,620 and that's mainly because printing all the output is seriously affecting the results. 9 00:00:36,680 --> 00:00:41,870 So in this video, we're going to start by modifying our code, so that all three of our code snippets do exactly 10 00:00:41,870 --> 00:00:47,300 the same thing. And we'll also defer the printing until after the lists have been created, so that the 11 00:00:47,300 --> 00:00:50,410 printing doesn't interfere with the results. 12 00:00:50,420 --> 00:00:54,670 Now when changing code that's appearing in strings, or that appears in strings, as we've got here, 13 00:00:55,010 --> 00:00:57,560 it's a good idea to convert it back to code first. 14 00:00:57,680 --> 00:01:02,030 So that way, we can run the changed code to make sure there's no errors, before attempting to time 15 00:01:02,030 --> 00:01:02,600 them. 16 00:01:02,600 --> 00:01:07,190 So what I'm going to do is delete the quotes, and then take the opportunity to show you the other way 17 00:01:07,190 --> 00:01:11,860 to pass your code to the timeit function, by enclosing it as a function. 18 00:01:11,870 --> 00:01:13,660 So let's go ahead and make some changes here. 19 00:01:13,700 --> 00:01:14,620 So I'm going to go back, 20 00:01:14,630 --> 00:01:19,340 starting at the top of our code. Well actually, we're not making any changes to the location, so we'll leave that. 21 00:01:19,330 --> 00:01:23,370 So basically starting here on line 20, we're going to delete these three lines, 22 00:01:27,200 --> 00:01:33,700 and I'm going to make that, type in def nested_loop() colon. 23 00:01:34,230 --> 00:01:41,150 We're going to indent all this for code, our for loop rather. I'm going to tab that. Leave those two prints 24 00:01:41,150 --> 00:01:48,620 there for now, and we're going to delete this other code, right down to the next for loop. And we're going to 25 00:01:48,620 --> 00:02:00,960 add a function name there, so def loop_comp() colon. We're going to indent the code, tab it over. 26 00:02:01,370 --> 00:02:07,350 And we're going to delete the next batch of code, and make a bit of space here. 27 00:02:07,670 --> 00:02:13,910 def nested_comp() colon. Indent that 28 00:02:13,950 --> 00:02:19,000 code as well. Then we're gonna delete this last one here. 29 00:02:19,340 --> 00:02:23,840 Now the IDE is complaining because there should be two blank lines, before and after function definitions, sso 30 00:02:23,870 --> 00:02:25,490 I'm going to fix that before continuing. 31 00:02:33,970 --> 00:02:39,990 OK, now we still got all sorts of errors and that's because now, our dictionaries aren't defined. 32 00:02:40,260 --> 00:02:44,520 So the easiest way to fix that is to duplicate the string, then remove the quotes from the copy. 33 00:02:44,520 --> 00:02:53,850 I want to go ahead and do that. So we're going to start on copying line five, and finish on the exits 34 00:02:53,910 --> 00:02:57,650 down here. And we'll paste that down here. 35 00:03:00,980 --> 00:03:04,720 Ok, and one more line there to keep IntelliJ happy. 36 00:03:04,790 --> 00:03:05,320 Alright. 37 00:03:05,420 --> 00:03:07,480 So at this point we should be able to run the program. 38 00:03:07,500 --> 00:03:09,110 Everything works fine. 39 00:03:09,140 --> 00:03:10,750 So let's confirm that is the case, though. 40 00:03:13,980 --> 00:03:17,610 Ok, so you can see we're able to run it - no problems there, that's working. 41 00:03:17,760 --> 00:03:22,260 But note here, I'm not sure whether you noticed when I was changing that - up here now, what we're doing 42 00:03:23,580 --> 00:03:30,360 with these lines, 59 through 61, we're basically, passing a reference to the functions rather 43 00:03:30,360 --> 00:03:35,520 than a string containing the code snippets. Because of course we removed the strings and replaced them 44 00:03:36,030 --> 00:03:37,250 with functions. 45 00:03:37,440 --> 00:03:42,030 Now I kept the function names the same as the string variable names, to avoid editing those lines, 46 00:03:42,030 --> 00:03:44,220 just in case you're wondering how come it's still working. 47 00:03:44,220 --> 00:03:49,030 Now that's how you can time functions using timeit, rather than wrapping things up in strings. 48 00:03:49,320 --> 00:03:53,150 But be aware that it will only work with functions that don't take any arguments. 49 00:03:53,310 --> 00:03:58,320 If you need to pass arguments to your function, then you can't pass the function directly to the 50 00:03:58,320 --> 00:03:59,760 timeit function. 51 00:03:59,760 --> 00:04:02,960 Now there is a way around that using decorators, and we'll cover that later. 52 00:04:03,150 --> 00:04:08,910 But now though, it's time to make some changes so that all three functions create the same list, 53 00:04:09,090 --> 00:04:11,110 and also don't do any printing. 54 00:04:11,190 --> 00:04:13,760 So let's go and start creating that code. 55 00:04:14,170 --> 00:04:16,089 So go back to the first one first - 56 00:04:16,110 --> 00:04:18,500 first function, the nested loop. 57 00:04:18,630 --> 00:04:20,470 So go ahead and make some changes. 58 00:04:20,470 --> 00:04:28,800 So I'm going to start by typing result=[], so creating an empty list. Then we've got our test 59 00:04:28,800 --> 00:04:35,120 here under if the test for locations, so under that, what we want to do is append that one level back. 60 00:04:35,130 --> 00:04:39,070 We want to append back after the for loop, so result here, 61 00:04:39,470 --> 00:04:46,830 result.append parenthesis, exits_to_destination_1. Then what we're going 62 00:04:46,830 --> 00:04:57,320 to do is delete these two printouts. And finally what we want to do, is return result. 63 00:04:57,360 --> 00:04:59,200 So that's the nested loop changes. 64 00:04:59,250 --> 00:05:03,530 So for the loop comp we also want to make a change to that. So I'll just add the second line there to keep IntelliJ 65 00:05:03,530 --> 00:05:04,650 happy. 66 00:05:04,650 --> 00:05:11,310 So we're also going to create an empty list at the start. So result equals left and right square 67 00:05:11,310 --> 00:05:18,240 brackets, to create an empty list. And after the exits_to_destination_2, 68 00:05:18,620 --> 00:05:25,020 we want to type result dot append, in parentheses it's going to be exits_to_destination_2. 69 00:05:25,340 --> 00:05:27,970 We're going to remove the two print statements. 70 00:05:27,970 --> 00:05:31,760 Then we're going to do a return after the loop, 71 00:05:31,860 --> 00:05:39,050 so return result, making sure we've got 2 lines there, between the function names, function definitions. And 72 00:05:39,050 --> 00:05:40,700 then the third one, what we're going to do, 73 00:05:40,920 --> 00:05:48,770 well we really only need to delete these three lines here, because it's just printing things out, and we 74 00:05:48,780 --> 00:05:55,880 want to just return exits_to_destination_3. 75 00:05:55,890 --> 00:06:00,210 So what we should also do, is print out the results to confirm that the three functions are doing exactly 76 00:06:00,210 --> 00:06:01,210 the same things. 77 00:06:01,330 --> 00:06:02,940 Let's do that as well. 78 00:06:02,940 --> 00:06:13,330 So we do that on line 58. I'll type print parentheses nested_loop and print parentheses 79 00:06:13,340 --> 00:06:13,880 loop_comp, 80 00:06:17,710 --> 00:06:24,970 two lines there, to keep things happy, keep IntelliJ happy, print parentheses nested_comp. 81 00:06:25,620 --> 00:06:34,840 Ok. Alright, so if we now run, we should get identical output. You can see that that seems to be, 82 00:06:34,840 --> 00:06:37,510 to me, that's identical. 83 00:06:38,350 --> 00:06:42,970 So the output's identical in all three cases, and we're getting our timings below that as well. 84 00:06:43,060 --> 00:06:43,630 So all three, 85 00:06:43,630 --> 00:06:46,440 confirming now that all three functions are doing the same thing. 86 00:06:46,780 --> 00:06:51,770 And notice also now that the results of the timings have got much smaller values. 87 00:06:51,800 --> 00:06:54,880 That's because we're no longer timing all that printing. 88 00:06:54,880 --> 00:06:59,550 And I think you saw in the last video that the timings were, I think, round about point 89 00:06:59,560 --> 00:07:04,370 four, or something along those lines. So it's seems to be much more efficient than what it was. 90 00:07:04,840 --> 00:07:11,020 So in this particular case, all we can really say, with any certainty, is it's not really a huge difference 91 00:07:11,290 --> 00:07:17,530 between the three codes snippets. So Loop and comp, you can see in my case, seems to be marginally slower. 92 00:07:17,530 --> 00:07:22,650 Well in this case, the nested comp is actually slower when I ran it, but when I run it again, we find that 93 00:07:22,650 --> 00:07:28,920 nested comp and loop and comp are still pretty similar, pretty even between all three, as you can see 94 00:07:28,920 --> 00:07:30,320 there. And I'll just run it a few more times. 95 00:07:33,670 --> 00:07:40,260 So perhaps loop and comp might be marginally slower than the other two. Nested loop is actually the slowest 96 00:07:40,260 --> 00:07:42,750 of the three there. Run it again. 97 00:07:43,140 --> 00:07:48,290 Loop and comp was generally the fastest there. So it does really depend on your computer. 98 00:07:48,570 --> 00:07:53,530 And keep in mind that the differences here are being measured in milliseconds, over 10,000 runs of each 99 00:07:53,550 --> 00:07:54,190 snippet. 100 00:07:54,270 --> 00:07:59,160 So unless performance is absolutely critical, there doesn't seem to be much to choose between the three blocks 101 00:07:59,160 --> 00:08:05,220 of code, as far as performance is concerned. And that shouldn't really be too surprising, because we haven't 102 00:08:05,220 --> 00:08:09,370 done anything silly, and each of the snippets is as efficient as it can be, 103 00:08:09,420 --> 00:08:10,470 for what it's doing. 104 00:08:10,800 --> 00:08:14,750 So all three blocks of code are pretty much doing the same thing. 105 00:08:14,790 --> 00:08:19,740 It now comes back to the question that I posed, quite a few videos ago and now, "has performance suffered when 106 00:08:19,740 --> 00:08:23,130 using generators rather than list comprehensions?". 107 00:08:23,130 --> 00:08:26,030 Now we can change the last function, and see what values we get. 108 00:08:26,080 --> 00:08:31,280 What I'm going to do is just that. So I'm going to close down the Run window. 109 00:08:31,850 --> 00:08:33,440 Let's take a copy of this function, 110 00:08:38,890 --> 00:08:42,630 and we'll call this one nested_gen. 111 00:08:47,470 --> 00:08:50,670 Of course, to do that we need to change our square brackets to parentheses, 112 00:08:53,700 --> 00:08:56,700 the start and end, like so. 113 00:08:56,980 --> 00:09:01,820 And what we want to do after, to test this, is add the new function in a call to timeit. 114 00:09:01,960 --> 00:09:04,100 So let's do that. 115 00:09:04,260 --> 00:09:07,250 So I'll just duplicate that line, result_4 116 00:09:08,520 --> 00:09:13,520 This is going to be nested_gen this time. 117 00:09:13,830 --> 00:09:19,430 Then we'll also print out the results, so result_4, and of course Nested gen. 118 00:09:19,680 --> 00:09:21,300 That's what it is - 119 00:09:21,430 --> 00:09:22,720 a generator. 120 00:09:23,230 --> 00:09:26,120 So now, let's try running this and see what results we get. 121 00:09:27,780 --> 00:09:33,370 And you can see there that that's a huge difference, compared to the other three. The nested_gen function 122 00:09:33,450 --> 00:09:35,660 looks to be over 10 times faster. 123 00:09:35,970 --> 00:09:37,320 So I'm gonna run it again - 124 00:09:39,150 --> 00:09:42,990 significantly faster, and consistently faster, as you can see there. 125 00:09:43,230 --> 00:09:45,390 And once again that's not really surprising. 126 00:09:45,610 --> 00:09:50,670 The generator isn't spending time building up the lists - it's just going to return each one when we iterate 127 00:09:50,670 --> 00:09:51,720 over it. 128 00:09:51,720 --> 00:09:55,730 So before you rush off and decide that you'll use generator expressions for everything, 129 00:09:55,830 --> 00:09:59,120 it is important to realize that you don't get something for nothing. 130 00:09:59,150 --> 00:10:05,190 So creating the generator is a lot faster than using loops or list comprehensions to build up the lists, 131 00:10:05,250 --> 00:10:09,030 but we'll pay for it when we come to iterate over the generator. 132 00:10:09,030 --> 00:10:12,420 Now we can check that, by adding a loop to each of our functions. 133 00:10:12,420 --> 00:10:14,700 Now we're not going to do anything inside the loop. 134 00:10:14,700 --> 00:10:17,910 We've just seen how something like printing can make the results unreliable. 135 00:10:18,210 --> 00:10:20,770 What we want to test here, is the comparative timings, 136 00:10:20,880 --> 00:10:26,310 when we loop over the list that we've created. So I'm going to add the same loop to each of our functions 137 00:10:26,310 --> 00:10:29,860 to achieve that. So going back up to our first function. 138 00:10:33,630 --> 00:10:40,140 Alright, so I'm going to start by, after the append, I'm going to put the same code in all three. So I'll type it once and 139 00:10:40,140 --> 00:10:47,760 I'll paste it in the others. So print the result before returning. 140 00:10:47,760 --> 00:10:50,700 I'm just going to do for x in result 141 00:10:50,750 --> 00:10:57,560 colon pass. And obviously that's not being used, but we're aware of that. So I want to copy those three 142 00:10:57,690 --> 00:11:00,520 lines. I'll copy the blank as well. 143 00:11:00,550 --> 00:11:05,690 And by the way, pass is actually needed here, so that the for loop is syntactically correct. 144 00:11:05,890 --> 00:11:12,830 So I'm going to come down here, and paste the same code at the right indentation level, there. 145 00:11:15,630 --> 00:11:15,840 Right. 146 00:11:15,840 --> 00:11:16,770 And then for the third one 147 00:11:21,230 --> 00:11:28,820 Add it, obviously, before the return. W'ell add it there. This one is slightly different, because we need to go 148 00:11:28,990 --> 00:11:32,870 through the exits_to_destination_3, because we haven't got a 149 00:11:33,210 --> 00:11:36,230 local list variable, result, defined there. 150 00:11:36,570 --> 00:11:39,480 And then lastly, let's do the same for our generator. 151 00:11:42,950 --> 00:11:47,970 That's going to be for x in exits_to_destination_3. 152 00:11:48,020 --> 00:11:50,070 Now we've talked about the pass statement before. 153 00:11:50,240 --> 00:11:54,610 It doesn't do anything, but we need it to make our loops valid, or to make the for loops valid there. 154 00:11:54,920 --> 00:11:56,010 So now if we run it again ... 155 00:11:59,360 --> 00:12:01,400 this time you can see we're getting a different result. 156 00:12:01,910 --> 00:12:08,210 You can see that we're getting some consistency with all four functions now - around about the same 157 00:12:08,210 --> 00:12:08,720 timing. 158 00:12:14,080 --> 00:12:19,120 The difference really isn't huge, but there is a slight performance penalty to be paid for the saving 159 00:12:19,120 --> 00:12:20,010 in memory that we get. 160 00:12:20,170 --> 00:12:24,150 So in general, you should probably find that the generator would be slightly slower, 161 00:12:24,380 --> 00:12:27,640 and again the results may be getting skewed a little bit on my computer. 162 00:12:27,640 --> 00:12:33,390 Now there's often a trade off between speed and memory use, and we have to decide which is more important. 163 00:12:33,490 --> 00:12:38,910 Now with massive lists, the performance penalty's probably worth paying, so that our code doesn't crash 164 00:12:38,980 --> 00:12:40,460 for a lack of memory. 165 00:12:40,480 --> 00:12:44,710 Alright, so at this point we've seen a couple of ways to use timeit to investigate the performance 166 00:12:44,710 --> 00:12:51,340 of different code snippets, so it's time for challenge. What I'm going to do here is paste in a 167 00:12:51,430 --> 00:12:59,320 challenge file. I'm going to create a new file here, New > File, Python File and I'm going to call this one 168 00:12:59,330 --> 00:13:01,510 timeitchallenge. 169 00:13:04,270 --> 00:13:07,180 I'm going to paste in some code. 170 00:13:07,310 --> 00:13:09,310 I'm going to close down the Run window. 171 00:13:09,680 --> 00:13:11,120 So that's the challenge. 172 00:13:11,180 --> 00:13:14,390 You can see there, I'll just read it out briefly, that in the section on Functions, 173 00:13:14,390 --> 00:13:17,840 we looked at two different ways to calculate the factorial of a number. 174 00:13:18,140 --> 00:13:24,650 Now we used an iterative approach, and we also used a recursive function. This challenge is to use 175 00:13:24,650 --> 00:13:30,490 the timeit module to see which performs better. And I've actually included the two functions below. 176 00:13:30,680 --> 00:13:36,770 And just as a hint here, change the number of iterations to 1000 or 10000. The default of one million will actually 177 00:13:36,770 --> 00:13:38,450 take a long time to run. 178 00:13:38,990 --> 00:13:42,170 So that's the code, so you can go ahead and actually perform that challenge. 179 00:13:42,170 --> 00:13:45,570 I'm going to end the video here, and we'll go over the solution in the next video. 180 00:13:45,800 --> 00:13:50,010 And also I'm going to show you how to use the repeat function, instead of timeit, in that video. 181 00:13:50,150 --> 00:13:51,620 So I'll see you in the next video. 182 00:13:51,680 --> 00:13:52,850 Good luck with the challenge.