WEBVTT 1 00:00:01.930 --> 00:00:05.790 so moving on although the pickle module which we just talked about 2 00:00:05.790 --> 00:00:11.090 great for serializing python objects it really does have a drawback that the 3 00:00:11.090 --> 00:00:15.310 objects all have to be loaded back into memory into the computer's memory that's 4 00:00:15.310 --> 00:00:19.380 fine for many situations but if you're dealing with a really large set of 5 00:00:19.380 --> 00:00:23.140 objects for example a single object very large single object like a large 6 00:00:23.140 --> 00:00:27.110 dictionary for example then loading that in entire thing to memory may not be a 7 00:00:27.110 --> 00:00:32.079 realistic option so you need to look at alternatives now Python's got your 8 00:00:32.079 --> 00:00:36.750 back to their and they have an alternative it and that's to use the shelve module so 9 00:00:36.750 --> 00:00:40.649 the shelve provides a shelve and you can think of it like a dictionary 10 00:00:40.649 --> 00:00:46.059 but its actually stored in a file rather than in memory now like 11 00:00:46.059 --> 00:00:50.079 a dictionary the shelve holds key value pairs and the values can be anything they can be 12 00:00:50.079 --> 00:00:54.129 pickled which as you found out was just about everything now the keys themselves 13 00:00:54.129 --> 00:00:57.920 must be strings unlike a dictionary where the keys can be immutable objects such as 14 00:00:57.920 --> 00:01:02.170 tuples so important distinction there now all the methods we use with dictionaries 15 00:01:02.170 --> 00:01:06.840 can also be used for shelve objects so it can be really useful to think of them as 16 00:01:06.840 --> 00:01:10.720 a persistent dictionary in fact it's a very easy to convert code using a 17 00:01:10.720 --> 00:01:15.280 dictionary to use a shelve instead because the values are pickled when saved the 18 00:01:15.280 --> 00:01:19.710 same warning applies about using files from untrusted sources so make 19 00:01:19.710 --> 00:01:23.520 sure that you keep that in mind because loading a shelf can also execute 20 00:01:23.520 --> 00:01:28.210 code that's not a reason not to use them because they are very useful but 21 00:01:28.210 --> 00:01:33.369 just don't you shelve files themselves from untrusted sources so let's have a 22 00:01:33.369 --> 00:01:37.259 look at a shelve in action and just to get an idea of how they can be use just like 23 00:01:37.259 --> 00:01:40.549 dictionaries were gonna start with same examples as used in discussing 24 00:01:40.549 --> 00:01:52.130 dictionaries so I'm gonna start by typing import.... 25 00:02:12.260 --> 00:02:50.510 ..and we will just copy and paste some of these.. 26 00:02:50.510 --> 00:03:12.639 ...next one....and next one... 27 00:03:12.639 --> 00:03:14.689 ....and last one of course is lime.... 28 00:03:14.689 --> 00:03:34.830 ...ok and I will also do here, print.... 29 00:03:34.830 --> 00:03:47.600 and print....ok so there is the base program, so you can see we are opening a shelve in the same way as opening a 30 00:03:47.600 --> 00:03:53.500 file we are using the width and shelves keep in mind are read and write by natures so you 31 00:03:53.500 --> 00:03:56.630 need to specify mode but other than that it looks like just like a 32 00:03:56.630 --> 00:04:05.090 real file so we just run and we got an error and this is quite useful to know because of 33 00:04:05.090 --> 00:04:11.600 what's happened here the reason you might think this Attribute shelve module has no open, when I just told 34 00:04:11.600 --> 00:04:15.790 shelve.open is the syntax and the reason is that I've used a file here 35 00:04:15.790 --> 00:04:21.000 python file called shelve.py and what has caused confusion is the face that 36 00:04:21.000 --> 00:04:25.170 python file name is called shelve we're trying to import import shelve so 37 00:04:25.170 --> 00:04:30.270 I'm gonna rename this to fix this refactor it click on shelve.py 38 00:04:30.270 --> 00:04:36.650 right click refactor rename, and will call it shelve. 39 00:04:36.650 --> 00:04:46.190 example.py, refactor so we got the name change there and that automatically 40 00:04:46.190 --> 00:04:49.360 refactor for us which i don't want so I'm gonna change that back to 41 00:04:49.360 --> 00:04:59.180 shelve and so the example should now work because before when we did something like open we 42 00:04:59.180 --> 00:05:02.370 probably wouldn't found that to come up with right away because it be looking 43 00:05:02.370 --> 00:05:06.180 in the current Python program for the open method 44 00:05:06.180 --> 00:05:10.550 open function which doesn't exist so this should now work so keep that in mind if 45 00:05:10.550 --> 00:05:16.150 you get a weird error and using an import make sure your imported your using and the file name 46 00:05:16.150 --> 00:05:21.200 are actually separate things ok lets try running it again okay so this 47 00:05:21.200 --> 00:05:25.550 time we actually go it to work and again just a confirmation, actually I'll closed down 48 00:05:25.550 --> 00:05:32.410 this window because we don't need it for a while so you can see now that the example work and we got 49 00:05:32.410 --> 00:05:36.330 to things printed out there as well which is sort of confirmation that we 50 00:05:36.330 --> 00:05:42.380 successfully save that information so we opened the shelve in a very similar way we to open a file and 51 00:05:42.380 --> 00:05:45.650 as I mentioned shelve are read and write by nature so there's really no need to specify the 52 00:05:45.650 --> 00:05:47.110 mode but 53 00:05:47.110 --> 00:05:51.550 other than it does really look like opening a file now note that the file is or may be 54 00:05:51.550 --> 00:05:55.720 created with a DB extension automatically so Python actually uses a 55 00:05:55.720 --> 00:06:00.470 database to store the Dada and pickles the values so that complex Python 56 00:06:00.470 --> 00:06:06.010 structures can be stored as database fields so just open it up again you 57 00:06:06.010 --> 00:06:10.140 can see their ShelveTest.db test so that's the database that has been created 58 00:06:10.140 --> 00:06:16.050 automatically by python now which database is used depends on the underlying Python 59 00:06:16.050 --> 00:06:20.330 implementation so you may or may not get the DB file extension automatically 60 00:06:20.330 --> 00:06:25.130 added actually what may happen is that you shelve might be stored might be stored 61 00:06:25.130 --> 00:06:29.900 in several files now once the shelve is open we can use it pretty much like a 62 00:06:29.900 --> 00:06:30.790 dictionary 63 00:06:30.790 --> 00:06:36.390 so we can assign values to keys to add to the shelve and we can access individual values using the 64 00:06:36.390 --> 00:06:41.180 keys as we do in the 2 print statements that you see on line 10 and 65 00:06:41.180 --> 00:06:44.850 11 and one difference to keep in mind between the shelve and a dictionary 66 00:06:44.850 --> 00:06:50.270 is that there's no shelve literal so what I mean by that can probably best shown as an 67 00:06:50.270 --> 00:06:55.850 example and what I'm going to do is just copy and paste example code 68 00:06:55.850 --> 00:07:00.980 to show you what I mean, and will undo it afterwords so I'll paste it and overwrite it so you can see 69 00:07:00.980 --> 00:07:04.370 the code their now bit of a difference from what we are using before we are now using 70 00:07:04.370 --> 00:07:10.340 key-value pairs their so if we run this it does seem to work but if you look 71 00:07:10.340 --> 00:07:17.650 carefully at the output here you can see that the fruits is a 72 00:07:17.650 --> 00:07:18.560 dictionary 73 00:07:18.560 --> 00:07:27.190 and just further proof that we can remove the indent from lines 10 and 11 over here and run 74 00:07:27.190 --> 00:07:31.850 that and still works so even though the lines are outside the width block 75 00:07:31.850 --> 00:07:36.150 in other words the shelve itself has now been closed 76 00:07:36.150 --> 00:07:41.030 obviously because we are out of that loop in that indentation level so what I'm getting 77 00:07:41.030 --> 00:07:45.620 is you can't initialize a shelve using literal as we could with a 78 00:07:45.620 --> 00:07:46.670 dictionary 79 00:07:46.670 --> 00:07:52.900 so a subtle difference to keep in mind so will just undo that now and go back to our original code 80 00:07:52.900 --> 00:07:56.639 come down here now and type print fruit.... 81 00:07:56.639 --> 00:08:03.580 you can see quite clearly their on line 13 the output over here you can see that 82 00:08:03.580 --> 00:08:08.710 fruit is a shelve rather than a dictionary that we showed previously and if I go back and try and remove 83 00:08:08.710 --> 00:08:14.900 these back to the previous indentation level like so we should get an 84 00:08:14.900 --> 00:08:18.770 error and we do get an error here in this case because we're attempting an invalid 85 00:08:18.770 --> 00:08:28.469 operation on a closed shelve, so undo those, so keep in mind that when using the with statement that 86 00:08:28.469 --> 00:08:33.159 that will automatic close the file when the with block finishes that's what was 87 00:08:33.159 --> 00:08:37.310 happening in that scenario if you want to keep the shelve active and happy to 88 00:08:37.310 --> 00:08:41.279 close it yourself manually then you can use some alternatives to syntax just try your 89 00:08:41.279 --> 00:08:48.110 luck with the file we can do that by changing line 3 so come up to line 3 90 00:08:48.110 --> 00:08:59.699 and comment out that out , here we can put fruit.... 91 00:08:59.699 --> 00:09:04.190 so that being the case we change the indentation level so come down here and just 92 00:09:04.190 --> 00:09:08.810 do a shift tab to move back to the previous indentation level because we 93 00:09:08.810 --> 00:09:13.240 are not using with and we can run that , actually before I do that we could 94 00:09:13.240 --> 00:09:18.870 do is on the bottom add a close statement so on line 13.... 95 00:09:18.870 --> 00:09:25.260 ..like so and leave the print statement, so if we run that 96 00:09:25.260 --> 00:09:29.170 its still working like it did before remember it's your responsibility to close the shelve 97 00:09:29.170 --> 00:09:34.570 yourself manually by doing what i've done there on line 13 create another examples 98 00:09:34.570 --> 00:09:39.329 so I'm just going to open another file because we're gonna come back to this one, actually what I'll do is open up 99 00:09:39.329 --> 00:09:49.560 projects being a better way of doing it and right click, Create a new Python module and will call it a motorcycle again and close it down again 100 00:09:49.560 --> 00:10:00.230 actually what I'll do, i'll double click that so its on the screen, close that one down ok we are going to do something similar as we did before so I'm 101 00:10:00.230 --> 00:10:05.070 gonna do import.... 102 00:10:05.070 --> 00:10:19.580 I'm gonna do with, put a separate line in their...and you've seen similar example of this before... 103 00:10:19.580 --> 00:10:22.460 what we are trying to do now is show the different types of information can be 104 00:10:22.460 --> 00:10:28.280 stored in a shelve, so..... 105 00:10:28.280 --> 00:11:05.320 and will try bike...and I put while there that should be with getting confuse there, and next thing is printing a couple of 106 00:11:05.320 --> 00:11:25.150 things out there lets do a print...ok we should be able to run that and 107 00:11:25.150 --> 00:11:29.280 that it works as you would expect the same result as the earlier dictionary example with 108 00:11:29.280 --> 00:11:35.850 very similar syntax and we got a new shelve now so if we open our project window again 109 00:11:35.850 --> 00:11:39.400 we should find that that will appear shortly be refreshed and should be able 110 00:11:39.400 --> 00:11:43.160 to see that on the screen and the reasons is not there I'm making the classic 111 00:11:43.160 --> 00:11:47.710 newbie mistake their I need to right click Run because of course what's happening is 112 00:11:47.710 --> 00:11:51.790 running the shelve program again and it's not really running the motorcycles example so we run 113 00:11:51.790 --> 00:11:55.630 that one now we get a different example we actually get the right information shown 114 00:11:55.630 --> 00:11:57.510 on the screen which is correct 115 00:11:57.510 --> 00:12:04.080 and open that file again now we can see bike.db is showing there on the screen so work just as you would expect 116 00:12:04.080 --> 00:12:07.030 and gives the same result with the output as expected from the earlier 117 00:12:07.030 --> 00:12:11.150 dictionary example and of course the syntax is also very similar now before we 118 00:12:11.150 --> 00:12:14.700 move on there is one aspect of shelve that we need to bear in mind 119 00:12:14.700 --> 00:12:19.480 if we're gonna use it successfully unlike a dictionary examples so where we could run 120 00:12:19.480 --> 00:12:22.220 the program again and everything started from scratch 121 00:12:22.220 --> 00:12:26.310 Shelves are persistent files now so far there has been nothing we've done to 122 00:12:26.310 --> 00:12:31.520 demonstrate some of the implications of this so lets modify the example and include a type so 123 00:12:31.520 --> 00:12:35.240 you can understand about the potential problem 124 00:12:35.240 --> 00:12:42.370 first change the name of the shelve to bike2 and line 7 we are gonna change here so its engine size 125 00:12:42.370 --> 00:12:47.190 without an e engine_size as you can see there so you can see clearly there 126 00:12:47.190 --> 00:12:52.380 we are adding engine and we are trying to print it out with an e there so 127 00:12:52.380 --> 00:12:57.290 there should be an error but will run that you can see we correctly do get an error 128 00:12:57.290 --> 00:13:03.830 but we open up our project again notice that a bike2.db has been created 129 00:13:03.830 --> 00:13:08.230 so the error arose because we attempted to retrieve engine underscore size, engine 130 00:13:08.230 --> 00:13:13.130 spelled with an e as you can see on line 9 but we spelled it incorrectly when assigning the 131 00:13:13.130 --> 00:13:17.160 value on line 7 now that's always the easy fix because we correct the error in 132 00:13:17.160 --> 00:13:23.630 the line 7 and run the program again and will do that now and run again 133 00:13:23.630 --> 00:13:28.880 that fixes things up but there is actually a problem in that these two entries for 134 00:13:28.880 --> 00:13:34.670 the engine size 1 spelled correctly and one misspelt you can verify this by commenting 135 00:13:34.670 --> 00:13:39.590 out all the assignments so we don't create them again i'll just do that and I'll close the 136 00:13:39.590 --> 00:13:48.880 run window briefly as well so we can see a little better so I'll comment these out so we should be able to print engine size as 137 00:13:48.880 --> 00:13:49.940 was correct 138 00:13:49.940 --> 00:13:55.220 correctly done a second time but in the first case without the e and I should be able 139 00:13:55.220 --> 00:13:57.980 to do that and run that and we should be able to get the values back from both of them 140 00:13:57.980 --> 00:14:03.200 notice how we got a value back from both of them in other words both keys exist 141 00:14:03.200 --> 00:14:07.750 in the underlying database the misspelled one from the first run of the program and of 142 00:14:07.750 --> 00:14:10.500 course the correct one when I went back and corrected the word 143 00:14:10.500 --> 00:14:14.820 engine to put an e on the end and this is because shelves persisted to a file so just 144 00:14:14.820 --> 00:14:18.440 just correcting the typo didn't remove the error from the underlying 145 00:14:18.440 --> 00:14:22.360 database pretty important distinction there to remember as you carry through 146 00:14:22.360 --> 00:14:26.550 just like a dictionary we can iterate through the keys in the shelve and print them out 147 00:14:26.550 --> 00:14:33.339 which really confirm that we got a rogue entry in there so lets do that before we finish this video 148 00:14:33.339 --> 00:14:45.380 and move to the next one so we come back here and put something like.... 149 00:14:45.380 --> 00:14:55.170 ..so print....so if we run that, again you can see we got make, colour 150 00:14:55.170 --> 00:15:00.589 engine_size, engin_size, and model, so its added both entries 151 00:15:00.589 --> 00:15:04.310 the good thing is just like a dictionary there we can use Del remove 152 00:15:04.310 --> 00:15:08.250 items from a shelve so we come down here let's close this down I'm going to comment out 153 00:15:08.250 --> 00:15:14.990 the for loop because we've already established that works, I'll just leave 154 00:15:14.990 --> 00:15:21.930 that in there and up what I'll do is just delete so.... 155 00:15:21.930 --> 00:15:25.730 the one with the incorrect spelling, so we run that 156 00:15:25.730 --> 00:15:31.389 value should have been deleted but of course we get an error because on line 17 we try 157 00:15:31.389 --> 00:15:34.769 to print out the value out so we could correct that again by removing 158 00:15:34.769 --> 00:15:41.540 to deletion which should already now finish and comment out line 17 and run it once again and you can see that it 159 00:15:41.540 --> 00:15:45.850 works correctly and our engine size with the incorrect spelling of the word engine has 160 00:15:45.850 --> 00:15:49.810 been removed because of the deletion that actually occurred 161 00:15:49.810 --> 00:15:53.260 so I'm gonna en the video here on the next video we're going to continue our discussion of shelve 162 00:15:53.260 --> 00:15:54.389 see you on the next video