1 00:00:05,400 --> 00:00:09,260 so moving on although the pickle module which we just talked about 2 00:00:09,260 --> 00:00:14,560 great for serializing python objects it really does have a drawback that the 3 00:00:14,560 --> 00:00:18,780 objects all have to be loaded back into memory into the computer's memory that's 4 00:00:18,780 --> 00:00:22,850 fine for many situations but if you're dealing with a really large set of 5 00:00:22,850 --> 00:00:26,610 objects for example a single object very large single object like a large 6 00:00:26,610 --> 00:00:30,580 dictionary for example then loading that in entire thing to memory may not be a 7 00:00:30,580 --> 00:00:35,540 realistic option so you need to look at alternatives now Python's got your 8 00:00:35,540 --> 00:00:40,220 back to their and they have an alternative it and that's to use the shelve module so 9 00:00:40,220 --> 00:00:44,110 the shelve provides a shelve and you can think of it like a dictionary 10 00:00:44,110 --> 00:00:49,520 but its actually stored in a file rather than in memory now like 11 00:00:49,520 --> 00:00:53,540 a dictionary the shelve holds key value pairs and the values can be anything they can be 12 00:00:53,540 --> 00:00:57,590 pickled which as you found out was just about everything now the keys themselves 13 00:00:57,590 --> 00:01:01,390 must be strings unlike a dictionary where the keys can be immutable objects such as 14 00:01:01,390 --> 00:01:05,640 tuples so important distinction there now all the methods we use with dictionaries 15 00:01:05,640 --> 00:01:10,310 can also be used for shelve objects so it can be really useful to think of them as 16 00:01:10,310 --> 00:01:14,190 a persistent dictionary in fact it's a very easy to convert code using a 17 00:01:14,190 --> 00:01:18,750 dictionary to use a shelve instead because the values are pickled when saved the 18 00:01:18,750 --> 00:01:23,180 same warning applies about using files from untrusted sources so make 19 00:01:23,180 --> 00:01:26,990 sure that you keep that in mind because loading a shelf can also execute 20 00:01:26,990 --> 00:01:31,680 code that's not a reason not to use them because they are very useful but 21 00:01:31,680 --> 00:01:36,830 just don't you shelve files themselves from untrusted sources so let's have a 22 00:01:36,830 --> 00:01:40,720 look at a shelve in action and just to get an idea of how they can be use just like 23 00:01:40,720 --> 00:01:44,010 dictionaries were gonna start with same examples as used in discussing 24 00:01:44,010 --> 00:01:55,600 dictionaries so I'm gonna start by typing import.... 25 00:02:15,730 --> 00:02:53,980 ..and we will just copy and paste some of these.. 26 00:02:53,980 --> 00:03:16,100 ...next one....and next one... 27 00:03:16,100 --> 00:03:18,150 ....and last one of course is lime.... 28 00:03:18,150 --> 00:03:38,300 ...ok and I will also do here, print.... 29 00:03:38,300 --> 00:03:51,070 and print....ok so there is the base program, so you can see we are opening a shelve in the same way as opening a 30 00:03:51,070 --> 00:03:56,970 file we are using the width and shelves keep in mind are read and write by natures so you 31 00:03:56,970 --> 00:04:00,100 need to specify mode but other than that it looks like just like a 32 00:04:00,100 --> 00:04:08,560 real file so we just run and we got an error and this is quite useful to know because of 33 00:04:08,560 --> 00:04:15,070 what's happened here the reason you might think this Attribute shelve module has no open, when I just told 34 00:04:15,070 --> 00:04:19,260 shelve.open is the syntax and the reason is that I've used a file here 35 00:04:19,260 --> 00:04:24,470 python file called shelve.py and what has caused confusion is the face that 36 00:04:24,470 --> 00:04:28,640 python file name is called shelve we're trying to import import shelve so 37 00:04:28,640 --> 00:04:33,740 I'm gonna rename this to fix this refactor it click on shelve.py 38 00:04:33,740 --> 00:04:40,120 right click refactor rename, and will call it shelve. 39 00:04:40,120 --> 00:04:49,660 example.py, refactor so we got the name change there and that automatically 40 00:04:49,660 --> 00:04:52,830 refactor for us which i don't want so I'm gonna change that back to 41 00:04:52,830 --> 00:05:02,650 shelve and so the example should now work because before when we did something like open we 42 00:05:02,650 --> 00:05:05,840 probably wouldn't found that to come up with right away because it be looking 43 00:05:05,840 --> 00:05:09,650 in the current Python program for the open method 44 00:05:09,650 --> 00:05:14,020 open function which doesn't exist so this should now work so keep that in mind if 45 00:05:14,020 --> 00:05:19,620 you get a weird error and using an import make sure your imported your using and the file name 46 00:05:19,620 --> 00:05:24,670 are actually separate things ok lets try running it again okay so this 47 00:05:24,670 --> 00:05:29,020 time we actually go it to work and again just a confirmation, actually I'll closed down 48 00:05:29,020 --> 00:05:35,880 this window because we don't need it for a while so you can see now that the example work and we got 49 00:05:35,880 --> 00:05:39,800 to things printed out there as well which is sort of confirmation that we 50 00:05:39,800 --> 00:05:45,850 successfully save that information so we opened the shelve in a very similar way we to open a file and 51 00:05:45,850 --> 00:05:49,120 as I mentioned shelve are read and write by nature so there's really no need to specify the 52 00:05:49,120 --> 00:05:50,580 mode but 53 00:05:50,580 --> 00:05:55,020 other than it does really look like opening a file now note that the file is or may be 54 00:05:55,020 --> 00:05:59,190 created with a DB extension automatically so Python actually uses a 55 00:05:59,190 --> 00:06:03,940 database to store the Dada and pickles the values so that complex Python 56 00:06:03,940 --> 00:06:09,480 structures can be stored as database fields so just open it up again you 57 00:06:09,480 --> 00:06:13,610 can see their ShelveTest.db test so that's the database that has been created 58 00:06:13,610 --> 00:06:19,520 automatically by python now which database is used depends on the underlying Python 59 00:06:19,520 --> 00:06:23,800 implementation so you may or may not get the DB file extension automatically 60 00:06:23,800 --> 00:06:28,600 added actually what may happen is that you shelve might be stored might be stored 61 00:06:28,600 --> 00:06:33,370 in several files now once the shelve is open we can use it pretty much like a 62 00:06:33,370 --> 00:06:34,260 dictionary 63 00:06:34,260 --> 00:06:39,860 so we can assign values to keys to add to the shelve and we can access individual values using the 64 00:06:39,860 --> 00:06:44,650 keys as we do in the 2 print statements that you see on line 10 and 65 00:06:44,650 --> 00:06:48,320 11 and one difference to keep in mind between the shelve and a dictionary 66 00:06:48,320 --> 00:06:53,740 is that there's no shelve literal so what I mean by that can probably best shown as an 67 00:06:53,740 --> 00:06:59,320 example and what I'm going to do is just copy and paste example code 68 00:06:59,320 --> 00:07:04,450 to show you what I mean, and will undo it afterwords so I'll paste it and overwrite it so you can see 69 00:07:04,450 --> 00:07:07,840 the code their now bit of a difference from what we are using before we are now using 70 00:07:07,840 --> 00:07:13,810 key-value pairs their so if we run this it does seem to work but if you look 71 00:07:13,810 --> 00:07:21,120 carefully at the output here you can see that the fruits is a 72 00:07:21,120 --> 00:07:22,030 dictionary 73 00:07:22,030 --> 00:07:30,660 and just further proof that we can remove the indent from lines 10 and 11 over here and run 74 00:07:30,660 --> 00:07:35,320 that and still works so even though the lines are outside the width block 75 00:07:35,320 --> 00:07:39,620 in other words the shelve itself has now been closed 76 00:07:39,620 --> 00:07:44,500 obviously because we are out of that loop in that indentation level so what I'm getting 77 00:07:44,500 --> 00:07:49,090 is you can't initialize a shelve using literal as we could with a 78 00:07:49,090 --> 00:07:50,140 dictionary 79 00:07:50,140 --> 00:07:56,370 so a subtle difference to keep in mind so will just undo that now and go back to our original code 80 00:07:56,370 --> 00:08:00,100 come down here now and type print fruit.... 81 00:08:00,100 --> 00:08:07,050 you can see quite clearly their on line 13 the output over here you can see that 82 00:08:07,050 --> 00:08:12,180 fruit is a shelve rather than a dictionary that we showed previously and if I go back and try and remove 83 00:08:12,180 --> 00:08:18,370 these back to the previous indentation level like so we should get an 84 00:08:18,370 --> 00:08:22,240 error and we do get an error here in this case because we're attempting an invalid 85 00:08:22,240 --> 00:08:31,930 operation on a closed shelve, so undo those, so keep in mind that when using the with statement that 86 00:08:31,930 --> 00:08:36,620 that will automatic close the file when the with block finishes that's what was 87 00:08:36,620 --> 00:08:40,780 happening in that scenario if you want to keep the shelve active and happy to 88 00:08:40,780 --> 00:08:44,740 close it yourself manually then you can use some alternatives to syntax just try your 89 00:08:44,740 --> 00:08:51,580 luck with the file we can do that by changing line 3 so come up to line 3 90 00:08:51,580 --> 00:09:03,160 and comment out that out , here we can put fruit.... 91 00:09:03,160 --> 00:09:07,660 so that being the case we change the indentation level so come down here and just 92 00:09:07,660 --> 00:09:12,280 do a shift tab to move back to the previous indentation level because we 93 00:09:12,280 --> 00:09:16,710 are not using with and we can run that , actually before I do that we could 94 00:09:16,710 --> 00:09:22,340 do is on the bottom add a close statement so on line 13.... 95 00:09:22,340 --> 00:09:28,730 ..like so and leave the print statement, so if we run that 96 00:09:28,730 --> 00:09:32,640 its still working like it did before remember it's your responsibility to close the shelve 97 00:09:32,640 --> 00:09:38,040 yourself manually by doing what i've done there on line 13 create another examples 98 00:09:38,040 --> 00:09:42,790 so I'm just going to open another file because we're gonna come back to this one, actually what I'll do is open up 99 00:09:42,790 --> 00:09:53,030 projects being a better way of doing it and right click, Create a new Python module and will call it a motorcycle again and close it down again 100 00:09:53,030 --> 00:10:03,700 actually what I'll do, i'll double click that so its on the screen, close that one down ok we are going to do something similar as we did before so I'm 101 00:10:03,700 --> 00:10:08,540 gonna do import.... 102 00:10:08,540 --> 00:10:23,050 I'm gonna do with, put a separate line in their...and you've seen similar example of this before... 103 00:10:23,050 --> 00:10:25,930 what we are trying to do now is show the different types of information can be 104 00:10:25,930 --> 00:10:31,750 stored in a shelve, so..... 105 00:10:31,750 --> 00:11:08,790 and will try bike...and I put while there that should be with getting confuse there, and next thing is printing a couple of 106 00:11:08,790 --> 00:11:28,620 things out there lets do a print...ok we should be able to run that and 107 00:11:28,620 --> 00:11:32,750 that it works as you would expect the same result as the earlier dictionary example with 108 00:11:32,750 --> 00:11:39,320 very similar syntax and we got a new shelve now so if we open our project window again 109 00:11:39,320 --> 00:11:42,870 we should find that that will appear shortly be refreshed and should be able 110 00:11:42,870 --> 00:11:46,630 to see that on the screen and the reasons is not there I'm making the classic 111 00:11:46,630 --> 00:11:51,180 newbie mistake their I need to right click Run because of course what's happening is 112 00:11:51,180 --> 00:11:55,260 running the shelve program again and it's not really running the motorcycles example so we run 113 00:11:55,260 --> 00:11:59,100 that one now we get a different example we actually get the right information shown 114 00:11:59,100 --> 00:12:00,980 on the screen which is correct 115 00:12:00,980 --> 00:12:07,550 and open that file again now we can see bike.db is showing there on the screen so work just as you would expect 116 00:12:07,550 --> 00:12:10,500 and gives the same result with the output as expected from the earlier 117 00:12:10,500 --> 00:12:14,620 dictionary example and of course the syntax is also very similar now before we 118 00:12:14,620 --> 00:12:18,170 move on there is one aspect of shelve that we need to bear in mind 119 00:12:18,170 --> 00:12:22,950 if we're gonna use it successfully unlike a dictionary examples so where we could run 120 00:12:22,950 --> 00:12:25,690 the program again and everything started from scratch 121 00:12:25,690 --> 00:12:29,780 Shelves are persistent files now so far there has been nothing we've done to 122 00:12:29,780 --> 00:12:34,990 demonstrate some of the implications of this so lets modify the example and include a type so 123 00:12:34,990 --> 00:12:38,710 you can understand about the potential problem 124 00:12:38,710 --> 00:12:45,840 first change the name of the shelve to bike2 and line 7 we are gonna change here so its engine size 125 00:12:45,840 --> 00:12:50,660 without an e engine_size as you can see there so you can see clearly there 126 00:12:50,660 --> 00:12:55,850 we are adding engine and we are trying to print it out with an e there so 127 00:12:55,850 --> 00:13:00,760 there should be an error but will run that you can see we correctly do get an error 128 00:13:00,760 --> 00:13:07,300 but we open up our project again notice that a bike2.db has been created 129 00:13:07,300 --> 00:13:11,700 so the error arose because we attempted to retrieve engine underscore size, engine 130 00:13:11,700 --> 00:13:16,600 spelled with an e as you can see on line 9 but we spelled it incorrectly when assigning the 131 00:13:16,600 --> 00:13:20,630 value on line 7 now that's always the easy fix because we correct the error in 132 00:13:20,630 --> 00:13:27,100 the line 7 and run the program again and will do that now and run again 133 00:13:27,100 --> 00:13:32,350 that fixes things up but there is actually a problem in that these two entries for 134 00:13:32,350 --> 00:13:38,140 the engine size 1 spelled correctly and one misspelt you can verify this by commenting 135 00:13:38,140 --> 00:13:43,060 out all the assignments so we don't create them again i'll just do that and I'll close the 136 00:13:43,060 --> 00:13:52,350 run window briefly as well so we can see a little better so I'll comment these out so we should be able to print engine size as 137 00:13:52,350 --> 00:13:53,410 was correct 138 00:13:53,410 --> 00:13:58,690 correctly done a second time but in the first case without the e and I should be able 139 00:13:58,690 --> 00:14:01,450 to do that and run that and we should be able to get the values back from both of them 140 00:14:01,450 --> 00:14:06,670 notice how we got a value back from both of them in other words both keys exist 141 00:14:06,670 --> 00:14:11,220 in the underlying database the misspelled one from the first run of the program and of 142 00:14:11,220 --> 00:14:13,970 course the correct one when I went back and corrected the word 143 00:14:13,970 --> 00:14:18,290 engine to put an e on the end and this is because shelves persisted to a file so just 144 00:14:18,290 --> 00:14:21,910 just correcting the typo didn't remove the error from the underlying 145 00:14:21,910 --> 00:14:25,830 database pretty important distinction there to remember as you carry through 146 00:14:25,830 --> 00:14:30,020 just like a dictionary we can iterate through the keys in the shelve and print them out 147 00:14:30,020 --> 00:14:36,800 which really confirm that we got a rogue entry in there so lets do that before we finish this video 148 00:14:36,800 --> 00:14:48,850 and move to the next one so we come back here and put something like.... 149 00:14:48,850 --> 00:14:58,640 ..so print....so if we run that, again you can see we got make, colour 150 00:14:58,640 --> 00:15:04,050 engine_size, engin_size, and model, so its added both entries 151 00:15:04,050 --> 00:15:07,780 the good thing is just like a dictionary there we can use Del remove 152 00:15:07,780 --> 00:15:11,720 items from a shelve so we come down here let's close this down I'm going to comment out 153 00:15:11,720 --> 00:15:18,460 the for loop because we've already established that works, I'll just leave 154 00:15:18,460 --> 00:15:25,400 that in there and up what I'll do is just delete so.... 155 00:15:25,400 --> 00:15:29,200 the one with the incorrect spelling, so we run that 156 00:15:29,200 --> 00:15:34,850 value should have been deleted but of course we get an error because on line 17 we try 157 00:15:34,850 --> 00:15:38,230 to print out the value out so we could correct that again by removing 158 00:15:38,230 --> 00:15:45,010 to deletion which should already now finish and comment out line 17 and run it once again and you can see that it 159 00:15:45,010 --> 00:15:49,320 works correctly and our engine size with the incorrect spelling of the word engine has 160 00:15:49,320 --> 00:15:53,280 been removed because of the deletion that actually occurred 161 00:15:53,280 --> 00:15:56,730 so I'm gonna en the video here on the next video we're going to continue our discussion of shelve 162 00:15:56,730 --> 00:15:57,850 see you on the next video