WEBVTT 00:05.440 --> 00:09.420 In this video we're going to continue our discussion of generators. 00:09.500 --> 00:14.360 There's also another way to use generators using something called a generator expression. 00:14.360 --> 00:19.520 But we can to come back to that after seeing what list comprehensions are now in the database section 00:19.520 --> 00:20.220 of this course. 00:20.240 --> 00:22.050 We used a music database. 00:22.220 --> 00:30.020 Now that database contained 440 albums part two and three artists with the title of Eivind 5000 songs. 00:30.050 --> 00:33.950 Now Australia is the land of sun and surf and spin all the time. 00:33.950 --> 00:38.780 Topping that data into the database we're going to have any time left to play with the crocodiles on 00:38.780 --> 00:39.920 the beach. 00:39.920 --> 00:45.690 I'm joking but of course I didn't top all that information in and to do it with me. 00:45.710 --> 00:49.700 So let's have a look at how I created the possum project. 00:49.700 --> 00:52.570 Now I've called my music files so you can say on screen. 00:52.820 --> 00:56.730 And the next thing I'm going to do is create a file called follow and play it 01:00.630 --> 01:04.810 like I said the source of that data was the MP 3 follow stored on disk. 01:04.830 --> 01:07.050 That's the music database I'm talking about. 01:07.050 --> 01:11.810 So in this video we're going to look at working with the computer's file system. 01:11.850 --> 01:16.800 Now you may not have any music files on your computer which could make this video difficult to follow 01:16.800 --> 01:19.410 along with with services like Spotify. 01:19.410 --> 01:21.240 That's definitely a possibility. 01:21.290 --> 01:22.860 Now you might not even like music. 01:23.030 --> 01:25.720 You might just listen to Justin Bieber instead. 01:25.800 --> 01:31.290 So we've included does it fall Katanning falls for older songs in the music database that we used in 01:31.290 --> 01:32.400 section today. 01:32.770 --> 01:38.160 You can download that folk music does zip from the resources section for this video. 01:38.170 --> 01:42.250 Now just to be clear these aren't real Empey straightforward and that's it file. 01:42.390 --> 01:44.090 Obviously the rule follows a copyright. 01:44.100 --> 01:49.440 So we are distributing them and the other thing would be that it wouldn't be realistic anyway because 01:50.030 --> 01:56.280 we have a 64 gigabyte of data lights to be accurately identified in my songs which will be only half 01:56.330 --> 01:57.400 of the top. 01:57.400 --> 02:01.880 Now the other thing we've done is we've used the extension a piece rate with each fall. 02:01.890 --> 02:07.110 That's because an A at the start so that if you music player automatically picks up you in P3 follows 02:07.110 --> 02:11.750 from your hard disk extracting the files from the zip point missed that Aprile. 02:11.960 --> 02:17.730 Or what so you want to download that file zip file and extract that file into the music directory from 02:17.730 --> 02:23.790 the zip file that you downloaded into the project directory and it should it up with a subdirectory 02:23.790 --> 02:25.160 similar to what you can say. 02:25.320 --> 02:33.140 This music does zip and I it up and you can say you've got to say and if it was I got the actual artists. 02:33.210 --> 02:39.280 The album name and then all the individual songs should end up in something similar. 02:39.420 --> 02:41.930 And obviously you can say that showing up in the project. 02:42.460 --> 02:46.170 Now there are a lot of followers in there but the whole thing expands to under 3 megabytes So it's quite 02:46.170 --> 02:49.040 a small roads simply download right. 02:49.080 --> 02:54.480 So let's start by having a look at how we can access follows on the computer's hard disk. 02:54.480 --> 02:56.510 Now what touched on this in an earlier section. 02:56.570 --> 03:02.810 The don't walk function can be used to recursively retrieve the contents of a directory as well as all 03:02.810 --> 03:04.510 its subdirectories. 03:04.590 --> 03:10.250 No we used it back then as a way to explain about recursion but I didn't mention that I was to walk 03:10.310 --> 03:11.460 it a generator. 03:11.790 --> 03:16.230 So in other words it doesn't put up a huge list of all the files and directories on the hard drive. 03:16.320 --> 03:18.260 It just returns them as needed. 03:18.450 --> 03:26.810 So to actually see how that works in practice I'm going to stop my lord working Portales that the root 03:26.930 --> 03:30.420 equals music that's going to be the root level folder. 03:30.570 --> 03:36.030 Can you specify a relative path that it's going to use the music some photos that you can see in the 03:36.120 --> 03:37.650 project pane. 03:37.680 --> 03:42.110 If you put these files somewhere else Obviously you'll need to pick the path to whatever wherever you've 03:42.120 --> 03:44.830 actually zipped falls falls. 03:45.390 --> 03:54.630 Next we want a top enforced by his path Kumba spies directories cover spies Farleys spies in. 03:54.720 --> 04:04.220 I always talk that we won't print the CS route Cola and top down is equal to truth in parentheses and 04:04.220 --> 04:05.890 a colon. 04:06.040 --> 04:11.100 And then you want to print Poth so I can see the path and when it goes through it's right through the 04:11.100 --> 04:11.550 files. 04:11.570 --> 04:20.460 It's going to be 4-F in Carlos Colon a print print to see these double quotes slushed tell you the tab 04:20.460 --> 04:21.260 character. 04:21.560 --> 04:30.850 You may want to do the left or right Kelly Price and the type of quartz dot flat and white. 04:32.640 --> 04:36.770 OK so that's how I get that you saw can on line of three. 04:37.080 --> 04:41.220 That's the path that's going to be found in our current directory because I've specified a relative 04:41.220 --> 04:42.300 path. 04:42.360 --> 04:43.230 If we run the program 04:46.460 --> 04:49.900 we can say this quite a bit of code showing the dead on the bottom of the screen. 04:50.230 --> 04:58.220 Is going to stop I can have a look the stacks of content they can scroll about it right up to the top 04:58.220 --> 04:59.930 to see what we're dealing with. 05:01.060 --> 05:05.330 I don't do it just to make things a little bit easier on the run plain to the right. 05:05.380 --> 05:10.990 So I can say it more it's a bit better now under Windows or a Mac machine. 05:10.990 --> 05:15.460 You should say the directories and files in alphabetical order and that's more or less working here 05:15.480 --> 05:21.310 but noting that the numbers because of the way of the track numbers 1 and 10 11 12 and 13 is sorting 05:21.310 --> 05:25.620 into in that order which is a little bit weird but that's how computers work. 05:25.700 --> 05:29.610 And obviously if we had a zero that would penalize them was but this year but it would have sorted correctly. 05:29.830 --> 05:33.560 But in general you say the directories and fall was in alphabetical order. 05:33.910 --> 05:39.520 That's sort of Windows or Mac machine but Linux doesn't present sorted directory listings by default. 05:39.520 --> 05:44.130 So you may well end up having directories appear in a different order if you're running on Linux but 05:44.200 --> 05:51.000 on my back as a rule you can say you've got a thousand maniac's 10 C C C D C and so on then within each 05:51.010 --> 05:53.790 directory we getting the files printed. 05:54.310 --> 05:56.130 I think you'd agree that's pretty powerful. 05:56.230 --> 06:02.570 I always start walk recursively visits every directory from the root and for everyone it returns a topic. 06:02.830 --> 06:08.440 Now the first Otterman to topple a lot Five is a string containing the name of the current directory 06:08.800 --> 06:11.550 which run packing into Poth in our code. 06:11.590 --> 06:14.560 Next is a list of all the directories in the current directory. 06:14.560 --> 06:18.340 And unfortunately there's a list containing the names of all the files. 06:18.340 --> 06:23.050 Now what we can do is see this little bit better but using that trick that we've worked on the section 06:23.050 --> 06:25.650 of pausing for input each time round the loop. 06:25.660 --> 06:28.400 So let's make some changes to the code to do that. 06:28.440 --> 06:35.380 So we're going to do is just tap these two lines the for loop and it's step what we'll do is we'll leave 06:35.380 --> 06:36.750 the print part in there. 06:36.980 --> 06:44.100 Will print parentheses directories print parentheses files. 06:44.950 --> 06:50.200 And we're going to do an import afterthought prentices software to run this program. 06:50.660 --> 06:54.800 Now let's expand this as a bit so I can save it more. 06:55.480 --> 06:58.060 So we get the path printed at the top in music. 06:58.180 --> 07:04.360 That's obviously where we're starting with the stock prices function is starting below that is a list 07:04.360 --> 07:09.580 of all the directories within music and it can say if I scroll over it's quite a few Either there's 07:09.610 --> 07:13.490 a scroll on either and at the end the last list. 07:13.750 --> 07:18.040 Well that's an empty list because there aren't any followers at this level and we can check that by 07:18.040 --> 07:20.380 expanding the music directory of the project. 07:20.950 --> 07:25.490 So we've got all these some directories but there aren't actually any fossils. 07:25.520 --> 07:31.940 We go right down to the end that we're actually in the music directory itself it's only some folders 07:32.830 --> 07:38.310 and by the way on Linux you might need to do a bit of scrolling around because the order of the project 07:38.330 --> 07:38.850 page. 07:38.920 --> 07:42.730 That's the output inteligente sorts the directories alphabetically. 07:43.170 --> 07:47.550 Now in fact we will have to scroll slot on Windows and Mac in a moment too. 07:47.660 --> 07:49.090 And that's because intelligence sorts. 07:49.090 --> 07:51.380 Lower case letters before numbers. 07:51.580 --> 07:55.240 So a thousand maniacs appease after Tensei say the project. 07:55.560 --> 08:00.970 You can see here at the top and you can say that's happening with Tensei say and 10000 Maniacs. 08:01.450 --> 08:03.680 Well it's back to my run pite. 08:03.700 --> 08:10.800 If I press Enter again the loot cards round this tub the stop walk function yields the next couple. 08:10.820 --> 08:13.750 Now it's gone into the thousand maniac's directory. 08:13.810 --> 08:16.240 So that's the value it's printed. 08:16.330 --> 08:21.330 That is only one directory and he and if you can go back and have a look how Tom and I hate this you 08:21.330 --> 08:25.280 can see and he can't there's no falls Rafa at that level. 08:25.350 --> 08:27.150 So you can get him to list. 08:27.300 --> 08:28.940 That's because that's the name of the album. 08:29.010 --> 08:32.960 But he got back again pressing into this tub. 08:32.970 --> 08:39.130 It's actually moved into the time an eight and directory and this table because there's no subdirectories. 08:39.140 --> 08:44.490 The second entry the list is actually empty but we can see all the and and track number one you can 08:44.490 --> 08:53.080 see them right across to one including trekked up and on and you can see that matches with the list 08:53.080 --> 08:53.670 here. 08:53.740 --> 09:00.560 Bearing in mind that some of the early ones because of the Y that have been actually numbered are actually 09:00.560 --> 09:07.130 appearing at the start of this Juden sorting and basically so on and so every time we press ENTER I 09:07.130 --> 09:09.390 would start walk yields the next couple. 09:09.620 --> 09:14.810 And once it's exhausted the contents of one directory it backtracks and continues with the next directory 09:14.810 --> 09:15.730 of the parent. 09:16.040 --> 09:18.410 So we get the contents of Tensei see a period of our presence. 09:18.410 --> 09:21.620 Now and say Tensei is showing up. 09:21.990 --> 09:28.790 Now looking at the contents of the 10 cc directory the path contains the artist Tensei see and the name 09:29.210 --> 09:31.720 of the album the best of the early years. 09:31.770 --> 09:34.980 Now the list contains all the songs on the album. 09:35.180 --> 09:42.320 We can expand that out to the earliest you can say and can there and there's all the songs once we actually 09:42.320 --> 09:45.170 go into that subfolder. 09:45.350 --> 09:50.840 So whenever we get a fall list that isn't empty we can extract all the details we need to populate a 09:50.840 --> 09:54.680 database and to set the code for doing that's pretty simple. 09:55.010 --> 10:00.410 But when dealing with Phillpotts Don't be tempted to just split the string at the slashes and that's 10:00.410 --> 10:06.430 because the always has a passed module that knows about file names and can correctly pass them for us. 10:06.760 --> 10:08.480 So the Remain then this is closed up. 10:08.480 --> 10:16.360 To run Windows I mean to get to change a bit of code so I won't I'm going to do is come up here I'm 10:16.360 --> 10:19.850 going to put if. 10:21.150 --> 10:29.290 And then what I'm going to do is leave the printing they do is so just to get that out. 10:30.280 --> 10:41.580 So print path and I'm going to do first of course split is equal to our stock lbf got split path that 10:41.590 --> 10:49.910 we're going to print and print for spit in parentheses and evidently print parentheses and mean to do 10:50.140 --> 10:53.770 whatever quotes I put a star in there. 10:53.830 --> 10:58.900 Tom's forties are either not right as you want to do is isolate the rest of the car they really don't 10:58.900 --> 10:59.850 need that anymore. 11:00.340 --> 11:08.000 So you run this and scroll back up to the top. 11:08.030 --> 11:13.810 Notice how I always top puffballs split has separated last Directeur the last directory how time in 11:13.810 --> 11:14.600 Aden. 11:14.770 --> 11:16.360 From the path is 12. 11:16.360 --> 11:18.710 So that's separated at the album name for us. 11:18.830 --> 11:23.050 And if we repeat the operation on the first element of the top Well we can actually get access to the 11:23.050 --> 11:23.960 artist knife. 11:24.060 --> 11:31.840 So I close it down again and what we're going to do is after the first split print out we get the second 11:32.110 --> 11:38.680 outscores split is equal to top pop of split parentheses. 11:38.800 --> 11:45.790 First on the first underscore split then in square brackets 0 that in parentheses then it's then print 11:45.790 --> 11:53.630 out the second Alaskan's split and get the list of songs we could do to write the following list. 11:53.680 --> 11:55.410 And we've already checked it's not empty. 11:55.660 --> 11:58.810 As you can see on line 6 pretty far all statement. 11:58.810 --> 12:03.270 So consequently we could just use a normal string splitting to split the name from the song. 12:03.580 --> 12:04.840 So let's do that. 12:04.860 --> 12:11.600 So after the second split printout we're going to put four F in phials colon. 12:11.830 --> 12:18.820 We're going to put some little small details is equal to if not split that imprint the SES single quite 12:18.820 --> 12:25.480 spice dash spice and then we're going to print Sunday tiles. 12:26.260 --> 12:30.580 And actually I guess what we should really be doing with the split is we should be stripping off the 12:30.580 --> 12:32.350 file extension as well. 12:32.350 --> 12:35.690 So let's actually do that so we've got the moment some details here. 12:36.020 --> 12:36.910 We're going to do that split. 12:36.910 --> 12:44.630 But let's start off by putting the hit of that clifftop split parentheses single quite dot in Page 3 12:44.960 --> 12:50.330 remembering that we've referred to these files or call them we have added a full extension of don't 12:50.330 --> 12:56.400 handpiece 3 parentheses and then of the result of that we do another if not split with a single quite 12:56.520 --> 12:58.390 space and a dash. 12:58.390 --> 13:00.230 So let's actually run this and see what happens. 13:02.070 --> 13:07.710 And whoops didn't quite work and going to top then let's just get back and fix that up when I should've 13:07.710 --> 13:13.560 done a put down if there of course is not to stop the code o f Don't do that and stop because I'm referring 13:13.560 --> 13:14.810 to an individual file. 13:14.970 --> 13:20.180 But the second split way is spitting at postone the output that comes back after the first one. 13:20.190 --> 13:28.170 So if you run that now and what that should actually be is we try to actually remove the file extension. 13:28.200 --> 13:33.080 Well that's not going to be a split like force that's going to be a strip to strip out. 13:33.360 --> 13:35.790 So that should be history on paper. 13:35.960 --> 13:42.680 So if not strip print the say single quites don't aim page 3 the results that we want to do a dot split. 13:42.940 --> 13:51.960 And then in parentheses and single quite spice dash spice that Tom lucky list will not well let's talk 13:52.140 --> 13:53.220 about put That's good. 13:53.350 --> 13:55.030 Is back up to the top and have a look. 13:57.480 --> 13:59.910 So I can say it's not quite working correctly. 13:59.910 --> 14:05.550 It almost worked but strip actually works with a sequence not a string so it doesn't actually remove 14:05.570 --> 14:11.400 the stream don't aim pay 3 chicks to beginning it into the phone I play any of the characters in Dot 14:11.430 --> 14:17.050 and page 3 and keep you from leaving them regardless of order until I can't remove any more. 14:17.310 --> 14:22.410 So the third songs and he can say he have a blank song number for example and he can see these other 14:22.410 --> 14:28.620 examples of five characters like and that would obviously be an 8 in the word fyce has been removed 14:28.620 --> 14:29.680 as well. 14:29.680 --> 14:33.830 All right so to do with that we can slice the string before splitting it. 14:34.030 --> 14:34.560 And we went. 14:34.620 --> 14:36.880 Everything except the last four characters. 14:37.080 --> 14:39.370 Well it would be four characters for MP 3. 14:39.390 --> 14:42.260 But extensions don't even pay three. 14:42.300 --> 14:45.000 So we went all about the last five characters. 14:45.000 --> 14:46.810 So let's actually have a go at doing that. 14:47.190 --> 14:50.310 It's down again and fixed this strip. 14:50.310 --> 14:52.830 And what we'll do instead is replace that card. 14:52.860 --> 14:59.230 We're going to put if left to right square brackets within there we're going to put Colin Martin's father 14:59.280 --> 15:06.210 out there but not specifying a start for the splice part that defaults to the start of the string for 15:06.210 --> 15:12.020 the stop and he will go up to but not including five characters in from the end of the string. 15:12.120 --> 15:16.840 We're not going to have of course let's just that. 15:16.920 --> 15:21.920 All right so now remove the extension and start adding the split on the end. 15:22.080 --> 15:28.670 To remove that from the Sunday Times so let's actually have a look at that and see what that does going 15:28.780 --> 15:30.250 back up to the top of the Block. 15:30.680 --> 15:32.310 And I think it's now looking a lot better. 15:32.310 --> 15:37.370 So you can say all right so at this point we've now folding got all the data we need and we can use 15:37.370 --> 15:43.460 it in Seacole insert statements to the data to our database and that's pretty much how the music database 15:43.460 --> 15:45.260 for Section 13 was created. 15:45.320 --> 15:48.100 Well pretty much I'll come back to that later. 15:48.530 --> 15:53.880 So as I said I always thought walk is a generator so it doesn't try to read every single follow the 15:53.880 --> 15:56.410 directory at once into a huge list. 15:56.480 --> 16:00.090 It's only using the details for a single directory as a token. 16:00.320 --> 16:04.880 So ultimately we quit millions of albums and we could process them in this way without running out of 16:04.880 --> 16:07.890 memory so that generates us can get even more useful. 16:08.060 --> 16:12.740 When you use them to create other generators and we start working on that in the next video.