WEBVTT 00:05.230 --> 00:10.870 In the last video we saw how to use the show was to walk a writer to work with followers and directories 00:10.870 --> 00:12.610 on the local file system. 00:12.700 --> 00:17.680 In this video we're going to use the hours to walk in generators of our own to give a lot of flexibility 00:17.680 --> 00:22.040 in working with files that will start out by credit and you fall. 00:22.080 --> 00:26.170 So I going to do that you fall. 00:26.250 --> 00:31.090 I don't recall this Wonderfalls search top they want. 00:31.950 --> 00:38.460 To go ahead and into the following generators I start with important ways that we're going to import 00:39.420 --> 00:44.720 if in match and I want to start with a function. 00:44.750 --> 00:46.860 Def for an underscore albums 00:49.350 --> 00:56.290 prentices route chromis by an artist on school night class print disease and a colon. 00:56.570 --> 01:06.210 So we're going to put a full path camos spice directories come a spice Farwell's in I was taught walk 01:07.410 --> 01:18.390 in parentheses and in the colon that mean do for artist in directory's colon I mean put some TIAA is 01:18.390 --> 01:28.410 equal to I always start Poth dot join parentheses path Cobbers spies artists sorry artist because we 01:28.430 --> 01:31.170 defined as artists on 1 7. 01:31.500 --> 01:41.490 And then we're going to do for the spice album I was whole path of spice albums from a spice school 01:41.700 --> 01:47.550 spice in which to walk subtil you are in parentheses 01:50.380 --> 01:59.970 forced for spice album in albums called when they're going to top in Ye Old Spice. 02:00.080 --> 02:03.150 I always stop path got going. 02:04.150 --> 02:13.990 That's going to be album honest call path call a spice album in Apprentice's and common spice album. 02:15.200 --> 02:15.900 OK. 02:16.320 --> 02:19.680 So this general is similar to the earlier ones so easy. 02:19.730 --> 02:24.070 I would start walking stick and see there to walk through the directories for each one. 02:24.090 --> 02:31.200 It joins a path to the directory name that Amox that tree to pick up the album's name at the outer level. 02:31.230 --> 02:34.040 We're not actually interested in the list of files that's were. 02:34.170 --> 02:34.880 This one here. 02:35.810 --> 02:40.580 Long Don and that's why I've used the discourse the night as I mentioned earlier. 02:40.660 --> 02:45.700 That's a possible convention to show that you're not using a valley but have to specify value when doing 02:45.700 --> 02:51.040 things like packing a top all or dealing with values returned by function call. 02:51.130 --> 02:57.760 For this example we return a Tapazole containing the album name and the path to songs just like the 02:57.760 --> 03:01.800 hour's top path that split function was used to split a path up. 03:01.810 --> 03:09.730 We use our stop path to topside joined to recreate the path to the album directory's So let's add a 03:09.730 --> 03:11.550 simple loop to check that it works. 03:11.650 --> 03:14.930 Then we're going to actually stop doing something interesting with it. 03:15.310 --> 03:17.050 So I'm going to talk here. 03:17.250 --> 03:22.630 Abhimaan this contest is equal to fine albums 03:25.810 --> 03:30.960 music in quites this is in parentheses and that's the path to them. 03:30.970 --> 03:38.360 The artist began to search for his Aerosmith so capital a capital I rather see our say Marty Hutch close 03:38.380 --> 03:42.260 parentheses when to talk for spice. 03:42.280 --> 03:48.040 I in album on a school it's cold print I in parentheses. 03:48.460 --> 03:49.460 From this prog program now 03:54.050 --> 03:57.830 up to the top so that's nothing we haven't seen before. 03:57.830 --> 04:03.800 We've got a series of who's contending the path to the directory as well as the album name but where 04:03.800 --> 04:08.630 it gets really interesting is when we stop matching fall times say you may have wondered why we got 04:08.630 --> 04:10.490 an unused import at the start of the fall. 04:10.510 --> 04:12.720 Not just close this one minute them. 04:12.940 --> 04:17.550 If you did you should certainly have wondered why we're passing the name of the band Aerosmith as an 04:17.550 --> 04:20.980 argument to find on the square albums when in fact we're not using it. 04:21.100 --> 04:27.610 So the phone name that's module if in match Well that's used to Metts files whose names follow some 04:27.610 --> 04:28.810 sort of pattern. 04:28.840 --> 04:34.750 So for now the score albums generator might be more useful if it just turned the happens for a single 04:34.750 --> 04:38.370 artist and with a quick change we can get it to do just that. 04:38.370 --> 04:42.400 So I can come up here and do the well actually on line 7. 04:42.550 --> 04:49.980 We've got four artists in directory's we could change that to put four artists in the top if match not 04:50.240 --> 04:54.700 FUTA then directory's will still be in parentheses. 04:54.960 --> 05:02.390 I mean a comma splice the artist on a score name close parentheses and leave the colon there. 05:02.770 --> 05:07.550 So instead of sleeping through all the directories we can actually filter them first. 05:07.600 --> 05:13.870 So in this case we don't get those albums from directories can exactly match the artist on a score that 05:13.870 --> 05:16.000 was passed to this function. 05:16.570 --> 05:22.400 So if we actually run this and check that it works and obviously I can't spell because I have been through 05:22.410 --> 05:25.860 so it's better. 05:26.040 --> 05:28.670 I wasn't really paying attention to the warning. 05:29.450 --> 05:33.430 We'd rather sit down and say this case will just let run to finish. 05:33.460 --> 05:39.040 But at this time it's not just at putting everything we've already got some airport based on Aerosmith. 05:39.430 --> 05:43.770 So the generator returns to top all that you can see the complexity there. 05:44.080 --> 05:49.810 So the generator retains a Tapazole containing the path to the directory and the name of the directory. 05:50.140 --> 05:54.720 So depending on what you want to do with the results you could just return the album name for argument's 05:54.720 --> 05:55.310 sake. 05:55.660 --> 06:00.990 So in this case I want to demonstrate how using generators can be a great way to simplify code. 06:01.030 --> 06:02.560 So what's an example. 06:02.560 --> 06:06.730 I think it would be more useful to have a way to get the list of songs on the albums. 06:07.150 --> 06:13.330 So let's add this the fine outscores songs generator after foreigners go albums and need to make sure 06:13.330 --> 06:15.000 we live to make loans between them I suppose. 06:15.010 --> 06:18.340 It's great to do that. 06:19.390 --> 06:27.500 So we're going to top def spice find on the songs and the premise is going to be albums put into say 06:27.550 --> 06:28.520 color. 06:29.230 --> 06:38.570 I'm going to talk for spice album in albums call in the next line for spice so spice in spice always 06:38.580 --> 06:39.490 top list. 06:39.490 --> 06:45.570 Do you know how to print the C's album then right square bracket and a zero. 06:45.940 --> 06:48.840 This is because you want the power if not the name of the albums. 06:49.310 --> 06:52.070 McCullen it just put a comment there to that effect. 06:59.760 --> 07:02.820 And then after that we're going to yield song 07:06.390 --> 07:12.470 and then we do album of his Kolesnik was fun and score albums music Aerosmith as it was before. 07:12.750 --> 07:17.520 But this time we know the directory and we aren't expecting any subdirectories so there's no need in 07:17.520 --> 07:23.430 any in that case to walk the entire directory structure and we can use the we start listeria instead 07:23.430 --> 07:26.550 of hours to walk and that's why I've done that 160. 07:27.000 --> 07:32.050 So this should be more efficient but it actually may not be when they come back to that point shortly. 07:32.100 --> 07:37.810 But first let's make use of four score songs to get the song list of all Aerosmith albums. 07:38.100 --> 07:40.010 So we're going to do that on the next line. 07:40.470 --> 07:50.290 So going to school just is equal to find songs an album on a school list and going to college to have 07:50.280 --> 07:52.150 this loop. 07:52.210 --> 08:00.950 Can you say that before deciding that all I'm going to look for is in song list colon print. 08:01.050 --> 08:02.380 Important to. 08:02.700 --> 08:09.790 So you run this nail and say here that we've got all the information we actually need. 08:09.800 --> 08:14.150 The interesting thing here is that we've removed the generator of the data from the code that makes 08:14.150 --> 08:17.890 use of the data that we could do that with ordinary functions. 08:17.980 --> 08:24.620 But people are turning lists and they could actually get huge and that will sustain memory into explicitly 08:24.620 --> 08:27.770 CLEET them which is just something else to forget to do. 08:28.070 --> 08:33.830 So when using this approach we can create a list of songs for any artist in just four lines of code 08:34.220 --> 08:36.110 with very little overhead. 08:36.110 --> 08:43.990 So we're basically changing generators together by passing one generator on the argument to the next. 08:44.000 --> 08:45.910 So this cleanliness again. 08:45.950 --> 08:48.680 So basically you're on line 15 here. 08:49.070 --> 08:55.220 When we loop through all these albums the generator returns the details for each item as we request 08:55.220 --> 08:55.750 them. 08:55.760 --> 09:01.430 So at no time does a complete list of the albums exist in memory and the same is true of all the songs 09:01.890 --> 09:02.210 we get. 09:02.210 --> 09:07.950 Each one has its needed and the program doesn't create your list of songs as such. 09:08.390 --> 09:15.110 Well I guess that's not strictly true because the walk function does put tennis in its top all that's 09:15.170 --> 09:16.580 likely to be a problem. 09:16.740 --> 09:20.670 Then stop walk may not be suitable for your particular application. 09:20.700 --> 09:25.760 We're looking at skin Diyar in the light of video and that can be useful if you want to avoid generating 09:25.760 --> 09:33.530 the lists that I was to walk creates creating and destroying a lot of lists will affect performance. 09:33.520 --> 09:38.700 So part of it has to take care of reclaiming the memory for the previous album socialist. 09:38.720 --> 09:44.600 Each time we process a new album with thousands of albums that memory management is going to have an 09:44.600 --> 09:46.480 effect on performance. 09:46.490 --> 09:51.440 All right so using generators like these could be very useful to things like interrogating log files 09:51.440 --> 09:57.910 on a server so there could be thousands of logs and Apache logs for example can be really huge. 09:58.160 --> 10:04.210 But using a generator to return the full path name of each farro then another one to check for follow 10:04.210 --> 10:04.570 would check. 10:04.580 --> 10:11.000 I fall for a particular matching string is a very very efficient way to look for particular errors or 10:11.000 --> 10:15.800 requests from a certain IP address or whatever you want it to find from the logs. 10:16.020 --> 10:20.750 So generators are really one of those features that make parts and an excellent choice for working with 10:20.750 --> 10:22.130 huge data sets. 10:22.130 --> 10:26.430 Once you get the hang of them they're actually quite simple but yet very powerful. 10:26.500 --> 10:30.310 Now there's more you can do with generators since part of the three point five. 10:30.470 --> 10:36.590 It's also possible to set a value into a generator using a slightly different form of yawed but that's 10:36.590 --> 10:38.780 enough to be getting on with for now. 10:38.870 --> 10:45.600 So I'm going to finish this video now with a quick comment about the F-in match margin's method. 10:45.620 --> 10:51.590 Now I can type Walcott's so it compresses all the artists whose names start with black by adding an 10:51.590 --> 10:52.390 asterix. 10:52.400 --> 10:53.460 Let's go ahead and do that. 10:53.480 --> 11:01.630 For example Lawhon 20 where Smith returns that to capital P L I C K and print etc. etc.. 11:01.730 --> 11:03.920 After that if we actually run this 11:06.870 --> 11:09.200 you can see we can see different output that we got last time. 11:09.390 --> 11:14.790 We get all the songs for the Black Crowes black case black hole Arkansas or Black River Project. 11:14.790 --> 11:17.690 Black Sabbath and black beans Tea-Party. 11:17.910 --> 11:21.720 So that's pretty cool and makes it easy to find all that to look forward. 11:21.730 --> 11:23.310 In the example I mentioned just now. 11:23.460 --> 11:24.910 Searching log files. 11:25.170 --> 11:26.870 There is one thing to be aware of though. 11:27.030 --> 11:29.900 The future is sort of case sensitive. 11:30.000 --> 11:36.270 Now that sounds a little bit vague its because the behavior depends on the underlying file system. 11:36.270 --> 11:41.400 Now Linux file system is a case sensitive so it's not surprising that if it matches all saké sensitive 11:41.400 --> 11:45.300 on Linux but Windows NTFS isn't case sensitive. 11:45.300 --> 11:48.320 So if I change black to lower case. 11:48.390 --> 11:50.690 In other words it closes down. 11:50.980 --> 11:53.590 Change it to a lower case pay for black. 11:53.640 --> 11:59.200 The program will still return the same results on Windows as long as I'm using an NTFS drive of course 11:59.200 --> 12:02.830 so that is a hard drive that's been formatted with NTFS. 12:02.890 --> 12:06.810 Now if you connect remotely to a Linux drive in the body it would be different. 12:07.230 --> 12:12.340 So I actually run this in my case because I'm on a Mac. 12:12.530 --> 12:13.930 This results in no app. 12:14.150 --> 12:18.520 But that example examples a change with the lower case pay would still work in Windows. 12:18.530 --> 12:23.270 Now I haven't mentioned the Mac yet and I should point out that I haven't changed the behavior of my 12:23.270 --> 12:24.530 next file system. 12:24.890 --> 12:30.440 So you might expect that the program to work for me is well after all the Mac file system HFS isn't 12:30.440 --> 12:33.230 case sensitive by default. 12:33.260 --> 12:35.620 It is however the case preserving. 12:35.660 --> 12:40.000 So there's a subtle difference between case sensitive and case preserving. 12:40.010 --> 12:44.180 So if you wrote a part in program that will run on different operating systems you need to be really 12:44.180 --> 12:45.450 aware of that. 12:45.470 --> 12:47.260 So if you saw it as. 12:47.270 --> 12:54.050 Plus you can't create followers with the same name using different case but you can refer to a file 12:54.050 --> 12:56.900 using its name and either uppercase or lowercase. 12:57.110 --> 13:02.350 When person reads the file he returns the name in the original case that it was created with. 13:02.390 --> 13:05.670 So that's why black and lowercase isn't matching directories. 13:05.900 --> 13:08.660 Even the HFS isn't case sensitive. 13:08.660 --> 13:14.420 And again if this anti-Chavez Plus a false systems on a Mac now you shouldn't need to do this for the 13:14.420 --> 13:15.190 next challenge. 13:15.200 --> 13:20.960 But if you want to create a full matching black regardless of case then what you could do is use a generator 13:20.960 --> 13:27.970 expression to convert the directory names to uppercase generator expressions will make much more sense 13:27.980 --> 13:30.040 when we've covered this comprehensions. 13:30.040 --> 13:32.570 So really don't worry too much about his next bit of code. 13:32.660 --> 13:38.500 I'm just including it here to show you one way of dealing with this rather than letting you hang Please 13:38.500 --> 13:41.260 don't ask questions about this generate expression at this point. 13:41.370 --> 13:45.110 And that's because it's going to be explained in detail in a later video. 13:45.520 --> 13:47.480 So let's actually have a look at that. 13:47.490 --> 13:50.390 So what I'm going to do first is coming out of the existing. 13:50.560 --> 13:56.390 If you matched up so for what you that what I would do is actually duplicate that loan and coming up 13:56.390 --> 13:59.990 in the first one you want to make a change it is going to be for artists. 14:00.040 --> 14:01.850 If you matched futa. 14:02.050 --> 14:05.510 But here is where it changes and I'm just going to delete that and just top it again. 14:05.860 --> 14:09.010 So I went to see teleprinter CS D. 14:09.080 --> 14:16.550 Oppa then left to rot parentheses for d in. 14:17.950 --> 14:25.930 Single parentheses Raupp prophecies that is Colma spice capsule's score 9 and another wrong parentheses 14:25.930 --> 14:27.050 and then a colon. 14:27.330 --> 14:31.850 I am actually adding another line here after the method declaration. 14:31.960 --> 14:37.830 The function declaration at the top the caps is called 9 is equal to artist. 14:38.090 --> 14:46.680 Don't name drop us and then let the parentheses to be honest on the score top line. 14:47.700 --> 14:48.550 If I run this 14:51.420 --> 14:56.040 you can say where are getting a working again on my map but tonight this still doesn't work on case 14:56.040 --> 15:01.890 sensitive file systems such as Linux uses the case sensitive file systems it's probably best to use 15:01.890 --> 15:03.700 regular expressions. 15:03.750 --> 15:08.910 However a slightly more complicated generator expression will actually do the job but isn't a sufficient 15:08.970 --> 15:10.610 as using future. 15:10.720 --> 15:14.370 So let's have a look at that so I'm going to close this down and I'm going to come into that second 15:14.370 --> 15:15.400 line again. 15:15.990 --> 15:22.050 Let's talk an example which is going to be for us in in parentheses. 15:22.050 --> 15:26.560 T for t in directories. 15:27.430 --> 15:39.030 If it didn't match dot if it match in parentheses data Anapa comma caps underscore 9 and to what parentheses 15:39.030 --> 15:40.120 and then a colon. 15:40.460 --> 15:43.230 So that's more complex but that should actually work. 15:43.230 --> 15:44.920 So just checked and it still works on a Mac. 15:45.240 --> 15:49.290 And you find this will also work on Linux and obviously it's going to work on Windows anyway because 15:49.290 --> 15:55.120 it's a case insensitive so they generate expressions in case you're wondering will make a lot more sense 15:55.130 --> 15:56.850 a bit later in the course. 15:56.870 --> 16:01.390 Right so let's end the video here in the next video I'm going to start you off with a challenge. 16:01.460 --> 16:06.290 Then look at another way for extracting the information we want without relying on having such a neatly 16:06.290 --> 16:08.060 organized directory structure. 16:08.150 --> 16:12.100 After all who has a music collection is organized I feel they had drives. 16:12.110 --> 16:13.300 See you in the next video.