WEBVTT 1 00:00:00.140 --> 00:00:01.290 Hi and welcome back. 2 00:00:01.290 --> 00:00:03.054 In this video we're going to be trying out 3 00:00:03.054 --> 00:00:05.480 what we've written so far by adding 4 00:00:05.480 --> 00:00:08.240 a few lines of code here into app.py. 5 00:00:09.100 --> 00:00:11.225 The first thing we want to do is of course 6 00:00:11.225 --> 00:00:13.560 be able to load the page that we want to scrape. 7 00:00:13.560 --> 00:00:16.500 So we need to download the html contents of it, 8 00:00:16.500 --> 00:00:19.330 and we're gonna do that with the requests library. 9 00:00:19.330 --> 00:00:21.500 If you've not installed the requests library yet, 10 00:00:21.500 --> 00:00:23.190 make sure to do that, and we've looked at how 11 00:00:23.190 --> 00:00:25.760 to do that earlier on in this section. 12 00:00:25.760 --> 00:00:27.950 Then we're going to import our page, 13 00:00:27.950 --> 00:00:32.950 so from pages all_books_page, import our page, 14 00:00:34.830 --> 00:00:38.570 and finally we are going to get the page contents, 15 00:00:38.570 --> 00:00:42.320 so to get a page content, we'll just do requests.get 16 00:00:42.320 --> 00:00:47.320 and then goes the url so that's http://books.toscrape.com. 17 00:00:50.970 --> 00:00:55.070 Now this is going to get us a response from the server 18 00:00:55.070 --> 00:00:57.340 which includes the page content and also includes 19 00:00:57.340 --> 00:01:00.390 a few more things, like a sort of a status code, 20 00:01:00.390 --> 00:01:02.650 whether everything went well or not and so forth. 21 00:01:02.650 --> 00:01:06.950 If we want to get the content, we just do .content. 22 00:01:06.950 --> 00:01:10.010 That's going to get us the content property of that request, 23 00:01:10.010 --> 00:01:13.450 of that response, which includes all the html. 24 00:01:14.950 --> 00:01:17.470 Then we're going to build our page object. 25 00:01:22.280 --> 00:01:26.900 And finally, we're going to go and get all the books 26 00:01:26.900 --> 00:01:31.500 out of that page, so the books are going to be page.books. 27 00:01:31.500 --> 00:01:35.750 When we do this, remember this is a property of our page, 28 00:01:35.750 --> 00:01:38.160 so if we go into our page, 29 00:01:38.160 --> 00:01:41.000 this is a property decorated method, 30 00:01:41.000 --> 00:01:42.690 when we do that this is going to run, 31 00:01:42.690 --> 00:01:46.090 and it's going to return us all the book parser objects 32 00:01:46.090 --> 00:01:47.920 for each of the elements that were found 33 00:01:47.920 --> 00:01:50.570 using this locator, which is the list item. 34 00:01:50.570 --> 00:01:53.900 So that's going to give us book parser objects. 35 00:01:53.900 --> 00:01:56.820 Then we can go back and say, for book in books, 36 00:01:56.820 --> 00:01:59.080 print book, for example. 37 00:02:00.370 --> 00:02:01.750 What this'll do is it'll call 38 00:02:01.750 --> 00:02:04.110 the rapr method of the book. 39 00:02:04.110 --> 00:02:06.360 So if we go over to our book, 40 00:02:06.360 --> 00:02:09.140 we can see that there is no rapr method here, 41 00:02:09.140 --> 00:02:12.180 so it'll just call the default python rapr method. 42 00:02:12.180 --> 00:02:13.570 If we want to make it a bit better, 43 00:02:13.570 --> 00:02:17.500 we can add our own rapr in here, 44 00:02:17.500 --> 00:02:22.500 and we can say for example return book with self.name, 45 00:02:26.170 --> 00:02:30.400 and costing self.price, 46 00:02:31.350 --> 00:02:36.350 and then self.rating stars, like that. 47 00:02:40.700 --> 00:02:42.750 Okay, so this is what we're going to print out now, 48 00:02:42.750 --> 00:02:45.160 and when we go back to our app.py, 49 00:02:45.160 --> 00:02:47.870 we can right click app.py and run it, 50 00:02:49.510 --> 00:02:50.820 and there we have it. 51 00:02:50.820 --> 00:02:54.780 We get all our books in that page. 52 00:02:54.780 --> 00:02:57.370 A Light in the Attic, costing $51.77, 53 00:02:57.370 --> 00:03:00.560 and has three stars, and so forth, for every book. 54 00:03:00.560 --> 00:03:03.280 Notice how we get the number three because 55 00:03:03.280 --> 00:03:06.700 of the conversion we did earlier on in the book parser, 56 00:03:06.700 --> 00:03:10.720 down in the rating, we are not getting the class anymore, 57 00:03:10.720 --> 00:03:14.540 we are now getting this thing using the book parser ratings. 58 00:03:15.900 --> 00:03:20.150 So this is how you can test out that everything's working. 59 00:03:20.150 --> 00:03:21.430 If you wanted to make it better, 60 00:03:21.430 --> 00:03:23.420 which I'd advise making it a bit better, 61 00:03:23.420 --> 00:03:26.445 to make it a bit more useful, you could create a menu 62 00:03:26.445 --> 00:03:31.010 for this programme that would allow you to, 63 00:03:31.010 --> 00:03:34.690 for example, allow the user to only get books 64 00:03:34.690 --> 00:03:36.920 that are five stars, or maybe get the books 65 00:03:36.920 --> 00:03:41.300 that are under 10 pounds or 10 dollars, and so forth. 66 00:03:41.300 --> 00:03:42.310 So there's a lot of improvements 67 00:03:42.310 --> 00:03:44.110 we could make here with filtering. 68 00:03:44.110 --> 00:03:47.320 Let's go and add some of those improvements, 69 00:03:47.320 --> 00:03:49.330 in the next video we're going to work on a menu 70 00:03:49.330 --> 00:03:51.400 for our users just to be able to interact 71 00:03:51.400 --> 00:03:53.630 with this data a bit more easily. 72 00:03:53.630 --> 00:03:55.180 I'll see you in the next video.