1 00:00:00,079 --> 00:00:08,010 Now let's think about how we can add all these values, so these values into one 2 00:00:08,010 --> 00:00:14,009 data frame, Pandas data frame so I thought about having like one, two, three, four, 3 00:00:14,009 --> 00:00:20,640 five, six, seven plus the lot size eight columns in the data frame and then for 4 00:00:20,640 --> 00:00:25,320 each row you'd have like for price you'd have 725 000 for this 5 00:00:25,320 --> 00:00:32,610 and then the next one for 452 000 and so on until all the 6 00:00:32,610 --> 00:00:39,809 properties are consumed so I hope the structure is clear and how to do that now? 7 00:00:39,809 --> 00:00:46,379 Well one solution would be to maybe iterate through the data frame, but 8 00:00:46,379 --> 00:00:52,110 that's a costly solution it takes a lot of time so it process may become slow 9 00:00:52,110 --> 00:00:57,530 when you iterate through data frames, because data frames are not built to 10 00:00:57,530 --> 00:01:03,390 iterate through them so it's probably better to create a data frame out of 11 00:01:03,390 --> 00:01:09,450 a Python dictionary or out of a list of dictionaries maybe, and actually 12 00:01:09,450 --> 00:01:15,630 that's what I'm going to do. Each iteration what I'll do, is I'll add these 13 00:01:15,630 --> 00:01:20,280 values to a dictionary, so for instance I'll start with the first dictionary and 14 00:01:20,280 --> 00:01:26,369 so in the first iteration I'd have like a price key and the value of the price 15 00:01:26,369 --> 00:01:32,369 as the first pair of the dictionary, and then the next pair of that same dictionary 16 00:01:32,369 --> 00:01:39,479 would be the address key and the address value, and then we go to the next pair so 17 00:01:39,479 --> 00:01:45,450 we add the third pair here, and then fourth pair, and fifth, sixth, and seventh, 18 00:01:45,450 --> 00:01:51,750 and eight so eight pair. A dictionary of eight pair in the first iteration. Then 19 00:01:51,750 --> 00:01:58,079 in the next iteration I need to create yet another dictionary with the same key. 20 00:01:58,079 --> 00:02:03,119 So we'd have again price and the value for that price which would correspond to 21 00:02:03,119 --> 00:02:07,619 the value of the second property and so I'd build the second dictionary, and then 22 00:02:07,619 --> 00:02:12,510 a third dictionary, and so on until all the ten properties here are 23 00:02:12,510 --> 00:02:20,129 consumed so I'd end up with ten dictionaries. Now you also need to store 24 00:02:20,129 --> 00:02:24,209 these dictionaries somewhere because if you just iterate through dictionaries and 25 00:02:24,209 --> 00:02:27,870 you lose them at each iteration, then you haven't done anything. 26 00:02:27,870 --> 00:02:33,930 So what we will do, we will store those dictionaries in a list, so first 27 00:02:33,930 --> 00:02:38,760 thing you may want to do is you want to store each iteration with an empty 28 00:02:38,760 --> 00:02:44,280 dictionary so that you can add the key value pairs to that fresh dictionary. 29 00:02:44,280 --> 00:02:50,340 So once you create an empty dictionary, then you go ahead and replace this print 30 00:02:50,340 --> 00:02:58,500 statements, so you'd want let's say price as the key of the first dictionary and 31 00:02:58,500 --> 00:03:07,139 that would be equal, we don't need the brackets, this one either, so that's it. 32 00:03:07,139 --> 00:03:15,349 Similarly you can do, this was address, remove that, again here, let's call this 33 00:03:24,239 --> 00:03:34,980 locality, so we are creating keys on the fly, so price, address and locality, and we 34 00:03:34,980 --> 00:03:39,810 are assigning values to those keys again on the fly so maybe it would make sense 35 00:03:39,810 --> 00:03:46,139 to have price down here so we start with address, locality, and then the 36 00:03:46,139 --> 00:03:58,500 price, and that would be the beds, and if there are no beds, 37 00:03:58,500 --> 00:04:01,040 than you need to pass none for the beds, and then again that 38 00:04:08,710 --> 00:04:26,020 would be area which is equal to this again. Same thing for area, 39 00:04:26,020 --> 00:04:45,730 none when there is no area and one more here. Full baths. We copy this. None and these 40 00:04:45,730 --> 00:04:59,740 would be half baths equal to none. We have print statement here which we 41 00:04:59,740 --> 00:05:05,890 don't need, so here we are going through each group feature name if lot size in 42 00:05:05,890 --> 00:05:16,919 feature group, text then d, let's call this lot size, is this equal to feature name 43 00:05:16,919 --> 00:05:22,780 text, and let's remove this print statement. And so at the end of this loop 44 00:05:22,780 --> 00:05:27,490 you'll have your first dictionaries, so at the end of the first iteration you'll have your 45 00:05:27,490 --> 00:05:31,540 first dictionary, and now you want to store that dictionary somewhere so 46 00:05:31,540 --> 00:05:37,120 let's store it in a list. So normally you need to create a list outside of the 47 00:05:37,120 --> 00:05:47,440 loop, an empty list there, and then at the end of the loop so which is exactly here, 48 00:05:47,440 --> 00:05:54,790 you want to append the dictionary, so you'll get a list of multiple dictionaries. 49 00:05:54,790 --> 00:06:01,360 In this case you'll get a list of 10 dictionaries, let's execute this. We've 50 00:06:01,360 --> 00:06:10,919 got any valid syntax there. Beds I forgot to have assignment operator there and… 51 00:06:12,520 --> 00:06:26,390 Alright, no error this time, let's print L here and it's looking good, so you can 52 00:06:26,390 --> 00:06:35,450 check the length of this, so it's 10. And now if you want to throw this list 53 00:06:35,450 --> 00:06:46,210 of dictionaries to a data frame, so this list to a data frame, 54 00:06:46,210 --> 00:06:55,900 well, you'd need to use Pandas and then df is equal to Pandas dot data frame, 55 00:06:55,900 --> 00:07:02,420 and here we need to write a really long statement. I'm joking, you just need to 56 00:07:02,420 --> 00:07:07,210 pass a list there and you create a data frame out of that list of dictionaries. 57 00:07:07,210 --> 00:07:16,070 So let's check the data frame and boom, here are the data! And as you can see we 58 00:07:16,070 --> 00:07:21,260 even got Nan values when there is not a lot size, so Pandas 59 00:07:21,260 --> 00:07:27,710 understands when there are no data in the dictionaries, so for instance here we 60 00:07:27,710 --> 00:07:35,120 didn't have a lot size and Pandas will assign NaN, a Numpy NaN value for that 61 00:07:35,120 --> 00:07:41,480 which if you want you can simply replace with a none value. And then you need to 62 00:07:41,480 --> 00:07:47,950 work a lot to save this to CSV file, a really long statement. 63 00:07:47,950 --> 00:07:56,390 Okay, so output dot CSV and you should be good to go. Let me check that. 64 00:07:56,390 --> 00:07:58,750 Here is the file. And here are the data. 65 00:08:07,270 --> 00:08:11,290 So it's looking good. Great! Now what comes next? 66 00:08:11,290 --> 00:08:19,570 Well, as you might guess we need to go and get the properties of all the pages, so 67 00:08:19,570 --> 00:08:27,000 second page and third page, so we'll do that in the next lecture. See you!