1 00:00:00,620 --> 00:00:01,490 Hi, welcome back. 2 00:00:01,520 --> 00:00:06,920 In this video, we are going to zoom into our data, in to our data frame by selecting particular 3 00:00:06,920 --> 00:00:13,140 columns of the frame, particular rows, multiple columns, multiple rows, specific cells in the data 4 00:00:13,220 --> 00:00:13,520 frame. 5 00:00:13,550 --> 00:00:20,360 We are going to learn how to apply conditions so that you can extract data based on different filtering 6 00:00:20,510 --> 00:00:21,460 and so on. 7 00:00:22,160 --> 00:00:24,140 So let's start. I'm going to open Jupyter. 8 00:00:28,820 --> 00:00:38,570 And I'm going to go to that review analysis folder and click on the Jupyter notebook file. Now, something 9 00:00:38,570 --> 00:00:48,470 you should know is that if I add a new cell here and I try to access the data variable, we are going 10 00:00:48,470 --> 00:00:49,850 to get this name error. 11 00:00:49,850 --> 00:00:58,280 'Data' is not defined because when you just open a Jupyter notebook, the cells are not automatically 12 00:00:58,640 --> 00:00:59,590 executed. 13 00:00:59,870 --> 00:01:07,900 So this is not a variable yet in the namespace of this interactive python session. 14 00:01:08,300 --> 00:01:15,770 Therefore, you should execute the cells either by going to the first cell and press shift enter, then 15 00:01:15,770 --> 00:01:25,910 shift enter to execute the other cell shift, enter, shift, enter, or you can go to this button to 16 00:01:25,910 --> 00:01:27,290 run all the cells. 17 00:01:27,560 --> 00:01:30,740 So press that, restart and run all cells. 18 00:01:31,790 --> 00:01:37,490 And then you are going to get access to the data variable like that. 19 00:01:38,420 --> 00:01:38,730 Right. 20 00:01:38,780 --> 00:01:45,230 So I just wanted to make sure, you know, you are aware of that specific behavior of Jupyter. 21 00:01:45,800 --> 00:01:51,650 Now I want to put some headings here, some text, for example. 22 00:01:51,650 --> 00:01:57,130 I want to put the title of this first section that we did in the previous video. 23 00:01:57,380 --> 00:01:59,120 I want to name that something. 24 00:01:59,600 --> 00:02:02,510 So I want to add a cell above this cell. 25 00:02:02,870 --> 00:02:09,170 Therefore, what I do is first I make sure that I'm not in insert mode, so I press escape for that. 26 00:02:09,440 --> 00:02:14,930 Then I press 'a' on the keyboard and the new cell is going to be added. 27 00:02:15,660 --> 00:02:18,710 Then I press without entering the cell. 28 00:02:19,220 --> 00:02:25,460 So without answering, make sure you press escape, without entering the cell 29 00:02:25,460 --> 00:02:35,540 we press 'm' and that will convert the cell into a markdown cell, which means the cell now does not 30 00:02:35,540 --> 00:02:42,890 expect Python called it expects a markdown text, which is basically some sort of language. 31 00:02:42,890 --> 00:02:44,420 For example, if we write 32 00:02:46,200 --> 00:02:55,430 two of these hash symbols and we are write 'Overview of the dataframe', and then we press command 33 00:02:55,430 --> 00:03:04,190 enter, that is going to be converted into a title, into a heading, basically an 'H2' heading. 34 00:03:05,330 --> 00:03:09,770 If you add one more, you're going to get the smaller font and so on. 35 00:03:10,130 --> 00:03:11,270 So two is fine. 36 00:03:12,350 --> 00:03:22,340 And then I'm going to add here, so convert this into markdown with 'escape m' and then write here: 37 00:03:25,570 --> 00:03:30,460 'Selecting data from the dataframe'. 38 00:03:34,590 --> 00:03:41,650 And then b2 enter the new code cell, so that is a code cell that is a marked down cell. 39 00:03:42,210 --> 00:03:47,100 So let me show you how you can select a column of the data frame. Escape m. 40 00:03:49,360 --> 00:03:54,040 Select a column, control enter. 41 00:03:55,790 --> 00:04:01,100 B, enter. To select a column, you can use that syntax, 42 00:04:02,250 --> 00:04:09,840 and then execute and this is the column. Now you could ask, why do we need to select a column? 43 00:04:09,840 --> 00:04:11,280 What's the interest behind that? 44 00:04:11,730 --> 00:04:17,770 Well, the answer is that this is the first step of doing further analysis. 45 00:04:17,940 --> 00:04:25,860 For example, if you want to extract the mean, the average of the rating column, then we first need 46 00:04:25,860 --> 00:04:28,250 to start the column and then apply 47 00:04:28,260 --> 00:04:33,820 dot mean, the method mean, execute, 48 00:04:34,110 --> 00:04:42,320 and that will give you the average rating of all the courses for the entire data frame. 49 00:04:42,330 --> 00:04:46,610 So the rating of all the 45 000 rows. 50 00:04:47,190 --> 00:04:50,980 So that is one example of why we would need to select a column. 51 00:04:51,000 --> 00:04:53,240 But let me delete that mean now. 52 00:04:53,460 --> 00:05:09,390 So that is how to select a column. Now 'escape b m', enter. Select multiple columns, execute, b enter. 53 00:05:09,390 --> 00:05:10,860 To select multiple columns, 54 00:05:10,860 --> 00:05:15,000 you'd use more or less the same syntax. 55 00:05:15,660 --> 00:05:23,840 But instead of inserting one single column here, as we did so string rating in this case, we would 56 00:05:23,910 --> 00:05:27,690 insert a list of multiple columns. 57 00:05:28,440 --> 00:05:31,140 Let's say we want 'Course name'. 58 00:05:32,760 --> 00:05:39,870 So one string, a comma after the string, another string, 'Rating' and then execute. 59 00:05:40,440 --> 00:05:46,710 And that will give you basically a data frame containing only those two columns. 60 00:05:47,170 --> 00:05:52,260 Now, there's a fundamental difference between this output and that. 61 00:05:52,650 --> 00:05:57,060 You can also see that this is formatted differently with lines. 62 00:05:57,070 --> 00:06:01,380 So it looks like a table because this is actually a data frame. 63 00:06:01,650 --> 00:06:11,820 So if you apply type of that expression, so that is expression we used, if you execute that, you'll 64 00:06:11,820 --> 00:06:14,760 see that this is a data frame, data frame. 65 00:06:14,970 --> 00:06:17,400 But if we see the type of that 66 00:06:19,240 --> 00:06:26,830 other expression you're going to see that this is a series, so serious is another data type of pandas, 67 00:06:26,830 --> 00:06:35,380 just like we have data frame, we have series and series are used to represent single columns like 68 00:06:35,380 --> 00:06:36,170 this one here. 69 00:06:36,190 --> 00:06:44,630 So the rating column only, but when we have more than one column, Pandas uses a data frame like this. 70 00:06:46,030 --> 00:06:49,330 So that was just an information to keep in mind. 71 00:06:49,510 --> 00:06:55,510 It doesn't change much in how you do the statistics later, but it just good to know how Pandas works. 72 00:06:55,990 --> 00:06:58,930 Let's see how to select a row. 73 00:07:05,380 --> 00:07:05,680 With rows 74 00:07:06,370 --> 00:07:14,260 it's a bit different, of course, you first need to refer to the 'data' variable and then to iloc, 75 00:07:14,980 --> 00:07:20,410 and then you need square brackets. Inside those square brackets 76 00:07:20,410 --> 00:07:24,500 you should put the index of the row you want to access. 77 00:07:24,910 --> 00:07:27,580 For example, let's say we want this 78 00:07:29,530 --> 00:07:37,380 row here, or let's refer to this better so 'The Python Mega Course' with a rating of five, which has an 79 00:07:37,380 --> 00:07:38,590 index of three. 80 00:07:39,480 --> 00:07:42,180 So let's see if we are going to get that through. 81 00:07:44,010 --> 00:07:44,730 Execute. 82 00:07:46,140 --> 00:07:53,670 So, yeah, it seems that is the row. 'The Python Mega Course', and it had a rating of five, the time stamp 83 00:07:53,670 --> 00:07:54,540 was this one here. 84 00:07:54,540 --> 00:07:56,220 So 3:33. 85 00:07:56,500 --> 00:07:58,560 Let's see the entire data frame. 86 00:07:59,900 --> 00:08:06,230 So 3:33, that one in here, that row is that. 87 00:08:08,790 --> 00:08:18,540 The type of this object is a series again, so Pandas uses series also for rows, for single rows. 88 00:08:19,380 --> 00:08:21,180 Let me delete that type. 89 00:08:21,600 --> 00:08:22,460 Execute again. 90 00:08:22,470 --> 00:08:24,870 This is the output, next. 91 00:08:28,490 --> 00:08:31,250 Selecting multiple rows. 92 00:08:33,450 --> 00:08:35,760 B enter, data. 93 00:08:36,390 --> 00:08:44,430 Again, iloc is what we use, and this time since we are working with multiple rows, we want to 94 00:08:44,580 --> 00:08:49,560 input a slice, let's say we want from 95 00:08:50,680 --> 00:09:00,010 index 1 to to index 3, those rows, and we get this data frame this time and not a serious so we 96 00:09:00,010 --> 00:09:07,480 got the row with index 1 and the row with index 2 because the upper index is not included in the slice. 97 00:09:07,480 --> 00:09:10,470 As it almost always happens with Python. 98 00:09:10,900 --> 00:09:13,300 So timestamp 5:12. 99 00:09:13,300 --> 00:09:15,100 Timestamp 5:11. 100 00:09:17,240 --> 00:09:26,510 Five, 12, five, 11, index one, index two, yeah, so that is how you get multiple rows. 101 00:09:32,000 --> 00:09:41,420 Next, selecting a section. What I mean by that is basically a cross section, so particular columns and particular 102 00:09:41,420 --> 00:09:45,050 rows will give us basically a slice of the data frame. 103 00:09:45,380 --> 00:09:50,760 For example, we want to select particular columns. 104 00:09:51,590 --> 00:09:56,810 Again, we use the same syntax as we did previously. 105 00:09:56,820 --> 00:10:01,610 So you see up here when we selected multiple columns. 106 00:10:01,880 --> 00:10:03,770 So, 'Course Name', 'Rating'. 107 00:10:04,040 --> 00:10:07,850 Inside a list, 'Course Name', 108 00:10:10,380 --> 00:10:11,270 'Rating'. 109 00:10:12,570 --> 00:10:20,490 So this will give us basically that and then out of that data frame, what we do is we apply 110 00:10:20,520 --> 00:10:25,520 iloc because this is a data frame object, right? 111 00:10:26,100 --> 00:10:28,260 That was also a data frame object. 112 00:10:28,260 --> 00:10:33,780 And since we could apply iloc to that data frame, we can also apply 113 00:10:33,780 --> 00:10:37,860 iloc to that data frame, it's the same object type. 114 00:10:38,460 --> 00:10:47,900 So out of that we get, for example, from row with index one up to row with index three. 115 00:10:47,970 --> 00:10:55,920 So with all three and we get this section of the data frame, which is again a data frame object type. 116 00:10:55,920 --> 00:11:01,760 So only 'Course Name' and 'Rating' columns and only those two rows. 117 00:11:02,250 --> 00:11:05,970 And lastly, selecting a cell. 118 00:11:13,260 --> 00:11:19,950 We follow the same syntax, the same logic, so let's say we want to select 119 00:11:22,830 --> 00:11:35,760 this cell here, so the cell that is the cross-section of row with index 2 and the time stamp column, 120 00:11:36,360 --> 00:11:37,890 that one in there, time stamp 121 00:11:38,880 --> 00:11:51,030 index 2, right. What we do is first we extract the column using the same syntax that we used to select 122 00:11:51,030 --> 00:11:51,500 a column. 123 00:11:52,500 --> 00:11:55,710 So which is this one in here, data. 124 00:11:57,430 --> 00:12:04,690 Rating, data 'Timestamp' this time, so basically that gives us a column, but out of that we can 125 00:12:04,690 --> 00:12:05,250 apply 126 00:12:05,250 --> 00:12:07,110 iloc for a series too. 127 00:12:07,120 --> 00:12:10,180 So this is a series, not a data frame, but series 128 00:12:10,180 --> 00:12:12,490 Also could use this 129 00:12:12,490 --> 00:12:17,080 iloc property and then get the index 2 out of that. 130 00:12:17,320 --> 00:12:21,850 And that gives us the value, which is a type 131 00:12:24,260 --> 00:12:34,600 string, so if you wanted a rating instead, you would get a float because ratings are float, so 4.0. 132 00:12:35,420 --> 00:12:40,310 That one in there and that is how you get a cell. 133 00:12:40,790 --> 00:12:47,140 Now, this method is very consistent because it uses the same syntax as the other methods. 134 00:12:47,240 --> 00:12:50,530 So it's consistent with the other methods of selecting data. 135 00:12:50,690 --> 00:12:57,860 But there is a faster way to do the same thing, using data the at property. 136 00:12:58,970 --> 00:13:07,180 And first you pass the index of the row and then just the name of the column rating, for example, 137 00:13:08,030 --> 00:13:10,140 and that should give you the same value. 138 00:13:10,460 --> 00:13:14,110 So it is recommended to use that when you are selecting a cell. 139 00:13:14,540 --> 00:13:15,920 So that's about this lecture. 140 00:13:15,920 --> 00:13:22,580 In the next lecture, we are going to go further into selecting data, but using conditions. 141 00:13:22,580 --> 00:13:30,830 For example, give me all the ratings that are greater than four, so filtering based on conditions in 142 00:13:30,830 --> 00:13:31,470 the next video. 143 00:13:31,490 --> 00:13:31,790 See you.