1
00:00:00,330 --> 00:00:07,110
In the previous lectures, we built a program that goes through the vegetables.txt file every

2
00:00:07,110 --> 00:00:10,030
10 seconds and it prints out its content.

3
00:00:11,160 --> 00:00:14,190
So, the program is running here. Now, vegetables.txt

4
00:00:14,190 --> 00:00:21,240
is just a simple text file with some useless data like garlic and onion.

5
00:00:21,960 --> 00:00:29,790
What happens if we had some more real world data such as these temps_today.csv

6
00:00:29,790 --> 00:00:30,410
file.

7
00:00:30,870 --> 00:00:37,590
So this is a csv file, which is basically a text file and it has a structure of multi-columns.

8
00:00:38,460 --> 00:00:40,710
So every column has some data. You see here

9
00:00:40,740 --> 00:00:41,550
this is a header.

10
00:00:41,730 --> 00:00:47,100
It has two columns "st1" for station one, and "st2" for station two.

11
00:00:47,100 --> 00:00:53,550
So we have two weather observation stations and they recorded these data, these temperatures.

12
00:00:54,060 --> 00:01:02,010
Let's say these are observations in one day, this is in the morning and this is later in the afternoon

13
00:01:02,010 --> 00:01:03,240
and in the night.

14
00:01:04,050 --> 00:01:05,220
So we have two columns.

15
00:01:05,580 --> 00:01:12,360
Our duty now is to read this file every 10 seconds, just like we are doing with the vegetables.txt

16
00:01:12,380 --> 00:01:13,350
file.

17
00:01:13,980 --> 00:01:22,110
But instead of printing out its content, we have to print out the average value of all these values.

18
00:01:22,500 --> 00:01:25,110
So we should do that every 10 seconds.

19
00:01:25,380 --> 00:01:31,200
Well, normally you'd have to do this every 24 hours because it makes more sense since this data are

20
00:01:31,200 --> 00:01:35,860
daily data, daily temperature data, they are being changed every day.

21
00:01:36,240 --> 00:01:40,110
It does make sense to print out the value of the average value every 24 hours.

22
00:01:40,290 --> 00:01:41,700
And of course, you can do that easily.

23
00:01:41,700 --> 00:01:48,140
Just change the value from ten seconds to whatever seconds is equal to 24 hours.

24
00:01:48,150 --> 00:01:49,670
You can easily calculate that.

25
00:01:49,830 --> 00:01:54,120
But for simplicity here, let's just use ten seconds.

26
00:01:54,360 --> 00:02:03,480
Now, while we can do this by using the open method to load the data in python, that would load the data as a string.

27
00:02:03,690 --> 00:02:09,810
So this entire text would be loaded as a string in Python, then we would have to apply some

28
00:02:09,810 --> 00:02:15,270
string processing operations to split all these values and convert them to floats.

29
00:02:15,450 --> 00:02:19,200
But that would be like reinventing the wheel.

30
00:02:20,070 --> 00:02:27,060
What I'm trying to say is that someone else wrote some python code that does this very easily in just

31
00:02:27,060 --> 00:02:28,590
one or two lines of code.

32
00:02:28,860 --> 00:02:34,220
And the guys who wrote this, they have built this as a third party library.

33
00:02:34,380 --> 00:02:42,420
So just like we used time and OS by importing them inside Python, we can also use another library for

34
00:02:42,420 --> 00:02:47,080
this case to load this data in a nice format in Python.

35
00:02:47,170 --> 00:02:48,180
Let me show you how.

36
00:02:49,290 --> 00:02:55,740
The library name I know is pandas and pandas doesn't come by default with Python, so when you install

37
00:02:55,740 --> 00:03:00,990
Python, pandas will not be there, but you can install it with pip.

38
00:03:01,590 --> 00:03:09,630
Pip is another library that comes installed by default in Python and it's used to install other third party

39
00:03:10,050 --> 00:03:10,710
libraries.

40
00:03:11,280 --> 00:03:17,760
Now, a very important point I want to make is that the pip command may vary between different operating

41
00:03:17,760 --> 00:03:20,350
systems and different installations of python.

42
00:03:20,580 --> 00:03:27,570
For example, if you are using Python version  3.8, you might have to type in 'pip3.8'

43
00:03:27,570 --> 00:03:30,010
instead of just pip.

44
00:03:30,030 --> 00:03:34,740
If you are using Python 3.9, you have to type in 'pip3.9'.

45
00:03:34,740 --> 00:03:40,080
If you're using Python 3.10, which is newer than 3.9, you have to type in

46
00:03:40,080 --> 00:03:42,060
'pip3.10'.

47
00:03:42,480 --> 00:03:46,140
So to sum it up, you have to use your own commands.

48
00:03:46,260 --> 00:03:52,230
Whenever I type in 'pip3', you have to use your own 'pip3.8', or  'pip3.9' etc.

49
00:03:52,950 --> 00:03:54,090
in your computer.

50
00:03:55,390 --> 00:04:00,070
So I'm going to use pip3 and then do install pandas.

51
00:04:00,760 --> 00:04:04,200
Note that you shouldn't change the install pandas

52
00:04:04,210 --> 00:04:04,960
part of the command.

53
00:04:05,300 --> 00:04:08,540
You should only change the pip part, as I explained to you.

54
00:04:09,010 --> 00:04:13,070
OK, so pandas is a name of the library we want to install.

55
00:04:14,020 --> 00:04:15,970
So the bars are filling in.

56
00:04:16,570 --> 00:04:21,310
That means the pandas library is being installed, successfully installed.

57
00:04:23,070 --> 00:04:26,550
Great, now let's go ahead and use pandas in our script here.

58
00:04:29,950 --> 00:04:33,060
Import Pandas, just like we did with other libraries.

59
00:04:33,720 --> 00:04:37,280
So these two, this was a standard library.

60
00:04:37,680 --> 00:04:40,620
It's built in inside the Python interpreter.

61
00:04:40,800 --> 00:04:44,790
And this is again, it's a standard library, but it's not built in.

62
00:04:44,790 --> 00:04:45,870
It's written in Python.

63
00:04:46,290 --> 00:04:49,530
This pandas, this is written in Python as well.

64
00:04:49,800 --> 00:04:53,850
And now it was just installed in our python installation files.

65
00:04:54,000 --> 00:04:56,720
Let me show you where that pandas is.

66
00:04:57,120 --> 00:04:58,680
Let's open a python session.

67
00:05:00,310 --> 00:05:06,370
So if you import sys, and then sys.prefix that will give you this directory or whatever

68
00:05:06,370 --> 00:05:13,180
directory you have, so copy it without the quotes, and then use 'start' on Windows and paste the directory.

69
00:05:13,720 --> 00:05:20,920
On Mac and Linux use 'open' and paste the directory, press enter that will open the directory, go to

70
00:05:20,920 --> 00:05:26,160
lib, to Python and then OS, for example, was here, os.py.

71
00:05:26,200 --> 00:05:28,220
That's why these were standard libraries.

72
00:05:28,570 --> 00:05:37,580
Now whatever you install with pip it's going to be inside the site-packages directory, which is down here

73
00:05:37,600 --> 00:05:38,290
somewhere.

74
00:05:38,350 --> 00:05:38,830
Yes.

75
00:05:41,180 --> 00:05:49,180
So pandas, this is pandas, we just installed pandas.
In this case, this is not a module, but it's

76
00:05:49,190 --> 00:05:57,110
it's a bunch of modules and this is called a package.
When you have several modules that is called a

77
00:05:57,110 --> 00:06:00,130
package, but you can still refer to it as a library.

78
00:06:00,530 --> 00:06:05,960
So a library is a name used usually to refer both to modules and libraries.

79
00:06:06,200 --> 00:06:11,990
So when you code, when you use pandas, it doesn't make any difference if it's a module or it's a bunch

80
00:06:11,990 --> 00:06:12,830
of modules.

81
00:06:12,840 --> 00:06:13,760
So a package.

82
00:06:14,180 --> 00:06:16,640
So the code of pandas is all there.

83
00:06:16,760 --> 00:06:19,100
And we can use this code now in our script.

84
00:06:19,490 --> 00:06:24,260
So let me show you in the next lecture how we use this pandas third-party library.