1
00:00:05,120 --> 00:00:09,264
In this video, we'll have a quick look 
at how to use Python's hashlib module. 

2
00:00:09,440 --> 00:00:14,197
We're not going to talk in-depth about security – 
that would be a complete course in its own right. 

3
00:00:14,400 --> 00:00:17,655
So this is a brief introduction 
to Python's hashlib module. 

4
00:00:17,840 --> 00:00:23,310
Let's start with the hashlib documentation. 
I'll go to that page in my browser. 

5
00:00:24,320 --> 00:00:30,240
The first thing you may notice, is that acronyms 
abound. There are 9 in the first paragraph. 

6
00:00:30,240 --> 00:00:32,960
You'll also learn a couple of new 
terms that describe the same thing. 

7
00:00:33,680 --> 00:00:38,560
As the last two sentences of that paragraph says, 
"Older algorithms were called message digest. 

8
00:00:39,360 --> 00:00:43,920
The modern term is secure hash.".
The documentation still refers to digest quite 

9
00:00:43,920 --> 00:00:50,268
frequently though. Just remember that a digest, or 
message digest, is just another name for a hash. 

10
00:00:50,480 --> 00:00:55,628
We saw that term in the Hash Functions video, when 
we looked at the Wikipedia article about hashes. 

11
00:00:55,840 --> 00:00:57,840
As you read through this documentation, 

12
00:00:57,840 --> 00:01:02,580
you'll appreciate why we're not going to attempt 
to cover secure hashes in a couple of videos. 

13
00:01:02,800 --> 00:01:06,189
It's a complex topic, and 
would need its own course. 

14
00:01:06,400 --> 00:01:11,491
Security is a very serious business, and you'll 
find that this documentation is quite intense. 

15
00:01:11,760 --> 00:01:15,280
Continuing with that first paragraph, 
it mentions several algorithms that are 

16
00:01:15,280 --> 00:01:20,748
supported by the hashlib module. We're 
going to use sha256, in our example. 

17
00:01:20,960 --> 00:01:22,800
If you want to use a different algorithm, 

18
00:01:22,800 --> 00:01:26,500
hashlib provides a couple of ways to 
be sure that your choice is available.  

19
00:01:26,720 --> 00:01:31,486
Scrolling down to just below those 3 code blocks, 
there are two set constants that we can check. 

20
00:01:31,963 --> 00:01:35,520
hashlib.argorithms_guaranteed will give 
the names of all the hash algorithms that 

21
00:01:35,520 --> 00:01:41,575
are guaranteed to be supported by this module.
Well they're, almost, guaranteed to be supported. 

22
00:01:41,760 --> 00:01:45,842
The note does mention that there are some 
Python versions that have md5 removed. 

23
00:01:46,000 --> 00:01:50,840
The md5 algorithm was commonly used, 
but is no longer considered very secure. 

24
00:01:51,040 --> 00:01:55,920
The American government's Federal Information 
Processing Standards prohibits using md5, and it's 

25
00:01:55,920 --> 00:02:00,926
been removed from some Python distributions.
The other set is algorithms_available, 

26
00:02:01,040 --> 00:02:05,477
which shows the algorithm names available in the 
Python interpreter that's running your code.  

27
00:02:05,680 --> 00:02:10,320
Let's have a look at these 2 sets.
Back to IntelliJ, and I've created a new 

28
00:02:10,320 --> 00:02:14,721
Python file called secure_hash.py.
I'll add some code: 

29
00:02:39,440 --> 00:02:43,922
After importing the hashlib module, we 
print out the contents of those 2 sets. 

30
00:02:44,080 --> 00:02:48,660
I've sorted them first, to make it easier 
to compare the two sets of algorithms. 

31
00:02:48,800 --> 00:02:52,137
Let's see what we get; I'll run the program: 

32
00:02:52,960 --> 00:02:58,160
sha256 appears in both sets, but notice that 
there can be more names available – in the 

33
00:02:58,160 --> 00:03:02,642
second set – than are guaranteed to be 
available on all Python implementations. 

34
00:03:02,800 --> 00:03:07,886
With Python 3.7 on Windows, both sets 
contained the same algorithm names. 

35
00:03:08,080 --> 00:03:11,360
But you can see, with Python 3.9 that I'm running, 

36
00:03:11,360 --> 00:03:16,242
that there are more algorithms available than are 
guaranteed to exist in all Python implementations. 

37
00:03:16,480 --> 00:03:20,624
That's something to bear in mind, if you 
have to choose a secure hash algorithm.  

38
00:03:20,800 --> 00:03:24,080
Stick to algorithms that are guaranteed 
to be available, unless you have a 

39
00:03:24,080 --> 00:03:29,200
good reason for using one of the others.
We're going to use sha256, which is guaranteed 

40
00:03:29,200 --> 00:03:34,686
to be available. But the basic approach is 
the same, whichever algorithm you choose. 

41
00:03:34,880 --> 00:03:38,109
We'll start off with a message 
to create a secure hash for. 

42
00:03:38,240 --> 00:03:42,571
This could come from a file on disk, or it 
could be an email that your user has typed. 

43
00:03:42,720 --> 00:03:48,800
To keep things simple, we'll create a string.
In the next section, we'll create a sha256 hash 

44
00:03:48,800 --> 00:03:53,633
for a file that we've downloaded. We haven't 
learnt how to read a file from disk – yet.  

45
00:03:53,840 --> 00:03:56,400
That's something we'll be doing in 
the next section, and we'll apply 

46
00:03:56,400 --> 00:04:01,761
what we're learning here, in that section.
For now, I'll type some code into a string: 

47
00:04:30,000 --> 00:04:34,240
Our python_program string is 
some python code. As I said, 

48
00:04:34,240 --> 00:04:39,200
we'd probably read that from a file on disk.
What we're going to do, is generate a secure hash 

49
00:04:39,200 --> 00:04:44,320
of this code, then detect if the code is changed.
I'll discuss why we might want to do that, 

50
00:04:44,320 --> 00:04:48,811
once we've seen it all working.
The first step is to generate the hash. 

51
00:04:48,960 --> 00:04:52,240
That documentation we've just looked 
at was very in-depth, but generating 

52
00:04:52,240 --> 00:04:55,840
the hash is really quite easy.
I'll add the code to do that: 

53
00:05:11,760 --> 00:05:17,646
We're using a SHA256 hash, so we 
use the sha256 method, on line 11. 

54
00:05:17,840 --> 00:05:23,513
If you wanted to use one of the other algorithms, 
you'd use its name, instead of sha256. 

55
00:05:23,760 --> 00:05:27,388
One slight complication is that these 
hashing functions need a byte array. 

56
00:05:27,600 --> 00:05:33,610
They don't work with strings. That's why 
we've got python_program.encode() on line 11. 

57
00:05:33,760 --> 00:05:38,720
It looks far more complicated than it really is.
If you remember, when we wrote our really bad 

58
00:05:38,720 --> 00:05:44,215
hashing function, I mentioned that characters are 
stored as numbers, using something called unicode. 

59
00:05:44,400 --> 00:05:47,760
Each character is represented 
by 1 or more bytes. Let's have 

60
00:05:47,760 --> 00:05:53,121
a look at what that encoded string contains.
I'll add a loop, to print out each character: 

61
00:06:13,040 --> 00:06:17,920
On line 12, we print the value of each item 
in the bytes array. I've also printed the 

62
00:06:17,920 --> 00:06:21,877
character that the number represents, to 
make it easier to understand the output. 

63
00:06:22,080 --> 00:06:25,274
Run the program, and we'll 
have a look at the result: 

64
00:06:26,640 --> 00:06:30,080
Reading down the output, we can 
see that each value, in the array, 

65
00:06:30,080 --> 00:06:32,446
represents a character from our string. 

66
00:06:32,560 --> 00:06:36,686
The value 32 represents a 
space, and 10 is a line break. 

67
00:06:36,880 --> 00:06:41,095
We've seen that hashing functions use the 
numerical values of the characters in a string. 

68
00:06:41,280 --> 00:06:44,480
So it makes sense to convert the 
entire string to an array of numbers, 

69
00:06:44,480 --> 00:06:50,090
when calling these secure hashing functions.
Ok, let's see the hash that's been created. 

70
00:06:50,240 --> 00:06:54,136
I'll comment out those 2 lines that 
print the encoded values first: 

71
00:07:11,200 --> 00:07:15,360
The hexdigest method produces a hexadecimal 
representation of the secure hash, 

72
00:07:15,360 --> 00:07:17,682
as we can see when I run the program: 

73
00:07:19,200 --> 00:07:22,640
It looks horrible, especially if 
you don't understand hexadecimal, 

74
00:07:22,640 --> 00:07:29,557
but it's just a 256 bit number – or 32 bytes, if 
you prefer to think about bytes rather than bits. 

75
00:07:29,760 --> 00:07:32,800
Remember, when we looked at the 
definition of hash functions, 

76
00:07:32,800 --> 00:07:39,040
that they produce a fixed size value from 
variable length data. sha256 produces 256 

77
00:07:39,040 --> 00:07:43,920
bit numbers – that's the fixed size.
So what can we do with this number? 

78
00:07:43,920 --> 00:07:47,708
Well, one thing we can do is use it 
to see if the data has been modified. 

79
00:07:47,840 --> 00:07:52,517
I'll append some more Python code to our 
data, and generate the modified hash: 

80
00:08:43,840 --> 00:08:47,788
On line 17, we append some 
more Python code to the data. 

81
00:08:47,920 --> 00:08:53,333
We print that out, on line 18, just to 
check that python_program has changed. 

82
00:08:53,520 --> 00:08:58,766
Next, we generate a new SHA256 hash.
When I run the program: 

83
00:09:00,400 --> 00:09:05,210
the new hash is very different to the original 
one. That's the last line of the output. 

84
00:09:05,360 --> 00:09:09,859
We could get the program to check the 
hashes for us. I'll add the code to do that: 

85
00:09:40,240 --> 00:09:44,560
Run the program:
and the program reports that "The 

86
00:09:44,560 --> 00:09:49,200
code has been modified", which is correct.
By storing the original hash, 

87
00:09:49,200 --> 00:09:52,855
we've got a very easy way to tell 
if the Python program was modified. 

88
00:09:53,040 --> 00:09:56,640
The hash of a file is often 
shown on websites. That way, 

89
00:09:56,640 --> 00:10:01,166
you can generate the hash after downloading 
the file, and check if it's been modified. 

90
00:10:01,360 --> 00:10:06,240
A lot of people don't bother checking the hashes 
– maybe because they don't know how. But checking 

91
00:10:06,240 --> 00:10:10,162
them can be a good way to make sure that you're 
not downloading files that have been meddled with. 

92
00:10:10,320 --> 00:10:15,040
Let's have a look at the file for the colorama 
package. We installed it from our IDE, 

93
00:10:15,040 --> 00:10:18,375
but it's available to download 
from the Python Package Index 

94
00:10:18,560 --> 00:10:21,166
I'll go to that page, in my browser: 

95
00:10:23,920 --> 00:10:28,970
There are 2 files available to download. 
The wheel is the file that pip can install. 

96
00:10:29,120 --> 00:10:34,720
What's interesting, at the moment, is 
the last column; the Hashes. Click View, 

97
00:10:34,720 --> 00:10:38,482
and you can see various hashes that have 
been generated from the original file. 

98
00:10:38,640 --> 00:10:45,920
MD5 used to be used a lot, but it's an old 
algorithm, and is no longer considered secure. 

99
00:10:45,920 --> 00:10:51,040
Once we've learnt how to read a file from disk, 
we'll come back to this, and generate a sha256 

100
00:10:51,040 --> 00:10:55,797
hash for the file we download. We can then 
compare our hash with the one published here. 

101
00:10:56,000 --> 00:11:00,240
If the hashes agree, we can be confident that 
the file wasn't tampered with, after being 

102
00:11:00,240 --> 00:11:05,487
uploaded to the Python Package Index.
So that's one use for secure hashes. 

103
00:11:06,880 --> 00:11:11,760
Another use is in version control systems.
You upload your Python code files to a version 

104
00:11:11,760 --> 00:11:16,160
control system, such as github.
After you've worked on your code, 

105
00:11:16,160 --> 00:11:20,640
you'll then upload the latest versions.
The VCS software calculates the hash of all 

106
00:11:20,640 --> 00:11:25,424
your files, and only uploads ones where the new 
hash is different to the hash of the old version. 

107
00:11:25,600 --> 00:11:27,840
That avoids uploading files unnecessarily, 

108
00:11:27,840 --> 00:11:33,193
and also means that the timestamp of files 
doesn't get changed, if the file wasn't changed. 

109
00:11:34,800 --> 00:11:38,800
Alright, that was a quick look at secure 
hashes. We didn't want you to think that 

110
00:11:38,800 --> 00:11:42,810
hashes are only used by dictionaries, 
which is why we've included this video. 

111
00:11:42,960 --> 00:11:47,120
It's been a bit artificial, because the 
string we hashed was stored in our code – the 

112
00:11:47,120 --> 00:11:50,560
python_program string.
In the next section, 

113
00:11:50,560 --> 00:11:54,560
we'll read the data from a file instead.
So if you're struggling to appreciate why 

114
00:11:54,560 --> 00:12:00,144
all this is useful, don't worry. It will all 
make sense when we use a more realistic example. 

115
00:12:01,680 --> 00:12:04,082
And that's the end of dictionaries and hashes. 

116
00:12:04,240 --> 00:12:09,840
We'll finish this section by looking at 
Python's sets. I'll see you in the next video.