1 00:00:00,350 --> 00:00:06,750 Welcome to 11:47 in the last lesson we went over how to send e-mails using the SMTP protocol in this 2 00:00:06,750 --> 00:00:09,530 lesson we'll learn how to check your emails inbox. 3 00:00:09,570 --> 00:00:17,040 So just as SMPTE is the protocol for sending e-mail the Internet message Access Protocol or IMAP specifies 4 00:00:17,040 --> 00:00:22,620 how to communicate with your email provider server to retrieve e-mails that were sent to your email 5 00:00:22,620 --> 00:00:23,790 address. 6 00:00:23,790 --> 00:00:28,980 Now unfortunately I IMAP is a fairly complicated protocol. 7 00:00:28,980 --> 00:00:33,540 The protocols for e-mail are pretty much as old as the Internet itself. 8 00:00:33,570 --> 00:00:37,300 They really were designed to be user friendly. 9 00:00:37,530 --> 00:00:45,080 So Python comes with with a module called iMap lib but we're not going to use that one. 10 00:00:45,210 --> 00:00:48,850 Instead we're going to use two third party modules. 11 00:00:48,930 --> 00:00:51,160 First is called IMAP client. 12 00:00:51,210 --> 00:00:54,110 And the second is called pyzmail. 13 00:00:54,120 --> 00:00:59,280 So you can install those using Pipp by consulting the course notes or Appendix A of the automate the 14 00:00:59,280 --> 00:01:01,410 boring stuff with Python textbook. 15 00:01:01,410 --> 00:01:07,200 So again I'm just going to go through all the code steps of checking your e-mail using these modules 16 00:01:07,620 --> 00:01:12,060 and once I'm done I'll go back and dive into each step in more detail later. 17 00:01:12,060 --> 00:01:15,790 So the first thing we're going to do is import IMAP client. 18 00:01:15,900 --> 00:01:23,520 Next we're going to create a connection object by calling IMAP clients IMAP client and pass it the domain 19 00:01:23,520 --> 00:01:26,520 name of the email provider server. 20 00:01:26,880 --> 00:01:31,260 In my case it's going to be mapped out G-mail dot com and then we're going to have to pass this keyword 21 00:01:31,260 --> 00:01:32,190 argument. 22 00:01:32,220 --> 00:01:36,740 SSL equals true SSL and encryption algorithm. 23 00:01:36,740 --> 00:01:42,260 This basically tells IMAP client that yes we do want to use SSL encryption. 24 00:01:42,360 --> 00:01:49,060 And next we're going to call the log in method log into my email account a swagger that you email outcome 25 00:01:49,360 --> 00:01:57,600 and then pass it the password and we get a bite's object which can be interpreted like a string and 26 00:01:57,600 --> 00:02:00,120 it basically tells us yes you have successfully logged in. 27 00:02:00,120 --> 00:02:02,260 That was correct username and password. 28 00:02:02,280 --> 00:02:05,060 So we've logged in sluffed a folder. 29 00:02:05,120 --> 00:02:13,970 99 percent of the time this is just going to be inbox and we're going to set the read only keyword argument 30 00:02:14,070 --> 00:02:14,880 to true. 31 00:02:15,000 --> 00:02:19,740 So unless you're going to delete e-mails you'll always want to set this as true just so that you don't 32 00:02:19,740 --> 00:02:26,280 accidentally delete any e-mails during this logged in session and that returns a whole bunch of stuff 33 00:02:26,880 --> 00:02:29,200 which we can pretty much mostly ignore. 34 00:02:29,460 --> 00:02:35,670 And now we have to find e-mails and we'll do that by calling the search method. 35 00:02:35,760 --> 00:02:40,650 So the e-mail protocol has its own special syntax for searching. 36 00:02:40,670 --> 00:02:47,430 You basically have to pass it as a list of strings and each string will be formatted in this special 37 00:02:47,520 --> 00:02:49,310 IMAP Search syntax. 38 00:02:49,320 --> 00:02:52,380 So I'm just going to use a really basic example. 39 00:02:52,380 --> 00:02:59,580 I want to grab all the e-mails since let's say August 20th 40 00:03:02,440 --> 00:03:04,100 of 2015. 41 00:03:04,470 --> 00:03:08,310 So remember this is a string that is inside of a list. 42 00:03:08,370 --> 00:03:11,230 And that's what we're passing to the search method. 43 00:03:11,310 --> 00:03:18,180 This will return a list of unique IDs and each unique ID references a particular email going to store 44 00:03:18,180 --> 00:03:21,360 this in a variable called you IDs. 45 00:03:22,110 --> 00:03:22,640 That's great. 46 00:03:22,680 --> 00:03:29,640 So these you IDs are just integers and we can see it's now returned all of the IDs for the emails that 47 00:03:29,640 --> 00:03:33,210 I've received since August 20th of this year. 48 00:03:33,240 --> 00:03:40,530 This integer is far from actually useful email you know with a subject line and to and from email addresses 49 00:03:40,530 --> 00:03:46,110 that we're used to we need to translate this ID into an actual email. 50 00:03:46,260 --> 00:03:51,740 And we do that by calling the fetch method for the connection object. 51 00:03:51,930 --> 00:03:54,930 But of course this is also kind of complicated. 52 00:03:54,930 --> 00:03:59,270 The first argument will be a list with the ID that we want. 53 00:03:59,400 --> 00:04:05,460 So let's say I want to grab this email I'm just going to copy and paste that you ID there and then the 54 00:04:05,460 --> 00:04:09,500 second argument is what parts of the email that were that we want. 55 00:04:09,660 --> 00:04:15,240 So you can look at the documentation for this later but 99 percent of the time you're just going to 56 00:04:15,240 --> 00:04:24,290 pass it this body with square brackets after it and and also flags. 57 00:04:24,330 --> 00:04:28,430 So you're just going to pass this exact list value. 58 00:04:28,920 --> 00:04:29,520 Oh whoops. 59 00:04:29,550 --> 00:04:34,780 I want to store this in a variable called raw message. 60 00:04:35,370 --> 00:04:39,900 So fetch is returned this dictionary like structure that you can see. 61 00:04:39,930 --> 00:04:45,930 It looks really complicated but it has all the information about the email inside of it. 62 00:04:45,930 --> 00:04:52,350 You can sort of see oh here's the from part it's an email I sent to myself Here was the subject line 63 00:04:52,590 --> 00:04:55,160 and then the main part of the e-mail. 64 00:04:55,200 --> 00:05:00,240 But you know we don't really want to parse this string on our own that will be kind of complicated. 65 00:05:00,240 --> 00:05:08,920 So instead let's just use the pyzmail module to do this for us so go ahead and put in port this. 66 00:05:08,940 --> 00:05:13,970 Remember that pyzmail is a third party module so you have to install it with Pipp first. 67 00:05:13,970 --> 00:05:23,850 And what we're going to do now is call Pyzmail Pyzmessage dot factory method. 68 00:05:23,870 --> 00:05:31,010 I'm really sorry about this this is super complicated and we're going to pass it raw message but not 69 00:05:31,010 --> 00:05:32,120 just draw a message. 70 00:05:32,120 --> 00:05:39,140 We actually have to pass it this you ID value which is the key here. 71 00:05:39,350 --> 00:05:45,860 And then even inside of that because that's the key and the value is another dictionary inside there 72 00:05:45,870 --> 00:05:46,360 . 73 00:05:46,580 --> 00:05:50,900 We're going to pass it at this value which is a bite's value. 74 00:05:51,020 --> 00:05:52,790 It's technically not a string. 75 00:05:52,910 --> 00:05:57,380 I mean see that because it begins with a lowercase be here. 76 00:05:57,380 --> 00:06:01,580 So I'm going to have to copy and paste this. 77 00:06:01,940 --> 00:06:11,480 And so this particular part of the raw message that was returned by fetch is passed to Pizey mailed 78 00:06:11,470 --> 00:06:13,670 out Pzymessage factory. 79 00:06:13,910 --> 00:06:14,410 Yeah. 80 00:06:14,420 --> 00:06:19,060 So as I said this is really complicated. 81 00:06:19,070 --> 00:06:21,900 The good news is you don't have to memorize all this. 82 00:06:21,890 --> 00:06:24,680 You can always just look up this information later. 83 00:06:24,990 --> 00:06:25,410 Oh right. 84 00:06:25,400 --> 00:06:28,200 And so this returns a Pyzmessage object. 85 00:06:28,310 --> 00:06:30,410 So we want to save that to a variable. 86 00:06:30,410 --> 00:06:33,230 We'll just say that to a very vocal message. 87 00:06:33,770 --> 00:06:39,770 And finally once we have this message object we have something that we can work with easily so we can 88 00:06:39,770 --> 00:06:44,990 call the Get subject method on this message object. 89 00:06:45,000 --> 00:06:49,280 You can see here is that subject line for the email that I sent to myself so long 90 00:06:52,380 --> 00:06:57,690 you can also get any addresses from it like the from address who this e-mail was from. 91 00:06:57,750 --> 00:07:03,830 And you can see Usually this tuple value will have a string of the e-mail senders name. 92 00:07:03,890 --> 00:07:06,300 If it was set and then their e-mail address. 93 00:07:06,360 --> 00:07:08,650 I don't have one set for this e-mail account. 94 00:07:08,750 --> 00:07:14,580 So it just sent a Swygert at gmail dot com as my full name. 95 00:07:15,290 --> 00:07:20,870 But we could also see who this was sent to which apparently is blank. 96 00:07:21,050 --> 00:07:27,270 That might have been something weird with the some T.P. lib method calls that I made. 97 00:07:27,690 --> 00:07:34,650 But you can see in the blind carbon copy this was s.c.i also to a Swygert. 98 00:07:34,910 --> 00:07:40,050 But if we want the actual body of e-mail we're going to have to go through yet some more complicated 99 00:07:40,040 --> 00:07:40,400 parts. 100 00:07:40,400 --> 00:07:47,110 So you may be familiar with the fact that you can send text e-mails which are just plain text or HTL 101 00:07:47,210 --> 00:07:48,200 e-mails. 102 00:07:48,440 --> 00:07:54,410 If you've ever received an e-mail and it had colorful text or different fonts or images embedded inside 103 00:07:54,410 --> 00:07:56,580 of it that was in each T.M. e-mail. 104 00:07:56,730 --> 00:08:02,750 If the e-mail was just plain text all the same font and all the same color and everything then that 105 00:08:02,750 --> 00:08:04,540 was just a text email. 106 00:08:04,820 --> 00:08:11,810 So we can see if this was a text email or an HTL mail email or if it had both text and HVM parts. 107 00:08:12,200 --> 00:08:15,480 By looking at the text part member variable. 108 00:08:16,250 --> 00:08:21,860 And if tech support is set to none then that means that there was no text part but that's not the case 109 00:08:21,870 --> 00:08:22,070 here. 110 00:08:22,070 --> 00:08:24,850 So we can say oh OK there was a text part here. 111 00:08:25,380 --> 00:08:27,950 Let's see if there was an h t m l part as well. 112 00:08:28,110 --> 00:08:30,010 OK that return None. 113 00:08:30,140 --> 00:08:37,050 Idylls interactive shell doesn't actually show none values but we can confirm that by just having this 114 00:08:37,050 --> 00:08:42,770 small expression HVM all part equal equals none and that evaluates to true. 115 00:08:42,770 --> 00:08:46,970 So this email only has a text part. 116 00:08:46,980 --> 00:08:55,410 So in order to get the main message of this e-mail we're going to have to call get payload method. 117 00:08:55,700 --> 00:08:59,870 And in turn we have to call de-code on that. 118 00:08:59,960 --> 00:09:02,430 And this is the string encoding. 119 00:09:02,490 --> 00:09:04,560 You don't have to know what that is. 120 00:09:04,550 --> 00:09:09,650 Just know that 99 percent of time you can pass UTF 8 to it. 121 00:09:10,740 --> 00:09:16,130 And here you finally get a plain and ordinary string. 122 00:09:16,130 --> 00:09:21,920 Now if this doesn't work you can actually try to figure out what encoding the e-mail was sent as this 123 00:09:21,920 --> 00:09:28,260 will usually be in the message to text part that char set. 124 00:09:28,280 --> 00:09:32,050 Except in this case this is set to the value nine. 125 00:09:32,820 --> 00:09:34,160 So we're just going to have to guess. 126 00:09:34,190 --> 00:09:38,270 99 percent of the time it'll be UTF Dash 8. 127 00:09:38,720 --> 00:09:42,020 So we can also search and fetch other e-mails. 128 00:09:42,170 --> 00:09:47,100 And then once we're done we can just call log out but I'm not done quite yet. 129 00:09:47,370 --> 00:09:54,410 Let's go ahead and dive into some more detail with each of these steps you see up here. 130 00:09:54,570 --> 00:10:02,140 When I connected to this IMAP domain name this will be dependent on what your e-mail provider is. 131 00:10:02,150 --> 00:10:04,290 You can usually just google for this information. 132 00:10:04,430 --> 00:10:10,820 But here's some common e-mail providers like Gmail or Yahoo or Comcast and you can see the domain name 133 00:10:10,830 --> 00:10:16,130 for their map servers are pretty typical usually it's just whatever the company is dot.com but with 134 00:10:16,280 --> 00:10:18,120 IMAP dot in front of it. 135 00:10:19,880 --> 00:10:22,830 And next will be called that search method. 136 00:10:23,060 --> 00:10:27,700 IMF has its own particular syntax for how you search for emails. 137 00:10:27,720 --> 00:10:33,920 In this case I just said I want to get all the emails that I over that have arrived since August 20th 138 00:10:33,920 --> 00:10:39,890 and you have to format this date exactly like this two digits here so if it was August 5th I'd have 139 00:10:39,890 --> 00:10:44,280 to have 0 5 then a three letter month name in English. 140 00:10:44,480 --> 00:10:48,520 And then the year with dashes in between. 141 00:10:48,530 --> 00:10:53,030 Now there's a whole lot of other search keys keywords like this. 142 00:10:53,030 --> 00:10:58,910 You could just send all but I wouldn't recommend that because that would return you ID for all of the 143 00:10:58,910 --> 00:11:01,310 emails you've ever received. 144 00:11:01,840 --> 00:11:04,600 You could also have before or on. 145 00:11:04,730 --> 00:11:10,760 And since fall by a date in that particular format that I just show you you could find all of the e-mails 146 00:11:10,760 --> 00:11:16,850 that have a certain string inside a subject or all the e-mails that have a certain string inside the 147 00:11:16,850 --> 00:11:23,870 body or using the text keyword here a certain string in either the subject or the body. 148 00:11:23,960 --> 00:11:28,220 And if your string has that you're searching for has spaces in it you'll just have double quotes on 149 00:11:28,220 --> 00:11:28,890 either side of it. 150 00:11:28,880 --> 00:11:34,040 So this string the value is what you would pass to the search method and see it's a string that has 151 00:11:34,040 --> 00:11:40,290 text meaning we're going to search for the subject or the body and then the string that we're searching 152 00:11:40,280 --> 00:11:45,530 for inside there has double quotes here. 153 00:11:45,750 --> 00:11:50,390 There's a whole bunch of these and you can see them all on table 16 dash 3 in the automate the boring 154 00:11:50,390 --> 00:11:51,980 stuff with Python textbook. 155 00:11:51,980 --> 00:11:56,710 But you can check them for who they're from or who they were sent to. 156 00:11:57,050 --> 00:12:03,320 If you've already read or haven't read the e-mail yet if you've replied or haven't replied to it now 157 00:12:03,320 --> 00:12:09,110 the I'm at Prog protocol is really old so it's kind of using these old fashioned terms instead of red 158 00:12:09,120 --> 00:12:10,730 it's seen and unseen. 159 00:12:10,730 --> 00:12:14,280 And instead for replied to it's answered or unanswered. 160 00:12:14,450 --> 00:12:20,660 You can see if it's a draft you get all your draft emails any emails are larger or smaller than a certain 161 00:12:20,660 --> 00:12:21,810 number of bytes. 162 00:12:21,920 --> 00:12:29,420 You can check the documentation for I'm at client for more details. 163 00:12:30,530 --> 00:12:36,080 So while we're still logged into the IMAP server I'm going to just go through into some more details 164 00:12:36,080 --> 00:12:36,420 . 165 00:12:36,440 --> 00:12:43,300 Now remember after we had logged in we called this select folder and I selected the inbox. 166 00:12:43,310 --> 00:12:48,230 Now the inbox is the main folder that you're going to be use using all the time but maybe you've set 167 00:12:48,230 --> 00:12:53,900 up other folders or work you want to check your spam folder or your draft folder. 168 00:12:54,050 --> 00:12:59,420 The way that you can find all of the folders is by using the connection objects list folders method 169 00:13:00,530 --> 00:13:08,570 which will return this really complicated looking list but in this list it will have a tuple value and 170 00:13:08,570 --> 00:13:14,820 the name of that folder is going to be the third item inside that tuple. 171 00:13:14,840 --> 00:13:17,300 So you can see this is my drafts folder. 172 00:13:17,300 --> 00:13:23,270 This is a filler or folder that I have now so you want to delete emails. 173 00:13:23,390 --> 00:13:29,540 First of all you would have to select the folder in this case. 174 00:13:29,540 --> 00:13:37,000 I'll just try inbox and then select it but this time I have the read only keyword set to false. 175 00:13:37,000 --> 00:13:43,740 This all allows to modify the Inbox folder which is what we want to do when we delete it so same as 176 00:13:43,730 --> 00:13:44,470 before. 177 00:13:44,570 --> 00:13:53,850 We're going to get the ID's by calling search and let me just find all the emails I received on today's 178 00:13:53,960 --> 00:13:58,880 date August 24th 2015. 179 00:14:02,230 --> 00:14:04,600 So here's the unique IDs for all those emails. 180 00:14:04,760 --> 00:14:12,860 I'm going to delete this one say all I have to do is call the delete messages and then pass it a list 181 00:14:12,870 --> 00:14:13,960 of all the ladies. 182 00:14:14,030 --> 00:14:15,180 I want to delete. 183 00:14:15,170 --> 00:14:19,600 So in this case I'll send a list with this new ID. 184 00:14:19,610 --> 00:14:25,340 To delete this particular email or if I just wanted to delete all of these emails I received on August 185 00:14:25,340 --> 00:14:29,950 24th I get just past that you ID's list itself. 186 00:14:30,940 --> 00:14:35,630 Now I don't actually want to delete those so I'm just going to erase that. 187 00:14:37,010 --> 00:14:40,990 So that wraps it up for us for checking your email using Python. 188 00:14:41,060 --> 00:14:43,360 Now this was really complicated. 189 00:14:43,620 --> 00:14:49,760 And unfortunately these were the best modules I could find to make this as simple as possible but even 190 00:14:49,760 --> 00:14:52,930 still this is pretty complicated there is lots of different steps. 191 00:14:53,000 --> 00:14:58,040 You don't have to memorize all of these steps but you can then look up the documentation later to figure 192 00:14:58,040 --> 00:15:02,620 out if you want to do particular things. 193 00:15:03,280 --> 00:15:08,250 If you want to look at the documentation for the IMAP client module you can read it here at. 194 00:15:08,270 --> 00:15:09,200 I'm out client. 195 00:15:09,200 --> 00:15:12,040 Read the docs dot org and Pye's emails. 196 00:15:12,080 --> 00:15:14,210 Documentation is that Magic System. 197 00:15:14,210 --> 00:15:21,890 Net slash Pizey mail and all of this is also covered in the automate the boring stuff with Python book 198 00:15:22,130 --> 00:15:27,000 that you can read online here at automake the boring stuff that com. 199 00:15:27,080 --> 00:15:28,210 Chapter 16