1 1 00:00:00,557 --> 00:00:02,230 Hey and welcome back. 2 2 00:00:02,230 --> 00:00:05,383 In this video we're going to be talking about stores. 3 3 00:00:06,350 --> 00:00:08,081 A store is what we're gonna use 4 4 00:00:08,081 --> 00:00:13,081 to have information about a particular set of webpages. 5 5 00:00:13,470 --> 00:00:16,710 And when a user wants to add an item to our application 6 6 00:00:16,710 --> 00:00:19,970 that is using a webpage in that store, 7 7 00:00:19,970 --> 00:00:21,730 we will know that the query 8 8 00:00:21,730 --> 00:00:25,360 and the tag name that we have to use for that. 9 9 00:00:25,360 --> 00:00:28,470 What that'll mean is that if a user gives us a url 10 10 00:00:28,470 --> 00:00:29,880 that we know about, 11 11 00:00:29,880 --> 00:00:32,794 we will need them to give us any technical information 12 12 00:00:32,794 --> 00:00:35,194 like the query and tag name. 13 13 00:00:35,194 --> 00:00:37,870 So the first thing we're gonna do is go ahead 14 14 00:00:37,870 --> 00:00:40,980 and create a new python file in our models, 15 15 00:00:40,980 --> 00:00:42,923 which is gonna be the Store. 16 16 00:00:44,300 --> 00:00:46,730 And here we're going to create a model. 17 17 00:00:46,730 --> 00:00:49,403 Very much like the item and the alert. 18 18 00:00:51,014 --> 00:00:53,097 (typing) 19 19 00:01:05,300 --> 00:01:08,570 So what I've done is I have created an init method 20 20 00:01:08,570 --> 00:01:10,400 that takes in a name for the store 21 21 00:01:10,400 --> 00:01:12,880 so that we can easily know what it is. 22 22 00:01:12,880 --> 00:01:15,670 The url prefix which is going to be 23 23 00:01:15,670 --> 00:01:20,463 what url does all the items in the store have to start with. 24 24 00:01:21,330 --> 00:01:24,310 For example something like https colon slash slash, 25 25 00:01:24,310 --> 00:01:29,260 www.johnlewis.com might be the url prefix 26 26 00:01:29,260 --> 00:01:32,110 for the items we've been using so far from johnlewis.com. 27 27 00:01:33,510 --> 00:01:37,050 And that means that when the user gives us an item 28 28 00:01:37,050 --> 00:01:39,683 that starts with this url we're going know 29 29 00:01:39,683 --> 00:01:42,010 that it belongs to this store. 30 30 00:01:42,010 --> 00:01:44,053 And the way we're gonna do that 31 31 00:01:44,053 --> 00:01:45,313 is we're gonna perform a search in MongoDB 32 32 00:01:45,313 --> 00:01:49,813 for screens starting with this content here. 33 33 00:01:51,550 --> 00:01:54,470 That is how we're going to use stores in the application. 34 34 00:01:54,470 --> 00:01:56,650 Of course in order to make all of this work, 35 35 00:01:56,650 --> 00:01:59,673 we do have to make a few imports. 36 36 00:02:04,040 --> 00:02:08,621 And these are the imports we need uuid, Dict and Model. 37 37 00:02:08,621 --> 00:02:12,310 And of course we do get our warning now from PyCharm 38 38 00:02:12,310 --> 00:02:15,190 saying that the class store must implement 39 39 00:02:15,190 --> 00:02:16,743 all abstract methods. 40 40 00:02:16,743 --> 00:02:18,640 If you remember our model class says that 41 41 00:02:18,640 --> 00:02:20,860 we need to implement a json method 42 42 00:02:20,860 --> 00:02:22,720 because that's the abstract method. 43 43 00:02:22,720 --> 00:02:24,870 So we have to go and do that. 44 44 00:02:24,870 --> 00:02:26,460 Do note 45 45 00:02:26,460 --> 00:02:28,310 that we get a second warning that 46 46 00:02:28,310 --> 00:02:31,200 they call to the init of superclass is miss. 47 47 00:02:31,200 --> 00:02:32,573 We do have to add that in. 48 48 00:02:34,827 --> 00:02:38,490 Like that. And we get a final warning down here 49 49 00:02:38,490 --> 00:02:41,640 that say PEP 8 no newline at the end of the file. 50 50 00:02:41,640 --> 00:02:44,433 That's just because python wants the new line there 51 51 00:02:44,433 --> 00:02:45,266 at the end. 52 52 00:02:45,266 --> 00:02:46,430 And then when you add an empty line 53 53 00:02:47,355 --> 00:02:48,969 at the bottom of the file, it disappears. 54 54 00:02:48,969 --> 00:02:50,250 The reason for that is for stylistical reasons 55 55 00:02:50,250 --> 00:02:54,920 but also cause when you're sharing code on GitHub and so on, 56 56 00:02:54,920 --> 00:02:57,553 its nice to have that final line at the end. 57 57 00:02:58,950 --> 00:03:01,240 Now we're gonna have the json method, 58 58 00:03:01,240 --> 00:03:04,860 which is going to return a Dict as usual 59 59 00:03:04,860 --> 00:03:08,544 and what its gonna return is 60 60 00:03:08,544 --> 00:03:09,550 the id 61 61 00:03:10,660 --> 00:03:12,633 and all the other attributes. 62 62 00:03:19,260 --> 00:03:20,970 That's our json method. 63 63 00:03:20,970 --> 00:03:22,843 Lets start with get_by_name. 64 64 00:03:29,950 --> 00:03:33,280 Notice that here I am making use of the method 65 65 00:03:33,280 --> 00:03:34,690 that we've written earlier on and 66 66 00:03:34,690 --> 00:03:36,704 that's part of model class Fine_one_by and 67 67 00:03:36,704 --> 00:03:39,690 then I'm passing with name property 68 68 00:03:39,690 --> 00:03:41,270 which is where we're gonna be saving 69 69 00:03:41,270 --> 00:03:43,750 and then store_name we're seeking. 70 70 00:03:43,750 --> 00:03:45,510 And so for example we may call this 71 71 00:03:45,510 --> 00:03:48,750 like so store get_by_name John Lewis. 72 72 00:03:50,234 --> 00:03:54,376 And that would go into the database find a store 73 73 00:03:54,376 --> 00:03:57,550 with a name John Lewis and give it back to us. 74 74 00:03:57,550 --> 00:04:01,980 The second method is going to be get_by_url prefix. 75 75 00:04:01,980 --> 00:04:04,960 This method is also going to return a store object 76 76 00:04:04,960 --> 00:04:06,170 and what this is gonna do is 77 77 00:04:06,170 --> 00:04:08,360 we're going to give it a url prefix 78 78 00:04:08,360 --> 00:04:11,200 that we are looking for then it'll go into the database 79 79 00:04:11,200 --> 00:04:14,133 and find a store that has that. 80 80 00:04:24,090 --> 00:04:25,940 Okay so I've written a bunch of code here 81 81 00:04:25,940 --> 00:04:29,609 and its not necessarily obvious what it means. 82 82 00:04:29,609 --> 00:04:34,298 You can tell that this method is going to find one store 83 83 00:04:34,298 --> 00:04:36,963 by its url prefix property. 84 84 00:04:38,467 --> 00:04:42,570 And we're passing in something that isn't a string, 85 85 00:04:42,570 --> 00:04:45,090 for example up here when we're finding by name 86 86 00:04:45,090 --> 00:04:47,170 we've passed in the name that we wanted. 87 87 00:04:47,170 --> 00:04:49,950 And that went into the database and tried to match 88 88 00:04:49,950 --> 00:04:52,800 this store name with the name properties 89 89 00:04:52,800 --> 00:04:55,317 saved in our database. 90 90 00:04:55,317 --> 00:04:57,150 Here's what we're going to say is 91 91 00:04:57,150 --> 00:05:02,143 find me the url prefix properties that start with 92 92 00:05:04,240 --> 00:05:07,913 this url prefix string but could be longer. 93 93 00:05:09,323 --> 00:05:11,780 This is what this regular expression means 94 94 00:05:11,780 --> 00:05:14,310 and this is a MongoDB query. 95 95 00:05:14,310 --> 00:05:16,997 MongoDB allows us to search just passing in a string 96 96 00:05:16,997 --> 00:05:20,439 or it allows us to perform more complex queries 97 97 00:05:20,439 --> 00:05:24,610 and this dollar regex is one of them. 98 98 00:05:24,610 --> 00:05:28,470 When we're passing dollar regex then MongoDB is going to 99 99 00:05:28,470 --> 00:05:33,300 treat this thing here as something that it should perform 100 100 00:05:33,300 --> 00:05:37,043 regular expression validation on while searching. 101 101 00:05:38,430 --> 00:05:42,623 So this does essentially make sure it starts with 102 102 00:05:42,623 --> 00:05:46,540 and then we're going to put url prefix in there. 103 103 00:05:46,540 --> 00:05:49,520 Make sure it starts with url prefix. 104 104 00:05:49,520 --> 00:05:53,220 And then its going to return all of those stores 105 105 00:05:53,220 --> 00:05:57,390 that have the url prefix property that starts with 106 106 00:05:57,390 --> 00:05:59,563 the parameters as to this method. 107 107 00:06:00,896 --> 00:06:05,896 So if we say cls dot or rather store dot get_by_url prefix 108 108 00:06:05,907 --> 00:06:09,880 and we're passing an https colon slash slash 109 109 00:06:09,880 --> 00:06:12,023 www.johnlewis.com, 110 110 00:06:13,320 --> 00:06:15,854 then its going to say find me stores 111 111 00:06:15,854 --> 00:06:18,550 that have a url prefix property 112 112 00:06:18,550 --> 00:06:21,790 that starts with this string. 113 113 00:06:21,790 --> 00:06:23,870 Its not going to match them exactly. 114 114 00:06:23,870 --> 00:06:28,870 Its just going to find those or one that starts with that. 115 115 00:06:28,980 --> 00:06:30,610 What does that mean for us? 116 116 00:06:30,610 --> 00:06:33,470 It means if the url prefix of the store is 117 117 00:06:33,470 --> 00:06:38,385 https colon slash slash www.johnlewis.com forward slash, 118 118 00:06:38,385 --> 00:06:40,035 then it would match 119 119 00:06:40,035 --> 00:06:43,220 because this url prefix starts with this string. 120 120 00:06:43,220 --> 00:06:45,107 Its just the same, 121 121 00:06:45,107 --> 00:06:46,583 it just has one extra character at the end. 122 122 00:06:47,490 --> 00:06:52,410 This is just a slightly less specific way of searching 123 123 00:06:52,410 --> 00:06:53,890 which gives us a bit more flexibility 124 124 00:06:53,890 --> 00:06:57,000 in case the url prefix is slightly different. 125 125 00:06:57,000 --> 00:07:01,410 For example imagine we've got a store with a url prefix 126 126 00:07:01,410 --> 00:07:04,930 of something like this, and we've put in an item 127 127 00:07:04,930 --> 00:07:09,930 that is www.johnlewis.com then it will match this store 128 128 00:07:10,920 --> 00:07:13,160 even though its not exactly the same. 129 129 00:07:13,160 --> 00:07:15,940 The assumption that I'm making when I'm coding is 130 130 00:07:15,940 --> 00:07:18,290 that that's okay. 131 131 00:07:18,290 --> 00:07:23,290 The store that starts with this will match this. 132 132 00:07:23,547 --> 00:07:25,300 And if that's not the case 133 133 00:07:25,300 --> 00:07:27,352 then we can always change it later on 134 134 00:07:27,352 --> 00:07:28,185 and make it more specific 135 135 00:07:28,185 --> 00:07:32,343 but I think it is going to help making it more resilient. 136 136 00:07:33,224 --> 00:07:35,600 Now we do get an error which says 137 137 00:07:35,600 --> 00:07:40,140 that expected time string by got Dict in stead. 138 138 00:07:40,140 --> 00:07:41,190 Why is that? 139 139 00:07:41,190 --> 00:07:44,770 Well our find_one_by method define our model class expects 140 140 00:07:44,770 --> 00:07:46,980 the value to be a string. 141 141 00:07:46,980 --> 00:07:49,560 But we have passed it in a dictionary. 142 142 00:07:49,560 --> 00:07:52,070 So we're gonna tell it that the value passed in 143 143 00:07:52,070 --> 00:07:54,202 should be a string or a dictionary. 144 144 00:07:54,202 --> 00:07:58,510 And we do that by using the union 145 145 00:07:59,680 --> 00:08:00,520 type 146 146 00:08:00,520 --> 00:08:02,020 from the typing module 147 147 00:08:02,020 --> 00:08:05,243 and then here in value we're gonna say union_str comma Dict. 148 148 00:08:07,520 --> 00:08:10,217 And we're gonna do the same for find_many_by. 149 149 00:08:11,653 --> 00:08:12,966 Okay. 150 150 00:08:12,966 --> 00:08:15,393 And now that warning goes away. 151 151 00:08:16,540 --> 00:08:21,540 Here comes the last method in the store class. 152 152 00:08:21,720 --> 00:08:22,700 Which is 153 153 00:08:23,716 --> 00:08:24,716 find_by_url. 154 154 00:08:25,870 --> 00:08:30,182 And this is slightly different from get_by_url prefix. 155 155 00:08:30,182 --> 00:08:34,390 Get_by_url prefix is going to take in a url prefix 156 156 00:08:34,390 --> 00:08:35,840 that we think will match a store 157 157 00:08:35,840 --> 00:08:38,526 and its gonna search the database for that. 158 158 00:08:38,526 --> 00:08:42,973 Find_by_url is going to take in a item url. 159 159 00:08:45,206 --> 00:08:49,420 So this is going to take an item url. 160 160 00:08:49,420 --> 00:08:50,420 What I'm gonna do is 161 161 00:08:50,420 --> 00:08:54,627 I'm just gonna copy some documentation that we have here. 162 162 00:08:54,627 --> 00:08:57,080 Its going to return a store from the url searches 163 163 00:08:57,080 --> 00:08:58,330 such as this one. 164 164 00:08:58,330 --> 00:09:03,215 So given is http or really s www.johnlewis.com slash item 165 165 00:09:03,215 --> 00:09:08,215 slash whatever is going to return a store object 166 166 00:09:08,420 --> 00:09:12,950 that matches the starting part of that url. 167 167 00:09:12,950 --> 00:09:15,240 So its going to take this bit here 168 168 00:09:15,240 --> 00:09:20,240 and its going to search in a database for that. 169 169 00:09:22,390 --> 00:09:23,780 Alright, so how're we gonna do that? 170 170 00:09:23,780 --> 00:09:27,790 Well we're gonna use the regular expression module. 171 171 00:09:27,790 --> 00:09:29,788 So we do have to import it. 172 172 00:09:29,788 --> 00:09:32,465 (clicking) 173 173 00:09:32,465 --> 00:09:34,548 (typing) 174 174 00:09:35,598 --> 00:09:37,580 And we're gonna pass in a pattern which is gonna be 175 175 00:09:37,580 --> 00:09:42,240 https question mark colon slash slash anything 176 176 00:09:45,559 --> 00:09:46,863 question mark slash. 177 177 00:09:47,830 --> 00:09:48,663 Like that. 178 178 00:09:49,700 --> 00:09:53,583 And of course we're going to put that in there. 179 179 00:09:56,240 --> 00:09:58,010 So what is this doing? 180 180 00:09:58,010 --> 00:10:01,033 Well lets try it out with a couple of websites. 181 181 00:10:03,596 --> 00:10:07,970 You can see that when we put https colon slash slash 182 182 00:10:07,970 --> 00:10:10,750 www.johnlewis.com forward slash 183 183 00:10:10,750 --> 00:10:13,082 this matches this pattern and then 184 184 00:10:13,082 --> 00:10:16,623 if I put slash item slash bla dot html, 185 185 00:10:19,614 --> 00:10:22,840 it takes away in a group this first part of the url. 186 186 00:10:24,299 --> 00:10:26,743 What happens if we don't have anything in here? 187 187 00:10:28,170 --> 00:10:31,690 Well the assumptions is that this is okay. 188 188 00:10:31,690 --> 00:10:33,363 Its probably not gonna be 189 189 00:10:33,363 --> 00:10:34,780 and I don't think we're gonna get any urls 190 190 00:10:37,022 --> 00:10:39,338 starting with this. 191 191 00:10:39,338 --> 00:10:40,171 If you take away the quote would be question mark though 192 192 00:10:41,142 --> 00:10:45,627 then its going to match the first part of the url 193 193 00:10:46,920 --> 00:10:50,341 in case we do have a misconstructed url there. 194 194 00:10:50,341 --> 00:10:52,576 So we're not gonna wanna do that 195 195 00:10:52,576 --> 00:10:54,370 but we do wanna keep the question mark there 196 196 00:10:54,370 --> 00:10:59,130 so that we match a group 197 197 00:10:59,130 --> 00:11:00,740 if it is in between 198 198 00:11:00,740 --> 00:11:03,653 the first and second and the third forward slash. 199 199 00:11:04,620 --> 00:11:06,730 And so this regex here is something that 200 200 00:11:06,730 --> 00:11:09,860 I've coded up and it may not be perfect 201 201 00:11:09,860 --> 00:11:14,280 but for now its going to do and what we're gonna do is 202 202 00:11:14,280 --> 00:11:18,140 put back the johnlewis.com and once again 203 203 00:11:18,140 --> 00:11:20,753 we're gonna give it to the item url and 204 204 00:11:20,753 --> 00:11:25,670 what its gonna do is give us the first part of that url 205 205 00:11:25,670 --> 00:11:27,400 up to the forward slash. 206 206 00:11:27,400 --> 00:11:31,669 Another assumption I'm making is that all item urls are 207 207 00:11:31,669 --> 00:11:32,870 going to have this structure. 208 208 00:11:32,870 --> 00:11:36,030 Where they have https or potentially just http. 209 209 00:11:36,030 --> 00:11:39,700 Its gonna match both because of this question mark there . 210 210 00:11:39,700 --> 00:11:43,260 Colon slash slash and then something and 211 211 00:11:43,260 --> 00:11:44,490 then a forward slash. 212 212 00:11:44,490 --> 00:11:46,300 And the reason why I'm making this assumption is 213 213 00:11:46,300 --> 00:11:48,530 because that just how the internet works. 214 214 00:11:48,530 --> 00:11:52,589 We normally have a top level domain and the domain name 215 215 00:11:52,589 --> 00:11:54,490 and then we have forward slash 216 216 00:11:54,490 --> 00:11:56,500 and then we have whatever we are requesting 217 217 00:11:56,500 --> 00:11:59,613 from that domain, so this seems like a pretty safe bet. 218 218 00:12:01,670 --> 00:12:04,240 Okay. Now if we go back to our project 219 219 00:12:04,240 --> 00:12:06,883 you'll see the python complaint says that 220 220 00:12:06,883 --> 00:12:09,450 this is a redundant character escape. 221 221 00:12:09,450 --> 00:12:13,440 In python you don't have to escape these forward slashes. 222 222 00:12:13,440 --> 00:12:16,680 So you can remove that. In Java you do have to escape them. 223 223 00:12:16,680 --> 00:12:18,577 And so that's my bad. 224 224 00:12:18,577 --> 00:12:20,280 Now that we have this pattern, 225 225 00:12:20,280 --> 00:12:24,730 we're gonna say match equal pattern dot search, url and 226 226 00:12:24,730 --> 00:12:26,510 like we did with the price, 227 227 00:12:26,510 --> 00:12:28,600 we're gonna say that the url prefix we're interested in 228 228 00:12:28,600 --> 00:12:30,840 is match dot group one. 229 229 00:12:30,840 --> 00:12:33,310 And that's gonna match the first group 230 230 00:12:33,310 --> 00:12:37,120 probably the only group in the url that we've been given 231 231 00:12:38,020 --> 00:12:39,210 and then finally we're gonna say 232 232 00:12:39,210 --> 00:12:41,600 return cls get_by_url prefix 233 233 00:12:41,600 --> 00:12:44,400 we're gonna use this method that we defined just earlier on 234 234 00:12:44,400 --> 00:12:46,173 and we're gonna pass that in. 235 235 00:12:47,090 --> 00:12:51,937 So why did we define the regular expression here 236 236 00:12:51,937 --> 00:12:54,487 and hope it starts to make a little bit more sense. 237 237 00:12:55,330 --> 00:13:00,330 The url prefix that we're finding is going to come directly 238 238 00:13:00,970 --> 00:13:05,970 from the item url. Which means that 239 239 00:13:06,110 --> 00:13:07,800 if the item url 240 240 00:13:07,800 --> 00:13:11,686 slightly different then what we might expect, 241 241 00:13:11,686 --> 00:13:16,030 we're still going to have the ability to find it 242 242 00:13:16,030 --> 00:13:17,593 in the database which is going to match with something 243 243 00:13:17,593 --> 00:13:21,653 that starts with this domain here. 244 244 00:13:22,780 --> 00:13:24,890 Which is going to be quite handy. 245 245 00:13:24,890 --> 00:13:28,370 So that's everything what we're gonna do in this video. 246 246 00:13:28,370 --> 00:13:32,190 I hope that's not too confusing. 247 247 00:13:32,190 --> 00:13:35,360 The way we're doing this is by using multiple 248 248 00:13:35,360 --> 00:13:37,900 regular expressions and that is going to give us 249 249 00:13:37,900 --> 00:13:40,687 the flexibility we need in order to find stores in 250 250 00:13:40,687 --> 00:13:42,380 our database 251 251 00:13:42,380 --> 00:13:47,380 when we only have partial information of an item url. 252 252 00:13:47,470 --> 00:13:50,670 What happens next is when we're given the item url 253 253 00:13:50,670 --> 00:13:53,990 we find the store and then we're gonna use that store 254 254 00:13:53,990 --> 00:13:56,470 to retrieve the the url prefix and the tag name 255 255 00:13:56,470 --> 00:13:58,170 and then we're gonna use those two 256 256 00:13:59,268 --> 00:14:02,280 to find the price in the item's url. 257 257 00:14:02,280 --> 00:14:07,280 So that means the item will no have to store the url prefix 258 258 00:14:07,644 --> 00:14:08,680 and a tag name itself 259 259 00:14:08,680 --> 00:14:11,480 and that means that this sort of search 260 260 00:14:11,480 --> 00:14:15,090 will work for all items on johnlewis.com 261 261 00:14:15,090 --> 00:14:17,350 and we won't have to again save this data 262 262 00:14:17,350 --> 00:14:18,733 on each item individually. 263 263 00:14:20,320 --> 00:14:21,390 As a side effect of that 264 264 00:14:21,390 --> 00:14:23,970 or rather direct consequence of that, 265 265 00:14:23,970 --> 00:14:26,930 users will no longer have to tell us the url prefix 266 266 00:14:26,930 --> 00:14:28,250 on the tag name. 267 267 00:14:28,250 --> 00:14:31,320 We can add the store into our database 268 268 00:14:31,320 --> 00:14:35,210 and the users can use our application on johnlewis.com. 269 269 00:14:35,210 --> 00:14:38,480 If the user wants to use it on something else, 270 270 00:14:38,480 --> 00:14:41,330 then they can ask us to add the store 271 271 00:14:41,330 --> 00:14:43,380 and we can go and add the store to our app 272 272 00:14:43,380 --> 00:14:45,500 and then users will be able to use our application 273 273 00:14:45,500 --> 00:14:46,810 in that store as well. 274 274 00:14:46,810 --> 00:14:48,003 Whatever that might be. 275 275 00:14:49,130 --> 00:14:52,060 Hopefully all that makes sense and its not too confusing. 276 276 00:14:52,060 --> 00:14:53,500 Again thanks for joining me. 277 277 00:14:53,500 --> 00:14:54,950 I'll see you on the next one.