Organizing Keys

[00:00:01]
>> So hopefully now, you've all had a little bit of time to practice thinking lexicographically. Thinking in terms of how strings have strings sort and how you can pull down ranges of keys. So, I'm gonna show you an example for how that might map to a more kind of like traditional web app.

[00:00:27]
So, for example, if we wanna build something, it's not the best example cuz we'll be, okay, at this kind of stuff. But if we wanna make a system where's there's gonna be users and there's gonna be posts, you could imagine we might use a prefix like user to list all of our users.

[00:00:50]
And we can use a prefix key called post to list all of the posts, and right here, we're using exclamation mark as a delimiter. To sort of separate out where the key prefix ends and where the other part begins. So the cool thing about formatting data this way, here, I can just copy this JSON file and we can insert in a batch into our database.

[00:01:20]
And the cool thing about this kind of data, whoops, is that we can pull it apart with greater than and less than keys. So, what I'm gonna do right now is I'm gonna insert this data into the database with the batch. And then I'm gonna show you how you can use less than and greater than to like pull down a list of all of the users, separately from pulling down a list of all of the posts.

[00:01:50]
And like within those queries, you can even like do sub queries, like all of the users whose names start with. Like between M and P or something like that. It's a very small amount of data but we can use our imaginations, I hope. So, can just load the data?

[00:02:09]
I think, right, so it has keys and values that should be fine to feed directly into the batch function. So I think I call it data.json. We can set up our set up our database into a directory called users.db, and we can just db.batch that data. Okay, so if I run this, it should just insert the data into the database and exit if everything went well, whoops.

[00:02:48]
I guess the data has the data.
>> And the line three.
>> Yeah, that should be a comma, I guess. Okay, maybe it'll work now. So if I run that program, it should insert all of the data and exit. There we go. So now we have a file, a directory users.db and we can start to pick it with create read stream.

[00:03:24]
So just go ahead and comment this part out and stick it at the bottom in case you want to replicate my results when I put it into the repo. So we can do db.createReadStream. Maybe let's get a list of all of the posts first. So that's gonna look like is we can use a greater than key for post, exclamation mark, and now we need an upper bound.

[00:03:49]
So that's our lower bound post exclamation mark and remember that keys that are shorter will come before keys that are longer. So now we just need an upper bound. There's a few ways to do this depending on whether you need, whether there's gonna be unicode characters and your keys or not.

[00:04:04]
If there aren't going to be unicode characters in your keys, what you need is just some high ASCII value. So you could do tilde is actually pretty high up on the ASCII table. So you'll see a lot of people who use level DB, they'll use tilde is all over the place.

[00:04:22]
Or you could do like uffff, I think that would also work as well. In case you have like different kinds of characters. I'm no unicode experts so this is probably subtly wrong and in slight ways but, yeah, maybe don't. If you're gonna have unicode data, maybe just convert it to base 64 hex first before you dump it in a key.

[00:04:43]
Let's just use a tilde. That's real nice. All right, like before, we'll pipe this into just a little consumer stream to print it out. And make sure that you're consuming stream that your, the writable stream that you're piping into is in object mode again cuz that's a common source of errors.

[00:05:07]
I know when I write this kind of code so. Also if your stream only seems to print one record, you probably forgot to call next, which I do also all the time accidentally. So things to watch out for. So here, we have a list of all of the posts.

[00:05:24]
That's pretty good. If we want to list out of the users, we can also do that. Right, and I guess I forgot to instantiate the database with the value encoding JSON, but whatever. Close enough.