LevelDB & Crypto Adding and Querying Data
Transcript from the "Adding and Querying Data" Lesson
>> We do all of that and we insert it into the database. So if I run this program now, just gonna do our userlist again so I can remember some of our wonderful names we made. So I'll call mkpost with the first argument which is the name. And the additional arguments are, The message.
[00:00:21] So this is something that a cat would say, I think, great. So I think that should all exist in our database now. Maybe we'll make a couple more posts. This cat also likes to eat garbage [LAUGH]. How are you gonna spell it correctly here? It's a cat, what do you expect?
[00:00:53] [LAUGH] Maybe we'll add some other posts by some other users as well. And then we can write our lists. So I'll just say hi, I'm pretty boring. And doggymcdoggerton can say, you want dog doggerson and I get confused with them all the time also woof, okay, fun. All right, so now we've got some posts into our database that should have timestamps, and we should be able to pull down a list of all of the feeds.
[00:01:30] And do the kinds of things that you might expect to normally do with a database. All right, so I'm gonna copy our user list file and call it postlist. And we can start to modify it. So there's different ways that we might want to sort our data. The first one we can do is just pull down all of the posts and just print them.
[00:01:53] So just to make sure that everything was written correctly into our database, I think that's a good thing to do. So these are gonna be the primary keys that we're gonna look at, because they start with post. If I run that, we get, yep, we get some stuff.
[00:02:10] I forgot to turn that entropy into hex, which I forget all the time. So we've got some nasty stuff in our dB. Why don't I go ahead and fix them, make a post function to do toString('hex'). Easy to mess that up. Fortunately, now we've got junk in our database, so the best way to deal with that, [LAUGH] this handy command called the rm.
[00:02:37] All right, so now actually if I grep for node mkpost, we've got all of our post commands. So why I don't I go ahead and I can grep- v so the history commands don't show up in there. There we go. So I'm gonna go ahead and set off the first bits of that, I guess, cleave off all of the integers.
[00:03:08] And then I'm just gonna run that. I think that will work. I don't think that that will do anything so terrible to my system. Great, so that probably works. Who knows. All right, so now we can run postlist. And yeah, so these are better keys, done. But that was kind of fun, anyways to do, to fix it.
[00:03:29] All right, so we've got our primary documents, it looks like everything is being stored correctly. I can make this a little bit prettier. So maybe we don't wanna see the garbage around how the key's formatted. We just want it to be maybe if the hash and the username.
[00:03:50] I see another problem with our mkpost file. It doesn't have the username in it. I think I meant to do that it just has the key. So we really ought to have the name in there, probably the timestamp as well. But these things can also live in our document body.
[00:04:08] So that's a good place to put them as well. So there we go. This is our good friend rm again, and use that sweet little bash script and that fixes it. There we go. So now we've got all of the data that we need to have kind of a pretty output.
[00:04:28] So let's go ahead and make some pretty output. So the row.key is gonna give us the idea if we split it on exclamation mark. So, go ahead and pull out some parameters. The next thing that we might want is we wanna know that username, which is the row.value.name.
[00:04:46] And then the timestamp is gonna be the row.value.time. And the body is gonna be the row.value.body. So these are the things that are just in our data. I guess I can say, maybe I'll do it like RC style with brackets and the body. And, I don't know, some clients have a timestamp as well.
[00:05:08] So that'll look kinda nice. Who knows? Whoops, you have to remember to put all of your parentheses in. So, cool. Here's a way that we can display our information that's kind of fun. It's like RC style. So maybe though we want to list all of the posts now for a particular user.
[00:05:30] Maybe we just wanna see tuxkitty or doggerton's posts. So we can use one of our secondary indexes now to show that. So, gonna make a copy of that file postlist and we'll call it post-by-name, post-by- author, that's more dignified. So we're going to post-by-author. And now if we go ahead and open up the mkpost file, just to look at our keys again.
[00:05:58] So to list posts-by-author, we can use this other key called post-dash-name. And we can make a range query with a greater than or less than that's gonna pull down everything by a particular author. So those keys are gonna look like this. So it's gonna be post-name. I'm gonna save that to separate objects so we got some more room.
[00:06:19] So we're gonna have our gt: 'post-name!' + name, which we can get from the command line arguments. And then the lt: 'post-name!', + name +! Because there's more information. There's, I think, also the timestamp. Yeah, at the end. Yep, and also the IDs, so that our keys don't clobber each other.
[00:06:46] Because if you write over the same key, it updates it, like the first example we did with the counter, it does it in place. Okay, so that's a lot to deal with. And I think we can preserve our original formatting. However, these rows are not gonna have the body in them.
[00:07:04] They're just gonna have this placeholder value of 0 or an empty string or something. So if we wanna actually pull down the original body, we have to do something like db.get. So what those posts are gonna look like is if we know the ID, which happens to be at the end of our key, then we can look up the primary document.
[00:07:25] So, In this case, the ID is gonna be the last one. So I think you can just slice that off, -1. Yeah, that probably works. So it's called 'post!' + id. Yeah, exclamation mark. And we get the original document. And now instead of row, we can use post.
[00:07:53] So doc, doc, doc. And now if we've done everything correctly, we should be able to say tuxkitty. Whoops, I guess there's a syntax error. There we go. If we fix that, Right? So it's not giving us what we want. So why don't I go ahead and, Print out the rows to see if we're getting anything from there.
[00:08:23] No, since we're not getting any rows, let's go ahead and look at our keys again. So here we have post-name. This should be stuffed with name in there. What we can do is just dump everything from my database. Let's just dump everything from our database to take a look again.
[00:08:44] Oops, and then we've got a, Bailout before any of this stuff happens. So I'm just gonna do an early return. So this is just the first one, also have to call next if we want it to keep going. All right, so here we have things like post-name, so that all looks good.
[00:09:10] Maybe I just didn't spell tuxkitty correctly. No, that looks good as well. Anyone sees anything, please shout it out on the chat or in person. It should be post-name, right? Let's put it
>> Is it because of that trailing exclamation though, because there's additional stuff after that author name, right?
>> You're right, I think. All right, that is a good insight. Okay, so let's just print out, and then you have to give it something. Yeah, that was it. Thank you for that, very good. All right, so here we have all of the posts, and I'm just gonna re-enable, so these are all of the secondary index keys and values.
[00:09:52] I'm just gonna re-enable the part that does the db.get. And then we should have our nicely formatted data except it's not value anymore. Cuz it's string db.get. And then you don't have a key value object, you just have the value. So there we go, cool. So let's just quickly modify this one to do posts-by-everyone ordered by timestamp.
[00:10:19] And then I think that'll be a good time to break. And you can all play around with this silly contrived example and then we can do some other stuff, all right? [LAUGH] All right, so posts-by-author now become posts-by-timestamp. Posts-by-time, I like that a little better. So now instead of doing post-name, here we have our original code that writes it.
[00:10:46] So we want to do post-time. And I'll just print out everything. We don't have to sort it by name or anything like that. --, if we wanted to, though, we could pass in additional arguments on the command line that could slice up that time range a little bit more closely.
[00:11:04] But for now, I think this is good. Just like post-name, the ID is the last element in the time. So we can leave the rest of our code alone and it should just print out everything in the database ordered by timestamp. There we go. So I think if I do postlist now, it's actually not ordered by timestamp.
[00:11:25] So the order might be a little bit different. Except because I did these with a little history, one liner, they're all like almost the same second. So whatever, you get the idea, so