This course has been updated! We now recommend you take the API Design in Node.js, v4 course.
Transcript from the "Q&A Part 1" Lesson
[00:00:00]
>> [MUSIC]
[00:00:02]
>> Scott Moss: All right, so now is the time for us to open up and ask questions about anything we've been exploring the last three days, or ask me anything about node, APIs, express, mongodb, mongoose, or anything internal. Yo.
>> Speaker 2: As far as API design goes, I've heard some people say like, let say you have, let say you're on, like, the Netflix home page and has all those movie tiles.
[00:00:32] They say, like, if you do a request for each one of those tiles, it's gonna be a lot of requests. So, there are like add parameters at the end. I want a group of six.
>> Scott Moss: Yeah.
>> Speaker 2: Stuff like that, so what I'm wondering is like how should you balance how many parameters and stuff you should put on, and how restful you want to design the API?
[00:00:51]
>> Scott Moss: There ares common patterns like ordering, sorting, filtering those are common limits. And then if you're doing something like consignation you have like a cursor where it says start, and you can figure out what page you are on. So those are common ones. As far as the properties, it depends on what type of properties you want people to be able to search by, for instance, if the user, all we have is a user name and a password here, but what if we had multiple properties on here.
[00:01:17] It also depends on what did you index right, so if you indexed the user name it might behoove you allow people to search by the user name do a query string, so you can get that benefit of that constant time look up speed of the indexed field. So, I guess it all depends, but there are the common ones like I said, and then it's gonna be up to you to pin on what you want the users to search by.
[00:01:40] Maybe you don't want the users searching by their Facebook access token. So, you don't want to put that there. But really what would happen is you would build like an agnostic query builder that doesn't know about what the query perimeters are and actually it doesn't care, and that's where Mongo kind of shines.
[00:01:56] We didn't get into the advance query of Mongo, because I told you that's a whole other thing by itself, but Mongo will allow you to like pass in different query params like that. And if you match them up exactly to rec.query.params, and pass in and out to Mongo query, you can pretty much get whatever they pass in right.
[00:02:16] And then you can leave it up to Mongo to decide if that is a valid parameter or not. So, you don't have to do the checking yourself right. Like for instance, on my user Hold on. Where's my user. Let's go to user. I only have her user name and password, but somebody typed in a parameter looking for an age.
[00:02:34] Mongo's not going to find anything with that on it. So, I don't have to check for it, although I could. You see what I mean? I could leave it up to Mongo to figure out, if it's going to find something with an age property on it. Which it's obviously not, cause no users have an age property on it.
[00:02:45]
>> Speaker 2: Yeah.
>> Scott Moss: Yeah, so like you can totally do that. Like, that would look something like this. So for like, for instance, I want to get users, but I wanna limit to 20, right. And you know, I want their age to be like equals and then like, you might do like some encoding here.
[00:03:03] It's like, this obviously isn't valid. You do like greater than. Something like that. You can do all types of things in here. You can even pass them like a JSON string. So it's up to you. There is no limit on what you can do as far as the query parameters.
[00:03:18] It's just like what type of- what does your client need? What type of data do they need? So, it's very much up to you. A good example, if you want to check it out is parts, parts api, they have one of the best apis I've seen, so I'd check that out.
[00:03:33] Yes?
>> Speaker 3: Some questions are rolling in here. What's the most effective way to identify the right module, or library when there's so many. Are there sites, or anything that help you?
>> Scott Moss: So, the way I do it, there is no, if it's really really good there will be blog posts about it.
[00:03:49] You just won't be able to, if there was something really good and you google how do I do this in node, it'll be the first thing to come up. If that's not the case, if it's not obvious, then what you do is you find something you think is good.
[00:03:59] The first thing I do is, I go to node, I click on the GitHub. That 404ed. Totally not gonna use that, right? So then I go, I stay in npm. I'll start Googling stuff, and a big indicator if something is great is when it was last updated. So, if I click on this.
[00:04:16] The thing's got 246 stars, relatively okay. It's got a link to a website that's being hosted somewhere. That means they probably put some work into it or they're just trying to market. I don't know. The readme looks pretty sophisticated. It looks like it's kept up to date. So I mean I guess it all depends on how well done is the re-meet, or the re-pull looks like.
[00:04:37] It was updated 17 days ago. That's pretty current. So, that's a good indicator that it's being used. So, obviously a combination of the stars on GitHub, the last time it was updated. If you can find any blog post, if you can find anybody talking about it. But if it's just something where it's just like, two stars, updated four months ago.
[00:04:56] The read me has like two lines in it, but you think it's exactly what you need? Probably not the thing. I would rather make my own thing than use that. So yeah, and it would be pretty obvious if somebody was, if this was proper. Just look at the activity, stars, watches, update times, and how nice is the readme.
[00:05:16] This is a pretty good readme. This person put a lot of time into it. So, yep.
>> Speaker 3: How does MongoDB performance compare with enterprise level databases like Oracle?
>> Scott Moss: Man. I don't know anything about Oracle, so I cannot compare the two. I've never had to use Oracle in my entire life, but I do know if you had Oracle, it comes with a db administrator that handles it for you, right.
[00:05:43] Doesn't it come with a person?
>> Speaker 2: [LAUGH]
>> Scott Moss: Right? Don't you think of Oracle, it comes with a person who handles that for you? But I can say that Mongo is very performant, because it's based off of an eventing system kind of like Note. So as far as performance, it's actually pretty fast, but it doesn't mean it's the best.
[00:06:02] Speed is not everything when it comes to database. It's all about all types of other stuff. Is it ACID-compliant? What are the reads, what are the writes? What is it good for, is it better for reads or the writes? There's a lot of the stuff goes in day-to-day support.
[00:06:13]
>> Speaker 2: Is the API usable?
>> Scott Moss: Yeah, is the api usable, can it [CROSSTALK]
>> Speaker 2: Mongo will allow it, because it's like a hybrid of
>> Scott Moss: Exactly.
>> Speaker 2: A lot of things are easier to get started with.
>> Scott Moss: Yeah, Mongo's very easy to get started with. Where it's like if you've never studied relational databases, and the first time you used a database was like I'm gonna use Postgre or SQL.
[00:06:35] You might have a hard time understanding why this stuff is. Like, what is a? What is a migration? What is this? What is that? Like, if you never studied databases, that stuff doesn't make sense to you, whereas Mongo is like just connect, and you're okay. Give me JavaScript.
[00:06:48] So it's like a lead, a lot of easier setup. So, yeah it's pretty fast because it doesn't have to do things like join tables and stuff like that. But again, it's not the best thing for everything.
>> Speaker 3: So I guess what they're asking, following up, if you're shipping an app in production with a huge amount of users, can Mongo support that?
[00:07:06]
>> Scott Moss: I don't know what a huge amount of users is, like I said at the end I don't think it comes down to how much it's storing. It's coming down to like what you're doing with the data, are you gonna be reading more than writing? What are you gonna be writing?
[00:07:18] How many concurrent users? How many concurrent writes are you gonna have, right? And I think that's more important than an amount of shared data that it's gonna store. I've heard problems with Mongo, where people are saying that sometimes things wouldn't write. I've never entered that problem personally. But again, I've never built a big data platform either.
[00:07:35] I'm not a big data person. So, I can not comment on that. But I've also heard other things about SQL and stuff. So, I mean, the the pros and cons are Mongo is easy to get started. It's really good for prototyping. You could probably get away using the production.
[00:07:47] Cuz it's really good. But then again, SQL is probably overall is gonna have better performance, Postgre SQL. But you actually need to know how to use it, it's only going to be good, if you know what you're doing. If you don't know how to administer a relational database, then you're not going to have fun with it.
[00:08:05]
>> Speaker 2: I think people tend to over think scalability, because they think my apps going to be huge.
>> Scott Moss: Right.
>> Speaker 2: Right, but when they actually run into issues in a company that's worth millions-
>> Scott Moss: Yeah. You got funny and you can fix it. Yeah. So, it's like, it's not even your problem now, like, just use it.
[00:08:22] I've never had a problem mongodb. I've built client work of mongodb. I've never had a problem. I have know people who have had that problems with, but I've never had problems with it.
>> Speaker 3: More stuff rolling in on that. So, when building a backend API like this, why use NoSQL database or relational database?
[00:08:40] Mongo seems pretty much trying to replicate much of the functionality of a relational database. But I think you kind of just covered that.
>> Scott Moss: Yeah, Mongoose is trying to, because the argument was Mongo doesn't care. You're right. Mongo doesn't care about the database. Mongoose is like, but we want that same thing as we did, relational databases, without the overhead.
[00:08:59] So, we're just gonna attach the Mongo. So yeah, but it's not exactly the same though, right. Like on Mongoose you have a schema that's guaranteed you that the data is going to be saved this way, whereas a relational database, it has nothing to do with the ORM. It's the database that's saying this is how it's gonna be saved.
[00:09:16] Relational database is that fix with columns, right. So like it's the database that's saying, this is how it's gonna be whereas like, Mongo, it's still just doesn't care. You just have this driver on top of it. This ODM, this ORM that's doing all that validation. So, it's really not the same thing.
[00:09:31] It's more so on a higher level. It's just very abstracted away. So it looks, you get the same result but the same thing is not happening. But there are pros and cons, Postgress users json now so you can throw json in Postgress, whereas you couldn't before so there's tons of support for that, and it also depends on what you're doing with your data.
[00:09:50] Like for a time series sort of data, I don't know, Mongo is probably good, but you probably want a better database for time series, but it also depends on the type of data. So and then there's like, you know, geospatial support. Mongo has pretty good geospatial support, but definitely not as good as like Postgress.
[00:10:05] Postgress has some of the best geospatial support. So, it all depends on the type of data you're having.
>> Speaker 4: Can you elaborate more on the time series data? If you're working on something like that? What do you recommend the most?
>> Scott Moss: I can't recommend anything, because I've never worked on time series data, but I can show you the best one.
[00:10:24] It's super easy. Best time series database. Okay that didn't turn- that didn't come back and do what it was supposed to do. I can't think about time, not influence, that's not the one. I can't, I can't recommend that. I have never had to deal with time series databases, so.
[00:10:44] But there are some, some good ones out there.
>> Scott Moss: I wouldn't recommend something like Postgress or something like that. I wouldn't recommend a relational database like that.
>> Speaker 4: How about something like Firebase or Mombo DB?
>> Scott Moss: Firebase.
>> Speaker 4: Or Mombo DB?
>> Scott Moss: I wouldn't recommend Firebase. No, I like Firebase, because they're realtime.
[00:11:02] I wouldn't recommend them as like my data store, like this is where I'm gonna store all the data for my applications. Time series data? No. You can't even query it, so you'd be screwed.
>> Speaker 4: If you had somewhere where you process all the work, but if you want to present data once the processing is done in time series maybe Firebase is good for that?
[00:11:22]
>> Scott Moss: When you can't process, you can't process correctly.
>> Speaker 4: But just to present the ones you like once your done processing in some other place to able to present it out and.
>> Scott Moss: I don't know I wouldn't use firebase for that. Your limited to what you can query.
[00:11:37] Like you can't say give me this specific thing and this specific time. You can't do that it's just like give me all the things. If you want to do that filtering for the client, you can do it. If that's what you're talking about. Yeah, you could totally do that.
[00:11:51] You're just like, dump me the thing and I'll do it over here. Yeah, you could do that. But for more fine-grained control, I want the database to do the sorting and filtering for me. You're not gonna, Firebase isn't meant for that. It's not what it's meant for.
>> Speaker 4: So then Mongo is better at that?
[00:12:04]
>> Scott Moss: Mongo is definitely better at that than Firebase, but there are still better things out there than Mongo as far as time series database goes. I just can't think of any, because I've never used them.