Full Stack for Front End Engineers Exercise 16: Tailing a log file

[00:00:00]
>> Jem: So, now, our app is up and running. It's putting all the output into logs. So if there's any errors, it's gonna show up in our log, so we can diagnose this later. So let's learn how to tail a log file. I think of all the tools, other than VIM and just basic command lines, how to tail a log file is essentially how to see what's going on in your server.

[00:00:23] We learned top, we learned h top, which is just a pretty version of top. And that tells you kind of a high level about what's going on. But if you wanna know what's going on in a process, let's tail a log file. So, which one did I decide?

[00:00:36] Our auth log, perfect. So on your server, go ahead and sudo tail -f,
>> Jem: /var/log/auth.log. So auth.log is just people logging in and out of your server, it's kind of what's happening. Tail looks at the bottom half of your log. There's also head, head will look at the top part of a file and tail.

[00:01:01] Yeah, just like there's less, there's also more. There's a lot of ways to do things in UNIX cuz everybody's opinionated, they're like no, I wanna look at the top of a file, no, I wanna look at the bottom. But the -f command means follow, which means it won't just print the last ten lines, it's gonna follow everything that's happening.

[00:01:15] So I'm gonna go ahead and do that now. So sudo tail -f /var,
>> Jem: /log/auth.log. So now I see a real time stream of what's happening on my server. So I'm just gonna jump over to a different command prompt.
>> Jem: Well,
>> Jem: I don't think I can fit both of these on the screen at the same time.

[00:01:52] But since this is my auth log, I'm gonna go ahead. Let me highlight this to show that the tail's moving. I'm gonna ssh in the server, and it will show up in my auth log in real time. So ssh -i ~/.
>> Jem: What's my key name, my_key2.
>> Jem: I forgot what my IP address was, look at my IP address.

[00:02:40]
>> Jem: Cool, yeah, we couldn't see it, but the log file didn't move. This is how you diagnose problems on your server. So if you have a SQL database, you have a problem with your Node app, server's not up and running. Tailing a log file will tell you exactly what's going on in real time.

[00:02:57] It tells you precisely what the output is and is one of the most useful tools you'll ever learn tailing a log file. Know it, love it, none of this is sexy, there's no cool UI, this is just as old school as it gets. But this is how you solve problems on servers.

[00:03:13] Look, failed password for root, someone is trying to root into my server. There's nothing even on there, they don't know that.
>> Speaker 2: Yeah, so if you're seeing those pop up, that's what someone's trying to do?
>> Jem: Yes.
>> Speaker 2: So I'm just watching them, they just keep coming.
>> Jem: Yep, that's why it's always fun to watch the auth log.

[00:03:28] Yeah, some people are literally right now trying to break into the server. Good luck to them though.
>> Jem: And quit, oops, [INAUDIBLE].
>> Speaker 2: Quick question though, so had we not set up any of this ssh stuff prior to, they could have gotten into our server, right?
>> Jem: No, by default, Digital Ocean really holds your hand.

[00:03:53] So let's say, yeah, if you didn't put ssh key, they'll email you a very long password for your root password, rather than, they don't just leave it open. Cuz they know that the minute you bring up a server, someone's gonna try to break in.
>> Speaker 2: Well, that was the thing, yeah, if they're getting in 30 seconds after you create the thing, how do you even protect against that if they're that quick?

[00:04:12]
>> Jem: Exactly, the smart way to do it would, you would do this all on a local area network not exposed to the Internet yet. Then when it's all set up, you actually plug it into the Internet. But Digital Ocean, again, is extremely user friendly, where they'll mail you the password if you don't put an ssh key.

[00:04:25] But we learned today ssh is just so much easier [LAUGH] and much more secure than password based authentication. Owen asks, how do they find our servers so quickly? Well, it's just, these are the scripts running across entire IP ranges. So, people, I hate to say hackers, that's such a generic word, bad people, they know that Digital Ocean owns certain blocks of IPs.

[00:04:50] So let's say they own 155.155.whatever, they own that entire block. All they have to do is just keep scanning that over and over and over again, until they get a hit on a port, which means that server's up and running. And at that point, they try to break in.

[00:05:03] It's not an actual person doing it. These are all just like scripts most of the time. Sometimes it's a real person, but usually, it's just automated. And Francois G asks, how can we keep those logs under control? Well, thank goodness Ubuntu is great. Ubuntu will automatically clean up your log files for you, and it's gonna tarball them and gzip them.

[00:05:25] Actually, I think it just tarballs them, and it's gonna archive them, so it compresses them and then stores them for you. Eventually, probably over months and months and months, you'll have to clean up your log files, but because they're compressed and that's all done automatically, you really don't have to worry about keeping track of your logs too much.

[00:05:40] It's a very nice feature of using a well-built Unix system, but great question.