Logging, Streams, & Redirection

[00:00:00]
>> So let's learn about logs and how to read them. And you saw right there why we need to log everything because other than that we don't know what's going on in our system. If it's not being logged by something we created, how do we know if it's broken?

[00:00:11]
How do we know if it's working? You just guess? I wouldn't like to guess here, we really wanna know. We want the details. So when it comes to log files, there's a few different ways of reading it. Cat, we've already looked at. Cat's useful, but it dumps the entire log file.

[00:00:28]
So if you have a massive log file, kind of like our syslog or /etc services or something like that, a giant file, not super useful. The command I used just there was tail and tail -f is probably your most useful command when it comes to tracking a log, cuz tail is just gonna output the last part of the file and then we can follow it.

[00:00:50]
So we can go line by line as everything happens. And you're like, why there's so many different ways of reading a log file? Because sometimes you need to read the end, sometimes you just want the top of the file. Sometimes you want the whole file, but you want to read it one page at a time.

[00:01:06]
Sometimes you just want the whole file. Hence, when you look at it initially, it doesn't make a lot of sense why there's so many, but now we understand all these use cases where sometimes you want the end, sometimes you want the beginning. And then the common logs that we want to look at generally are syslog, what's going on our system.

[00:01:23]
The auth.log, what's going on, people try to log in, which we've looked at a few times. Then our nginx, what's happening on our nginx connections as well. So those are some common logs we wanna look at. And where are logs always located? It's on the screen.
>> Var/log/.

[00:01:41]
>> Var/log/, yeah. So we always want our logs in var/log/, and we want this consistency across our system. Yeah, you can put logs anywhere if you want to. However, it makes it easier when all the logs are in the same place. So we're gonna see how to read logs, all the different ways of reading logs.

[00:02:03]
Let's talk a bit about standard streams. And I've touched on this a lot, or a few times. I've said standard in, standard out, standard error. You're probably nodding along, you're like, yes, yes, yes. But, this is one of the most amazing things about Unix, it's called the POSIX standards.

[00:02:20]
POSIX is the portable operating system interface. But it just means every single command we run has an input, called standard in, it has an output, called standard out, and it has a standard error. And every single function we write takes these three arguments every time. That's all they take.

[00:02:41]
Which is really powerful cuz that means we can chain together commands to do whatever we want, because we know they all have the same function signature. And I'm really not emphasizing this enough how powerful tool this is, that every function we ever wanna run, all take the same arguments, every single time.

[00:02:58]
Imagine if you can write your code like that. Think of what you can do, you can do anything. You can chain everything together. And that's what we can do with standard streams. And we can chain commands so long, which we have a few times already, into anything we want in that.

[00:03:13]
That's why people are really good at Shell programming and Shell scripting command line. We'll run this super long commands and they have exactly the data they need and you're like, how did they do that? It's because standard in, standard out, standard error, everything takes the same thing. And we've already used the pipe command before.

[00:03:30]
So pipe is just gonna say, that's the vertical bar. It just reads from standard out. And we've used that already to pipe things from one place to another without writing to a file. Now if we actually wanna write it, we use the greater than symbol, the right arrow, whatever you wanna call it, so that'll writes anything to a file.

[00:03:51]
And don't just take my word for it. Let's check it out. So we'll use our good echo command say echo hello. I can write it to a file called foo. And then if we cat foo, says hello. Pretty cool. But, if I want to do this again, but I wanna say hola.

[00:04:17]
The problem with that is it just overwrote, hole, [LAUGH]. So it just overwrote what we had. So we're gonna use the double arrow, And that will append to the file So now we're kind of saying, this is a good way of writing to log files. So we can use a double arrow and point to wherever we wanna write and will append to a log file.

[00:04:52]
And we can go the other way too, we can read input going the other way. And this is probably the one that kind of messes with your head a little bit. We use the arrow pointing the other way, which means we're piping from the right side to the left side, where normally we always read from right to left.

[00:05:09]
So it gets a little tricky. But we don't use this one as much, at least in this class. And then there's this weird symbol down here, which is the 2, the right side, and the ampersand 1. And that just means we're gonna redirect both all the outputs. So standard in and standard out are both considered output.

[00:05:32]
We're just gonna redirect both of those into a new file, and this is just a shorthand of doing that.