Linux Core Utils

[00:00:00]
>> We're gonna move on a little bit more. I'm gonna give a little bit of a Linux Core Utils like quick ones that I think are super useful. And just to kinda show you the power, just because a lot of times I do things that I feel kinda dumb about.

[00:00:14]
And so just recently, maybe like three months ago, we had kinda a competition on Twitch to see who was the greatest blindfolded programmer. Now I was second then I became first, and I became second and then someone cheated. But because he cheated in such a clever way, they gave me the loss instead, so I was in last place.

[00:00:31]
And they forced me to use Linux. It was very terrible, I hated everything about it. But nonetheless, if I would have just known the Core Utils a little bit better, I could have actually solved all the problems honestly in like 20 characters. They are so dang simple, except for the last one, I probably would have lost the last one no matter what turns out I don't know what some of these things mean.

[00:00:52]
So what are core utils, really how I think of them is they're just a set of programs that make your life easier doing ad hoc tasks, right? I know we use some of them in our scripts that we're building, like the cheat sheet and all that we're using fine, fine score Utah, right.

[00:01:06]
But they are just things that gonna make your life easier. And if you just don't know about them, you just don't know what you don't know, and you don't know why they are so useful. It's the exact same reason why when I was in that interview and the person said what about grap?

[00:01:19]
And I was like, what is grap? It was just obvious that I had no experience here, and I just wanted to kind of give you some nice things here. So here's a real-world use case. So I don't know if anyone knows anything about making YouTube videos, but I do.

[00:01:32]
And something that happens with YouTube videos is that YouTube doesn't care what you tag your video with. What it actually does is it goes through and looks at your subtitles, either the ones that you hand in or the ones, they auto-generate and go, okay. This is really what the things about.

[00:01:49]
And so often because I type so fast and ice cream so much I could not almost all my subtitles are generated assuming that I speak Korean, this is not a joke. I don't know what it is, it's just the way it is. And so I have to go in there and I have to correct it every single time.

[00:02:04]
But what I also do is go, okay. Did they even translate the words correct because if vim is translated them, which happens all the time, they're not gonna know how to categorize what I have made? And so here's a great example. I want to see what kind of words am I using in my subtitles?

[00:02:21]
Let's stack rank them, I wanna see the most use words to the least use words, right? So I'm gonna go download the subtitles script from YouTube and see what it is. So let's just do that now, if I'm not mistaken, I actually already have it in here, right.

[00:02:35]
The sub, I did want to touch on a couple of things. We were talking about Linux core utils, and I wanted to show just showcase their ability. I didn't want to go into in depth explanations, but really just to show you that, there's a lot that is available.

[00:02:51]
And if you're ever doing anything with logs, with this, that, and the other, there's just so much out there that actually can make your life a lot better. And so I have to be a little bit careful, I can see William in the feed right there, and he knows more than I do, likely in this category.

[00:03:05]
So I got to be careful, I don't wanna be wrong here. So it's just the way it is. But I'm gonna give you a couple examples of some classic problems that can be easily solved. But if you're not familiar with them, you will likely build a Node, Python script that does these things for you.

[00:03:25]
And they could have been done dramatically simpler. So let's get started, so as I told you, I was talking about YouTube subtitles. I want to find the most frequently used words in the YouTube videos that I'm producing. So let's do that right now, so I'll go in here.

[00:03:44]
I actually did find the subtitles, right? So here's an example thing of the subtitles, right? You can say look at all that, just a lot of words, right? So, likely before I knew anything about some core utils. What I would have done is created a little node script, right?

[00:03:57]
I would have read that, and I would have gone through each line from the content fact here we can even do this. Let's go like this, test dot j s And go const Fs equals require Fs. There we go, take that, take that go Fs Pap, right, replace that out and go Fs read file.

[00:04:17]
I'm gonna use refile sync. I know I couldn't use Fs promises, but come on, who does, who wants to use all that good stuff when you could just use old stuff. All right dirname and then go, what is it called sub, what is called? It's called x f subtitles, that's what it's called.

[00:04:33]
Ali right, there we go, I'm going to read that to string spook. Let's see, man, I can't even say that split on new line. And so now we have a bunch of lines, and so let's map that. My goodness, this is getting way too long. This is why you should never program JavaScript blindfolded.

[00:04:50]
This is the reason why it just keeps on getting longer and longer, I swear. Let's split each one of these lines on spaces, there we go. Look at that, it's just so small. My, let's keep going, you know what? Screw it, I only usually go to 80 characters.

[00:05:04]
Right here, but we're gonna just keep on going, let's see how far we can go. Now we have our accumulator and our word. And we're gonna go in here and that's that, okay? So we take our accumulator, if acc word, there we go. And not one of those, why are you out here?

[00:05:21]
Nobody knows, acc word =0, that's just so beautiful, isn't it? x word ++ go like that, return x. There we go do one of those and now 10 up go here constant words equals that back. Back down console dot log words, right? I believe we've done it, and let's just see what happens.

[00:05:46]
Node, what did I even call it, by the way? I call it a program, okay? Know to test dot JSON, right? And obviously I did a really good job I split it really, like really some serious IQ here. So let's redo that, obviously kind of a little bit low IQ there.

[00:06:04]
I wasn't technically planning on doing this, it just felt right. So split on the new lines and then split on spaces. And why did I get it, 0, 0, and 0, of course. So each line, of course, each word is not technically a word. It's, actually, like a line, and so let's do this again.

[00:06:24]
Yeah, this is such a great idea, I was like this ACC line, right? It's gonna be a line of white spaces. Let's return ACC concat wine, right, that creates a new one empty array. Yeah, how long is this line? We're going out of control here, it's so long.

[00:06:41]
I think that should work, there we go. And it didn't work, but you get the idea, whatever. I'm not even gonna continue on with that. It was fun to see how ridiculous I could make it. But you get the general idea what was happening there, right? Is that every single word I'm gonna have to go through I'm gonna have to Adam nailed it.

[00:06:57]
That was a serious first drive right there. Thank you, Taylor, for saying those nice words anyways, so you get the idea right? That is it's a lot of programming, right? And clearly I love going off to 170 characters. Also, my inner need to solve that has just gone so through the roof, it's like painful, but we're not gonna do it.

[00:07:16]
So instead let's just cut out subtitles, what do we see here? Well, we see a bunch of lines, right? And then of the lines, let's see those. Yeah, there has to be a better I know there has to be a better way, right? So of all these lines, there's just a bunch of text and then separated by white spaces.

[00:07:34]
So let's try, can we take all of these white spaces and tournament lines? So let's try some things. So I can cut out the subtitles, and we can pass it to something called xargs. Now notice what xargs did, look at that, that's kind of nice. We're getting a little bit better, each line got reprinted back out.

[00:07:51]
All right hey, all right, rock on, right? That's pretty cool, so now there's just white spaces in between every single item, right? Okay, so we could also pass it to tr. We can convert every whitespace to new line. Look, I mean look, look at that, look at how easy that was.

[00:08:08]
Telling me that was not easier than reading from a file, doing taking all those lines, blah, blah, blah, like I mean, this is already 10 times better. And I'm pretty sure I could have somehow reduced these two into something that made more sense. But we're not even trying to be efficient here.

[00:08:21]
I'm just showing you how easy knowing some of these tools can be to be really nice. Next, I can just sort it, look, we just sorted the thing. How simple was the average, just some sort, right? And then after that we can go like this, unique with a count, right?

[00:08:36]
And there we go, and then now they're kinda out of order. Still, so we can sort it and reverse it, or we can sort of remember their numbers, and then there we go. So now it's here's all my one, here's all my single occurrences on my double, triple, 5,10 all the way up.

[00:08:50]
So looks like the word two is my most said word. So when you're making YouTube videos, somewhere between eight to 12 mentions is kind of like your sweet spot as far as I can tell. And so as you can see here, I have like create need number this game.

[00:09:04]
Yeah, right player, these are all kind of good. I don't have them enough. So this is actually something I did was to intentionally try to mention. He works 8 to 12 times, and so I actually have to start looking at my subtitles and seeing. Am I mentioning the words that make sense for this thing to be promoted to the correct people enough time, so that Google going to favor this, right?

[00:09:25]
Some real like 4000 iq, by the way, haven't nailed it. And I also use Dvorak, by the way, so it's okay. I think it's okay, but there you go, I just solved such an easy problem. Like it was just that was like what 40 characters, it was extremely simple to solve.

[00:09:43]
And I used it with tools that have existed longer than I have been alive, right. This should be I mean we should all know these things unfortunately a lot of us don't I definitely didn't. If you would have asked me that just even a year ago, I would not have used that way right there.

[00:09:56]
So anyways, so I'm curious how many people chose awk how many people chose this like cat sort unique. Or how many people wrote a program If you're honest with yourself, I bet you there's a lot of people out there that probably just wrote the program. As opposed to this one off little simple thing that you can just execute.

[00:10:14]
And the best part is cuz I know this stuff pretty well if I need to rewrite it. It's not like this takes a lot of time to write, this is actually pretty straightforward, right. I just know off the top of my head, so I'll go, I don't have to remember and if I'm like I can't quite remember it, I can always use fcf and go.

[00:10:31]
Okay, yeah, here we go, I remember it's cat subtitles. Okay, there it is, I can just select the best one, right? So it just is very simple for me to use this. And I don't have to write a script, I don't have to maintain a script. I don't have to add features, right?

[00:10:45]
It's good enough.