Where to Measure Performance

[00:00:00]
>> What this was, was lab data, it was our own private measurement. And so this brings up an important point about where should we measure performance from? So let's think for a minute about how a website works, how web application works. You have your servers, and you have your users.

[00:00:19]
And there's a whole bunch of magic network things that happen in between. Fundamentally, what your servers need to do is produce some documents, some JavaScript, some CSS, some images, and send those across the wire to your users, who will run it in their browser and render a page.

[00:00:39]
So how do we measure how fast this is happening? Well, what we just did, was lab data, is we connected directly to our server from our dev machine, from our local dev machine. And we measured how well does this perform for us using our laptop. Not really for the user, not really for anything else, just how we see our data.

[00:01:05]
If this is something that we're running locally, like if we're running the site on our machine and we're testing it from our machine, then we've taken the network out of the equation entirely. We've not tested anything about how a real network performs. We're just testing how fast our loopback interface in our own machines can run, which is generally really, really fast.

[00:01:26]
So these are tools like Lighthouse. Well, there's another way you can capture data. We can capture things synthetically, which is there's a robot somewhere out in the network that will periodically load your page and tell you how it works. In this case, we're testing some of the network.

[00:01:46]
There's a test server, it'll run a window of Chrome, and it'll give you some performance metrics about how their instance of Chrome in that server ran it. And this is pretty good. There's a lot of tools out there on the marketplace that will do this. There's things like New relic ,and Pingdom, and those sorts of things.

[00:02:06]
You can buy a service that will do this for you and periodically tell you about your performance. Well, the third way that we could do this is called field data. Because even synthetic data isn't what our users saw, it's what a test machine that's running on very stable hardware on a stable network sees.

[00:02:25]
It'll be best if we could actually capture information from the people who actually use the site, on the networks that they use it from, and see how the real users experienced the site. So if we could gather an experience report, if we could capture that lighthouse report from every user every time they load the page, and send it back somewhere so that we could analyze it.

[00:02:51]
Well, this is what's called a run tool, a real user monitoring tool. And it's a tool like the one I work on called request metrics. And there's a lot of good things about doing it this way, as in it's much more accurate. It's showing you what the real user sees.

[00:03:07]
But it's also really noisy, because a lot of users run on really terrible networks and terrible computers. And you don't necessarily care what all of your users experience. As in, let's say if you're running a jobs board in India, and you all of a sudden get a whole bunch of traffic from outer Mongolia.

[00:03:26]
Well, you might not care what the people from outer Mongolia care, what they see, like it doesn't matter what their performance was. They're not your audience, they're not the people who are most interested in your site, they're likely bots.