Unit Testing a Dataset & Algorithm

[00:00:01]
>> So let's make a new file called clustering.spec.ts, describe clustering, Okay. My cluster should load data. So one of the very simple tests we can do is just say, hey, are we even having a data here? Dataset, dataset, No, I expect this to be imported, Sorry. Expecting it to come from here, To match snapshot, Okay, so we can create a simple test to just see if the data is matching.

[00:01:19]
So, this test is again, not very useful, because we're just asserting that the data hasn't changed. But I usually like to create a simple test just to kind of get the harness working and proving to myself that things are working. So, where is it? Running the test, Why is our test not showing up?

[00:01:42]
It is showing up, okay, great. So it's showing up now, we create a snapshot. This doesn't really do anything, it just proves that the data hasn't changed over time. Okay, but let's make a useful test, it should create a cluster, Now, You might be tempted to do something like this, let's see, data set, You might be tempted to do something like that, but a couple of problems with this approach.

[00:02:24]
First of all, as you can see, this is slow, because data set is huge. So a matter of fact, I think the data set is so large that we have to slice this which is the 100 I see. So you have to do 0,100. Sorry, let's try this again, Okay, so it ran, it asserted something.

[00:02:53]
But the first problem is it's slow. And the second problem is, is this the correct answer? I don't know, right, if things go wrong, how would you know? Like yeah, you would know that things have changed, something that [INAUDIBLE] But would you actually know if this is a correct answer or not?

[00:03:12]
And so, one of the things to kinda point out is that when you write your tests, typically you don't wanna test it on large data sets or on the actual datasets because you wanna have it something very simple. So let's do something very simple, let's create our own datasets in here.

[00:03:30]
So what is a dataset? Well, the dataset is just something like this, right? So now if you look at this, let's make this easier here, let's do 10, 11, right? And now we just have four points. And when we do clustering, I think one of the arguments is the distance.

[00:03:49]
So in this case, we would say if a point is within five units, then we would expect that to be the same thing. So if you look at it 00 will be in the corner, right? 01 would be less than five units away from the first one. So the first two should be a cluster, right?

[00:04:06]
10, 10 and 11, 11 they should also be within the cluster of each other, right? So what we should expect to see is two clusters in here. So one thing we can do is we can focus the test this is always useful to do only. So we're running the test by itself.

[00:04:23]
So now we have a snapshot. Now, I think that snapshotting is really not the best way to do this. So I actually would prefer to just copy this and put it in the test directly. So you do this. And basically instead of saying expect, you say to equal, Okay?

[00:04:54]
And can we format this a little better? I think this is kind of easier to read and it is basically doing what we expected, right? We expect to be passed in four things and we're seeing the distance to be considered the same cluster as five. So 0, 0 and 10, 10 is clearly more than five units apart and so we got a cluster one, which contains 0 and 0 1, right?

[00:05:17]
The first two points we get the minimum and maximum for these values and we get the second cluster which also gets its minimum maximum for the values, etcetera and then we get the minimum maximum for the whole thing. And so the thing I wanna point out is that when you're writing a unit test, you really want to have super simple data sets.

[00:05:39]
You don't wanna go on the big data sets because you just can't really assert anything useful about them, right? And so here we could then maybe create other unit tests that would create smaller things or something of that sort. By the way, you can also create your own matchers.

[00:05:54]
So if you get tired of copying this particular thing, because this is pretty long, maybe you can have a matcher that does something simpler for you. Also, we don't need to copy everything. So for example, let's say we don't care about the min and the max like this to make the test easier to read, we can just delete them.

[00:06:14]
And of course, it's gonna complain that we have missing things. And so there is another function called toMatchObject, I think it is. Right. So toMatchObject says it has to have these properties. It may have other things, but it has to have these ones, right? And so in this way, you can kind of simplify it because you can tell the developer, look, these are the important bits, right?

[00:06:45]
The important bits is that we ended up with two clusters, the first cluster has these two items inside of it, the second cluster has those two items inside of it. And we have this particular output in terms of min and max values. So that when the algorithm that actually paints the stuff knows where to paint it.

[00:07:02]
Yes.
>> So here does it make sense to also mark the DBSCAN algorithm or is not because it's what we want to test?
>> Does it make sense for the-
>> [CROSSTALK]
>> To mark it,
>> To mark it, okay,
>> To mark Dbscan.
>> No, so I think in this case you don't wanna mark it because you specifically wanna make sure that the interaction between, the code that prepares the data for DBSCAN and the DBSCAN and what comes out of it and how you prepare it out, actually works, right?

[00:07:30]
So here's kind of the problem, we say unit test, because we consider this a unit but you can always argue that that's just a different layer. If you just go a little lower, like it's an integration between a DBSCAN and whatever, right? So these are kind of just words that we use very loosely with it, right?

[00:07:50]
And it's kind of what makes sense to you as a developer, right? And so the reason I wouldn't mark a DBSCAN is, first of all, it's fast. Nothing behaves more true to itself than a thing itself, right? So if you don't have to mark it, you don't. The reason we marked fetch is because fetch is not trustworthy, right?

[00:08:12]
It's an external system, it takes time, the system can change the output, it isn't trustworthy, right? Whereas the algorithm that we happen to use called DBSCAN, that actually works. And so now we can go back to clustering and for example, there are other algorithms that you could select, I think if I remember correctly, Is it from here?

[00:08:37]
Right, you can do KMEANS/ghich I I believe has this, no, let's do optics. So this is a different algorithm that you can replace. And you still see that it clustered the same way. So in this particular case, these two algorithms didn't have any material impact. And we could maybe come up with a corner case for the data points where there would be a material impact between the two things in there.

[00:09:01]
I'm just gonna put it back,
>> I feel like when you're working with legacy code, then something like the snapshot could be really useful.
>> Yes.
>> Where are you actually just want to make sure it doesn't change at all.
>> Correct.
>> Yeah.
>> Correct. Yeah, snapshots are super useful where everything is guaranteed not to change, right?

[00:09:25]
And legacy is an example for that, right? But if you're developing and you wanna move things around, snapshots are difficult to read, right? Because when something does change, it's like, well, was it the intended change or not? Then how do I review it? How do I make sure that that was what I wanted?

[00:09:44]
Also sometimes you have things like timestamps embedded inside of your snapshot in which case, those things become useless. You can't just do you have to kind of massage it before you do the next step, right? Yes, Mark.
>> It something like this be caught statically by a strict return type in something like TypeScript?

[00:10:05]
>> So type system is a form of testing, but it is different, right? The type system could never ever assert, let me go back to my test here, the type system could never assert that these two items end up in the same cluster. The type system can assert that the output shape of the object is correct.

[00:10:26]
That I haven't made kind of a logical mistake in terms of the way the types flow, but it will never assert that this will return two separate clusters, right? And just kind of to prove to ourselves, if I change this 5 to 15, you would expect to get a single cluster, right?

[00:10:46]
I just save this, you'll see that there's a whole bunch of red here. But fundamentally, you see that we have a single object now that has, Doesn't it show me? It just shows me the array clusters.clusters, let me print it again. Come on, .data
>> [LAUGH]
>> Clusters.

[00:11:15]
Right, no, but we do see the fact there is a single item inside of it, right? You see now there's a single item in here that has a minimum of 0 to 11 and four items are inside of it, right? So it decided that all of these items ended up in the same place.

[00:11:32]
So the clustering is doing the right stuff,