Preventing Bugs

[00:00:00]
>> So think about in your app, how do you lower the risk of change? Once we've fixed a bug, how do we prevent it from happening again? This is really just about having a good think about it, after you've fix the bug, over a beer at the pub, cuz you've all ready released it and you're good.

[00:00:20]
Think about what could we have done to have that not happen? Am I missing a test somewhere? Maybe I didn't test a particular scenario in IE nine that I really should have. And I should need to go back and write a new end to end test or maybe I need to update my process.

[00:00:36]
Maybe I need to say, you know what, I really should have a code review that covers this particular thing every time I do this kind of change. Maybe I should be pairing with somebody or maybe it's nothing even that maybe I just need to run one more kind of update as part of my automation.

[00:00:52]
Think about what kind of automation you can put in place so that you're not introducing manual steps or you might just do nothing. Remember that there is a cost to everything we do. Tests are not free. Automation is not free. Process not free. If it's a stupid little bug, that was a stupid little mistake and you'll never make it again, then don't increase your overall operation cost for something that's never going to happen again.

[00:01:18]
>> I do feel like I have a story around that whole like-
>> Yeah, bring up. What's your story?
>> Yeah, so I mean with our, with our player experiences obviously what we do for our members is we deliver video like that needs to happen,
>> Delivering the video is the competitive advantage of your company.

[00:01:37]
It's like you have to do this really well.
>> Yeah, exactly, it needs to be a great experience. And so, we had a video host and I don't wanna mention them by names to throw him under the bus or anything, but we noticed errors coming through and there's literally nothing that we can do to fix it because it's third party code unless we go in and patch there minified mucked up.

[00:01:59]
>> Just go ahead and monkey patch.
>> So we actually got them to install TrackJS to prove to them that this error was indeed happening. And so I think they still have Track, they're using it, but it was, no, you need to install this to see this error happening because it's happening and it's happening to a large percentage of our users.

[00:02:20]
And so yeah, that's actually how we had to come to the conclusion of building everything custom, building everything from the ground up. This is our main core business, so we had to rip that third party code out, which was the player, right, and build from scratch. And it did take us over six months, there's a lot of engineering.

[00:02:42]
There's a lot of decisions to be made, but we deployed it. And now when a bug comes up in Track, right-
>> [LAUGH].
>> We can actually fix it now, because we own that entire stack from nuts to bolts, every part of that code is within our system delivered via CVN.

[00:03:05]
So it has been wonderful, right? Because there's people that just could not use our system before. And there was third party things that were happening, especially in certain countries and now we're able to verify that indeed that it's actually a there are no errors happening on that page for those users in that country in that region.

[00:03:30]
And it's fantastic. And so yeah, thank you Todd for making such a great product that we were able to develop-
>> [LAUGH]
>> And have it running and so we we do rely on Track and we, it's a never ending battle, especially with third party JavaScript. It's like, you've got Intercom, you've got trackers, you've got all this stuff.

[00:03:49]
>> Cause they all have teams that are running it stuff just as fast as you are.
>> Yeah.
>> And they're all running in different directions.
>> Yeah, I mean the more you can eliminate third party code and have it in house, at least for your core, the things that you're supposed to do well as a business.

[00:04:04]
>> Yeah.
>> That's really important. That's my little-
>> Good thank you. That was actually a fantastic example of thinking through the bugs that you were having as Mark would run into problems with the video player. And having a good think on why were those happening? What could you do to change it and ended up making an architectural change, which kind of goes to this next point of thinking about your bug code as a proxy to what part of your code has different levels of design quality.

[00:04:38]
And so here is my super awesome generic, all purpose architecture. You get requests coming in the front door and it goes to the Doerthinger, which does some stuff and might send it to the all powerful cloud, or it might send it to the Widgetizer both of which ended up dumped into the Enterprise- Ready Cylinder.

[00:05:02]
I used to be an architect. If you take a look at your architecture, which hopefully doesn't look anything like this, and you just start attributing to where are your bugs going? When you get bugs in over the course of a Sprint's iteration week, month release whatever you want to call it.

[00:05:26]
Where are they? What parts of your system have issues? If the Doerthinger has zero bugs being reported, it's probably relatively stable for what you need to do with it. But if the all powerful cloud, or the video player has 51 bugs out of this, that's probably a part that warrants some development attention.

[00:05:49]
Maybe it's a redesign that needs to happen there. Maybe there's something about the development team or the design or the architecture or the operations of that component that is causing bugs. And so, using just the idea of how many bugs are happening as a proxy for where, what parts of your system need more design attention, and you don't need anything big for this.

[00:06:11]
You don't need to go out and buy JIRA and track system architecture components and cross link and everything. Just anecdotally, you and your team know this, right? If you just have, sit down and think about all right, in the last month, wherever your bugs been? And you just think through what were the big, hairy, nasty ones that really pissed everybody off, and what parts of your architecture did they come from.

[00:06:36]
You have a pretty clear idea of where you need to focus attention. So that's thoughts on design for debuggability. Paying more attention to what are some things you can do in your app to make each of these phases of bugs easier.