CI/CD Q&A

[00:00:00]
>> Yeah, so you say a manual script, you mean like, a script that has manual prompts in it or a script that you-
>> No, I mean, you run it manually, yeah, yeah, without using CI/CD, the only other option you really have is just, [LAUGH] doing it yourself, right?

[00:00:14]
Or taking scripts and you mentioned as well, running it in a pipeline or something like that. Running scripts in a pipeline isn't as bad, but I would also say lean on the CI/CD provider. If they have tools that make it easier to like run a container, do that.

[00:00:32]
Don't run docker containers and stuff if they can do it easier for you. And GitLab and CircleCI even have that, where they have this concept of services. Here's a common example, I wanna test my code against my database. How do I do that, right, in a CI pipeline?

[00:00:50]
Well, there's two options, one, you mock every request, which is really the better way of doing it, so that there's no database required in the pipeline. Or your CI actually, while it runs tests, creates that service or creates that database for you, run some tests, and then gets rid of it, some CI platforms can do that.

[00:01:12]
And so if that's something that you need, would you rather write that manually, and then have to figure out how to maintain those commands, right? What happens if the cloud provider changes how you run it, right? That's where it become, basically, as much as you can avoid going into Bash.

[00:01:28]
[LAUGH] Unless, you really unless you really need to, yeah.
>> So is a script that's like run periodically on a cron job, only a step above manual scripting?
>> Yes, yeah, it's not what would be the difference. It's just not you doing it every six hours, [LAUGH] it's kind of the same there.

[00:01:48]
Now, if that's all you need, that might be valuable, okay, cool. Do we have any questions about CI and CD in chat or anywhere else? Yeah.
>> Would you consider git hooks pre-commit, pre-push as part of the CI?
>> No, I wouldn't, cuz they're not, [LAUGH] they're run on your computer locally.

[00:02:14]
Do I think that they're helpful? No, not really, nobody knows how to set them up right. Has anyone here used git hooks efficiently? You have almost, you're the only person in the room, [LAUGH].
>> What do you use it for?
>> Yeah, like what do you use it for?

[00:02:32]
>> Basically, check the code before it actually even gets pushed into the into git.
>> What do you check?
>> Basically, linters upgrade versions, change logs, it adds a bunch of information about the changes, make sure that it's compliant, it's basically linting.
>> Yeah, okay, so it's like pre-commit.

[00:02:51]
[INAUDIBLE]
>> Gotcha, so, My problem with git hooks and stuff like that in general is nobody knows how to set them up. Sometimes they'll use node packages to automatically add the hooks. Regardless, you have to figure out, it sounds like you guys probably have a specific way to how you set up hooks.

[00:03:15]
>> Yeah.
>> Yeah, if you do that and you can make it, so that they are not bypassable, then maybe, it might be worth it. But in a scenario where, like, are git hooks valuable if a person can commit without it running, would be my question. What are they gonna do in that case?

[00:03:39]
If I can commit, but they never run, and we've just found that pipelines guarantee they always run. And so, [LAUGH] it's kind of like a why can't we have nice things, [LAUGH] scenario and it's just unfortunate. Git hooks are really powerful, they're really nice, they are really nice, but they don't always run, they might not be set up properly.

[00:04:02]
Depending on the implementation, the developer might not know how to run them. So they're just not as dependable maybe would be the right word. You can use them, but use them wisely, yeah, yeah.
>> Can you explain that scenario again, like, the ideal scenario of me wanting to test my latest code against a database, and you said the ideal scenario is-

[00:04:24]
>> The ideal scenario is to mock those calls, so say I wanna test a database call for like a query, right? If you didn't mock it, again, you have to go through the process of figuring out how to get that database in the pipeline when it's running, right?

[00:04:40]
But if you mock it, what it means is is that when your logic goes to make that request, it doesn't actually make a request. You just have a piece of what we call mock data that you expect it to return. And so what'll happen is, it's kind of writing a script that only calls itself, versus writing a script that actually has to call something.

[00:05:01]
And so, for example, one of the words that they have is called fixtures, which is just the idea of like, this little piece of data is gonna be used in this test, and it will be used as a response for this result. And then that mock makes it, so you don't have to run databases, and I really strongly recommend that approach.

[00:05:22]
Hippo doesn't do that, and I can tell you right now, we probably spend half of our CI money on databases. Just because every pipeline needs one, and every developer got so comfortable with it that to the point now where I'm like, can you just mock it? And they're like, it's not mocked everywhere else, and I'm like, [SOUND] dang it.

[00:05:44]
[LAUGH] It's a tough situation that you can't get stuck in, but like mocking is the best way to go.
>> You mean like at a unit test level?
>> Well, it's not even, yeah, yeah, but like end-to-end testing, isn't in the same pipeline. Yeah, end-to-end testing should be ran elsewhere, I mean, on a cron or something like that, I guess, does that answer your question?

[00:06:09]
Okay, cool, yeah.
>> Yeah, people are saying they use pre-commit hooks for linting and running tests, and then also it gets a faster feedback loop. So the code doesn't need to leave the developers environment to check those tests, but it doesn't replace the need for a CI/CD pipeline.

[00:06:33]
>> Exactly, yeah, and that's the challenge is like, if you're still gonna make the pipeline, what is running in pre-commits that isn't running in the pipeline, right, and where's that validation? With you guys, I'm sure you said, there's actual commits happening, files are getting updated. You don't wanna actually push that, because it's manipulating the code locally.

[00:06:54]
But if it's just like running tests and stuff like that, that can always be ran in the cloud, and guaranteed to be ran every time you push it, so, yeah.
>> Why would you not run Cypress end-to-end tests in the CI?
>> So the easiest way to put it is, you have to think about what your goals are with testing, and what are you testing, how often are you testing, why are you testing?

[00:07:32]
I do think that there is over testing, where you test tons of tons of things, where it's like you don't really test one edge case or two edge cases. If you have a scenario where you wanna do actual end-to-end testing, then maybe at least it should be its own pipeline that runs separately, right?

[00:07:53]
Because when you think about what end-to-end testing requires versus just running unit tests requires, it's massively different. So then we say, okay, now, I wanna workflow that does both, [LAUGH] you know what I mean? And that becomes really challenging, I actually did implement it, so that our workflows at Hippo run Cypress end-to-end, and they can run it in the pipelines per push.

[00:08:19]
Do you know what that did? It added 20 minutes to every PR. Why? Because that's how long it takes to run the end-to-end tests. So that's what we ended up doing was this, we ended up creating end-to-end test repositories. So those run on their own, those have the end-to-end tests in it for Cypress.

[00:08:44]
The Cypress code doesn't exist with the service, it exists over here in the end-to-end test repo. And then that runs against the service whenever we wanna update or change the tests, see what I'm saying? And so then we can have it, so what we recently did is we actually made it, because they're over here.

[00:09:02]
We can run end-to-end tests before deployments, because we can trigger the end-to-end tests, and if they're successful, then go, okay, cool. But if they're joined together, you're just messing with a can of worms. So I would at least just say, separate them if you can to make your life easier, yeah.

[00:09:19]
>> Just somebody commented that they're gonna move from Azure to GitHub, because it seems like GitHub is investing a lot in this area.
>> Yes, yeah, they are, GitHub actions is really nice and it's getting a lot better.