Model Selection

[00:00:00]
>> Sabrina Goldfarb: I have found that most people never try new models other than the ones that they already feel super comfortable with, and if they do try a new model, they are very unlikely to try a new provider. And this is exactly why today I decided to bring up Claude and chat and bring up Copilot using GPT-4o, because I think it's important to see that all of these providers are different, but their models are all incredible.

[00:00:28]
Some models will be faster, others will be slower, some will be better at making things look good at a first pass. Some will be better at handling complex logic, right? So sometimes when we're building AI applications, certain models won't be worth the money, right, because they're too expensive and we don't need them. An example that happened with me with that was one of the side projects that I work on pretty consistently is a fintech application, and I was creating a very simple classification system for some, you know, information we were pulling in.

[00:01:08]
And I checked out the difference in pricing between what we were using, which was Haiku and Sonnet 4 when 4 came out. Sonnet 4 was at the time at least 80% more expensive than Haiku was, and I'm running this thousands of times a day. So I was like, well, is the model that I'm currently using providing me accurate enough and factual enough information where I don't need to make the swap? And to me, I didn't need to make that swap because 99% of the time Haiku was enough.

[00:01:41]
But that doesn't necessarily mean that a new model won't come out, make Sonnet 4 super inexpensive, and I won't swap. It's also important to know that eventually models will be deprecated, right? So Haiku could go away tomorrow. And when you're building AI applications, it's really important to make sure that you are ready for that, right? We have to prepare for the future because we don't get a lot of notice in this world of AI tools.

[00:02:09]
Sometimes a new tool will come out and it'll be a surprise tomorrow. Sometimes a new tool will disappear, and it was a surprise yesterday. So you want to make sure that the code that you're writing is going to be flexible enough and the prompts that you're writing are going to work well enough that if you need to change these tools at the drop of a hat, that you absolutely can, which is why I say try multiple models, try multiple providers, because maybe Sonnet 4 will still be too expensive for me, but maybe one of the GPTs will work just fine, right?

[00:02:39]
And maybe it'll be less expensive because maybe it's an older GPT. So try new models, try new providers, try everything. The last thing I'll end off on that note is I was once taught in college in a psychology class about the fact that the reason commercials like when you're watching TV and watching commercials, no one does that anymore, I know. But the reason that they target people that are like 18 to 24, like 16 to 24, especially like makeup commercials, the reason that they target people in that age range is because once you use their product at that age, at that really important age of like 18 to 24, like early adulthood, you are likely to be a loyal customer for your entire life and never try anything else.

[00:03:30]
And so I just want to put that out there that innately we get comfortable and we feel comfort about, I really like the way Claude talks to me and so I use Claude most of the time. But I have to push myself sometimes out of that comfort zone to say, especially as an engineer and a developer, like, what is the actual best tool for what I need to use? And that might mean using Google Gemini, even if when it first came out, I was like, there's no way, right?

[00:03:58]
But now it's pretty incredible. So just make sure that you are changing the tools based on your needs and you're giving every provider and every model a fair try.