MCP Tools Q&A

[00:00:00]
>> Student: How does Claude know that it should ask the MCP server about the weather?
>> Brian Holt: It's a good question. So when Claude Desktop is starting up, it goes through its MCP config file, which is this. It sees that it has Demo Server and Weather Server. It runs both of these commands to get both of these MCP servers running in the background.

[00:00:24]
Then can we scroll up and still see it? We don't have it up here, but it calls that. Do you remember where we called Tools list, where it listed out all of the various different tools that it has? And so, Claude is just given just this box of tools when it starts up.

[00:00:41]
And then it decides at the beginning, someone asked me what the current weather is, how am I going to go find it? It makes a plan for how it's going to go do that. If we didn't give it the weather tool, it would probably just go search on the Open Web, what's the current weather in Minneapolis, and just use whatever it found there.

[00:01:00]
But it sees that it has a tool and it has a strong preference to use tools that it has available, that it has a tool for finding local weather and it'll decide to use that. So, it's always terrifying giving live demos with LLMs because you just don't know what's gonna happen, right?

[00:01:17]
There's a strong probability that at least one of these times it's going to be like, yeah, I have a tool for that, but I don't like that guy. I'm going to just go use the Open Web for this. Luckily, so far that's not been the case. And I look really smart right now.

[00:01:30]
Have you ever seen the temperature for an LLM? Like, whenever you're invoking an LLM, you have to give it a temperature. That's basically telling it how much chaos can you introduce into your system? If you give it a temperature of one, it's just always going to choose the very best result and it will have no variance.

[00:01:50]
So, in theory, if you call something twice with a one temperature, you should get the same thing both times, not always the case. But if you call it with like a 0.6 or a 0.8, I think most people recommend like a 0.8, which is like, hey, have a little bit of variance in how you respond to these things so that everything's a little bit different.

[00:02:06]
So that's where that temperature can kind of start coming in of, I should use the MCP tool that I have available to me, but I'm not gonna, right, I'm gonna go do something different. So that's probably a bit more of an answer than you expected, but that's in theory, it should have a tool available to it.

[00:02:22]
It should be strongly biased towards choosing the tool and it should use the tool that it has to find weather.
>> Student: How well does the LLM handle collisions? If we had a similar weather type of function in our tool in our other server and we had both of them enabled, does it get confused?

[00:02:40]
Does it ask for clarification?
>> Brian Holt: The answer to your question is just yes. Poor results is the answer to your question. We'll talk about this a bit later on. You want to expose as few tools as possible to Claude, because it does. Even if you have things like find the weather or find animal facts, the fact that you have multiple tools to choose from just adds confusion.

[00:03:04]
If you have a tool that you just know it's not going to use, just don't expose it. And the way you do that with Claude. Where's Claude is I would go in here and I would just, I would turn off getweather, right? If I'm not gonna use that, I just would turn it off.

[00:03:21]
And then it's not even exposed. It's not even causing problems, cause you get kind of two problems here. One, it adds confusion if you're asking it about animal facts and you have a weather function there until you're just wasting tokens. Because every time that you're providing this whole context to the anthropic model of, here's all the tools that I have, here's all the parameters, they expect, all that kind of stuff and it's just wasted context and wasted tokens because that all counts as input tokens.

[00:03:51]
Claude, as of right now, this will definitely change as time goes on, really can't deal with more than 40 tools. There's kind of like an upper limit there where things start getting wacky above that many tools. So yeah, especially if you have two things that do the same thing, it's going to pick one.

[00:04:13]
That would be my guess. It might give up, but usually it's going to try and it might even try both, which would be worse. Right.
>> Student: Could you instruct the LLM to specify which tools to use in various scenarios, like a system prompt?
>> Brian Holt: Yeah. So, I mean, you can see here the reason why I did that is because frequently if you say, hey, I want you to add two and seven, it'll be like, yeah, it's nine, what do you want, right [LAUGH]?

[00:04:41]
So sometimes I will get very specific of like in these cases, be sure to use the tool. Actually, I have one of these directly in my course, which let me show you here at the end. It's in the vibe coding one. So, this is for Claude code. You can see here, Neon Auth, it's like auth writer, clerk or something like that.

[00:05:13]
I've noticed a lot of times that the LLMs will get this wrong. And so I will say, please use Context7, which is an MCP server that I will show you here momentarily to make sure that you have up to date docs on this and this fixes it. As long as I tell, it's like please use docs for this, and that's what Context7 does, it provides docs dynamically.

[00:05:34]
That helps quite a bit right here. Again, please use context 7 liberally to make sure you have latest docs. Then this one, which goes on everything that I use. Do not write migrations yourself. Please use Drizzle or else it won't, use it every single time, right? And so, you have to get really specific of what MCP servers you want it to use when.

[00:05:56]
Because otherwise this will just try and write, it'll just YOLO SQL every single time, right? So yes, I do that all the time, in these cases, please use the MCP server. Do not try and do this yourself. Yeah.
>> Student: I might be getting ahead here, but the Context seven thing, if you're pulling in docs like Chad CN docs or something that's taking up your context window, right?

[00:06:17]
>> Brian Holt: Yeah, okay, 100%. The key here is to find someone else's credit card to put in there, not yours. No, I mean, I'm giving you some hints and tips and tricks around here, how to optimize your tokens a little bit. Unless you're coding a lot and doing this 100% through these, generally there's enough tokens to go around.

[00:06:40]
I wouldn't worry too much about fine tuning what's getting loaded when. This can make a big difference if you're like a massive scale company of what you're putting into context. But if you're just coding yourself, should be okay. So don't have a panic attack about it. It's what I'm trying to say, don't try and over optimize this problem.