Few-Shot Prompt Q&A

[00:00:00]
>> Sabrina Goldfarb: How often do you see few-shot prompting used over one-shot prompting? For example, is it worth the extra effort to craft that few-shot prompt versus maybe just using a one-shot prompt or using a one-shot prompt and having the agent create a few prompt? Yeah. So few-shot prompting is used approximately 15% of the time from the research that I've seen. And the reason for it is again, few-shot prompting got exponentially better, the larger that the models were versus one-shot prompting did get better, but not with that same growth of better.

[00:00:39]
So if you have any very complex tasks, you're going to want to use few-shot prompting. It is worth taking the extra time to say, let me think of a couple of additional cases or, like you said, have the LLM provide you a couple of additional cases, but make sure you go through them with kind of a fine-tooth comb of like, does this look correct? Is this the right schema I'm using if you're using it to generate code, and those kinds of considerations.

[00:01:06]
But I would say that from the research I've seen, 15 to 20% of prompts are few-shot prompts. So this actually is a pretty commonly used prompting technique. It just does take that little bit of extra time, but it's really good for those complex use cases. Yeah. And is there an easy way to monitor how you're using the context window? For example, with the many shots in your prompt, then filling the context window and potentially not being used then in the generation.

[00:01:37]
Yeah, unfortunately there's not. So there used to be—OpenAI used to have a ton of tools that were free and open to the public to really monitor and see. They do have a tokenizer, which you can use. So if you go to OpenAI, I believe it's just OpenAI slash tokenizer, but I can put that in the resources after this as well. You can actually copy and paste prompts and see how many tokens you're using.

[00:02:03]
So you have to remember that your tokens and your whole history are going to be appended every single time you talk to the model, right? So just make sure you're copying that input and that output and remembering that that system message is also back there stealing some of your context, but yes, you can keep track of it that way. Most of the time you're not going to need to, right? So if I'm talking about using a model that has over 100,000 tokens in its context window, if you really think about that, I mean, I could write like a whole book and be fine, right?

[00:02:37]
So the stuff that you're using, utilizing these models for day to day will be totally fine within that context window. The time that you're going to want to really take a step back and think, should I start a new chat, is if the performance of your model is degrading. So once you see any degrading in your output quality, that's when you're going to want to say either something is lost from context or something just isn't, you know, is confusing the model within that context and kind of start over from there.

[00:03:10]
I do have one other note because we're talking about some really long prompts, and this might be something that someone has run into today. This might not be something that you've run into yet today, but it's just called the continue pattern, very, very short. If your response ever cuts off in Claude and ChatGPT and Copilot, you can usually either there's a button that says continue or you can literally type the word continue and press send, and it'll just continue outputting whatever its output was.

[00:03:41]
But this is important to know because I have done this a million times where I have gotten cut off in the middle of an output and I'm like just waiting and I'm like, where, where is the output? So all you have to do is type continue or press continue if that ever happens to you. I imagine it'll probably happen to some folks at some point here today, so I just wanted to make sure I mentioned that.