Few-Shot Prompting at Scale: Example Selection & Format
# Few-shot prompting provides examples of desired input-output pairs. Done well it's more powerful than fine-tuning for many tasks. Learn how to select, format, and dynamically retrieve the right examples.
Few-shot prompting — providing 2–10 examples of desired input-output pairs before the actual query — consistently outperforms zero-shot for specialized tasks. When done carefully with the right example selection strategy, it can match fine-tuning quality at zero training cost.
Why Few-Shot Works
Large language models are pattern-matchers. When you show them:
Input: "The product broke after one use" → Output: sentiment=negative, category=quality
Input: "Arrived two days late" → Output: sentiment=negative, category=shipping
Input: "Best purchase this year!" → Output: sentiment=positive, category=general
Input: "Works great but packaging damaged" → ?
The model infers the format, categories, and reasoning pattern from the examples. No fine-tuning required.
Static vs. Dynamic Few-Shot
Static: same examples for every query (simple, good enough for uniform tasks)
Dynamic: retrieve the most relevant examples for each query (better for diverse inputs)
// Static few-shot
const STATIC_EXAMPLES = `
Example 1:
Input: "Calculate the SHA-256 hash of 'hello'"
Output: {"function": "hash", "algorithm": "sha256", "input": "hello"}
Example 2:
Input: "Convert 3.14 from float to integer"
Output: {"function": "convert", "from": "float", "to": "integer", "value": 3.14}
`;
// Dynamic few-shot: retrieves similar examples from a library
async function getDynamicExamples(query: string, exampleLibrary: Example[], topK = 3): Promise<string> {
const queryEmbedding = await embed(query);
const exampleEmbeddings = await embed(exampleLibrary.map((e) => e.input));
const ranked = exampleLibrary
.map((ex, i) => ({ ex, score: cosineSimilarity(queryEmbedding, exampleEmbeddings[i]) }))
.sort((a, b) => b.score - a.score)
.slice(0, topK);
return ranked.map(({ ex }) => `Input: ${ex.input}\nOutput: ${ex.output}`).join("\n\n");
}
Example Selection Principles
Not all examples are equally effective. Prioritize:
1. Diversity — cover the space of input types, not just the most common:
function selectDiverseExamples(library: Example[], topK: number): Example[] {
if (library.length <= topK) return library;
// Max marginal relevance: diverse AND relevant
const selected: Example[] = [library[0]]; // start with most representative
while (selected.length < topK) {
let bestScore = -Infinity;
let bestExample: Example | null = null;
for (const candidate of library) {
if (selected.includes(candidate)) continue;
// Relevance to query
const relevance = candidate.relevanceScore;
// Diversity from already-selected
const minSimilarityToSelected = Math.min(
...selected.map((s) => cosineSimilarity(candidate.embedding, s.embedding))
);
// MMR score: balance relevance and diversity
const mmrScore = 0.7 * relevance - 0.3 * minSimilarityToSelected;
if (mmrScore > bestScore) {
bestScore = mmrScore;
bestExample = candidate;
}
}
if (bestExample) selected.push(bestExample);
}
return selected;
}
2. Correctness — bad examples teach bad behavior:
// Every example in your library should be manually verified
interface Example {
id: string;
input: string;
output: string;
verified: boolean; // must be true before using
quality: number; // 0–1 human rating
category: string;
embedding?: number[];
}
// Only use high-quality verified examples
const productionLibrary = exampleLibrary.filter(
(e) => e.verified && e.quality >= 0.8
);
3. Format consistency — all examples must follow the exact output format:
function validateExampleFormat(example: Example, formatSchema: z.ZodSchema): boolean {
try {
formatSchema.parse(JSON.parse(example.output));
return true;
} catch {
console.warn(`Example ${example.id} has invalid format`);
return false;
}
}
Example Ordering Matters
Research shows the last few examples have the most influence. Put your best, most representative examples last:
function orderExamples(examples: Example[]): Example[] {
// Sort by: increasing difficulty, with highest quality last
return examples.sort((a, b) => {
if (a.difficulty !== b.difficulty) {
return a.difficulty - b.difficulty; // easy first
}
return a.quality - b.quality; // best examples last
});
}
Format: What to Show in Examples
function formatFewShotPrompt(
instruction: string,
examples: Example[],
query: string
): string {
const orderedExamples = orderExamples(examples);
const exampleBlock = orderedExamples.map((e, i) => `
Example ${i + 1}:
Input: ${e.input}
Output: ${e.output}
`.trim()).join("\n\n");
return `${instruction}
${exampleBlock}
Now handle this:
Input: ${query}
Output:`;
}
For complex reasoning tasks, include chain-of-thought in examples:
function formatCoTFewShot(examples: CoTExample[], query: string): string {
return examples.map((e) => `
Input: ${e.input}
Reasoning: ${e.reasoning}
Output: ${e.output}
`).join("\n") + `\nInput: ${query}\nReasoning:`;
}
interface CoTExample {
input: string;
reasoning: string; // step-by-step thinking
output: string;
}
const COT_EXAMPLES: CoTExample[] = [
{
input: "Should I use an index on a column with only 2 distinct values?",
reasoning: "2 distinct values = ~50% selectivity. Index scans are only faster than full table scans when selectivity is <5-15%. At 50%, a full table scan is typically faster because it has better cache locality. Exception: if the table is very large and you always filter by the rare value.",
output: "No for typical queries. Indexes help when selectivity is <15%. With 2 values and ~50% distribution, the query planner will likely skip your index. Exception: if one value is rare (<5%) and you frequently filter for exactly that value.",
},
];
Automatic Example Generation
Build your example library automatically from successful real-world examples:
// After each successful LLM call, optionally save as example
async function maybeSaveAsExample(
input: string,
output: string,
score: number,
threshold = 0.85
) {
if (score < threshold) return; // only save high-quality outputs
const embedding = await embed(input);
// Check if too similar to existing examples (prevent duplicates)
const similar = await vectorSearch("examples", embedding, 1, { threshold: 0.97 });
if (similar.length > 0) return; // too similar to existing
await db.insert("examples", {
input,
output,
quality: score,
verified: false, // human must verify before going to production
embedding,
createdAt: new Date(),
});
}
How Many Examples?
| Task complexity | Recommended examples | |----------------|---------------------| | Simple classification | 2–3 | | Structured extraction | 3–5 | | Complex reasoning | 5–8 | | Code generation | 3–5 (longer examples) | | Creative tasks | 2–3 (too many constrains) |
Beyond 8–10 examples, you're using a lot of tokens for diminishing returns. Consider fine-tuning at that point. See Fine-Tuning vs RAG vs Prompting for the decision.
FAQ
Does few-shot work better than zero-shot for all tasks? No — for tasks where the model already performs well (translation, summarization), zero-shot is fine and saves tokens. Few-shot helps most for tasks with specific output formats, domain-specific terminology, or unusual reasoning patterns.
How many tokens do examples consume? Depends on example length. Simple JSON examples: 50–100 tokens each. Code generation examples: 200–500 tokens each. With 5 examples: 250–2500 additional tokens per request. Budget accordingly.
Should I use the same examples for all users? Static examples for uniform tasks. Dynamic retrieval when users have varied needs (support agents with different product areas, coding assistants with different languages).
Can I combine few-shot with RAG? Yes — retrieve relevant examples from an example library the same way you retrieve relevant docs. This is "RAG for examples" and works well for diverse task types.
How do I know which examples are actually helping? A/B test: same eval dataset with/without examples, measure quality score difference. Remove examples one at a time and see which removal hurts quality most — those are your most valuable examples.