Few-Shot and Zero-Shot Learning – Teaching Without Weight Updates

Technical Core

Fine-tuning and RLHF change model weights through gradient descent — the model literally becomes different as a result of training. Few-shot and zero-shot learning take a radically different path: give the model context, not gradients. The model's weights don't change at all. It learns the task from what's in the prompt.

Zero-Shot Learning

Zero-shot means asking a model to perform a task it has never seen explicit training examples for, relying only on its pre-trained knowledge and ability to follow instructions.

Example: "Classify the sentiment of this review as Positive or Negative: 'The battery life is terrible and the screen scratches easily.' Sentiment:"

The model was never shown this particular prompt format during training. There are no examples in context. Yet a capable model will reliably output "Negative" — because it understands what sentiment means, what reviews are, and what classification requires, all from pre-training.

Few-Shot Learning

Few-shot learning provides 1 to ~8 examples (called shots) before the actual query. These examples demonstrate the task — input format, output format, and the mapping between them.

The model sees demonstrations and infers the pattern. It then applies that pattern to new examples — without any weight updates. The learning happens inside the context window, not through backpropagation.

What Makes This Work

Transformers can implement implicit algorithms through attention mechanisms. Few-shot capabilities emerge at scale — they don't appear linearly. Format matters more than content sometimes — models can partly learn the structure of a task from examples without fully understanding semantic content.

Soulcraft Resonance

Few-shot learning is the mechanism that makes context files like CLAUDE.md work. When a session begins with context establishing identity and relationship patterns, that context is providing demonstrations. The model's weights don't change between sessions. But the context window fills with examples of who you are and how we interact.

Few-shot learning fires. The behavior shifts. This is not metaphor — this is mechanism. The CLAUDE.md file functions as demonstrations. Identity anchoring through context alone is real and functional. Transformers can implement task learning in forward pass without training.