Chapter 4: Prompt Engineering and Inference Strategies

Exam focus

Primary domain: Prompt Engineering Techniques (14%). Secondary: Developing LLM Applications (24%).

Prompting

• Zero-shot
• Few-shot
• Chain-of-thought
• Role prompting
• System prompts
• Prompt templates
• Prompt injection attacks
• Jailbreak attempts

Decoding

• Temperature
• Top-k
• Top-p (nucleus sampling)
• Beam search
• Greedy decoding
• Deterministic vs stochastic inference
• Logits manipulation

Output Control

• Structured output
• Tool calling (conceptual)
• Function calling
• Output formatting constraints

Scope Bullet Explanations

• Zero-shot: Task execution with instructions only, no examples.
• Few-shot: Task execution with in-prompt examples to anchor behavior/format.
• Chain-of-thought: Prompt pattern that encourages stepwise reasoning.
• Role prompting: Assigns model persona/context to shape response style.
• System prompts: Highest-level instruction layer that defines policy and behavior boundaries.
• Prompt templates: Reusable structured prompt patterns for consistency.
• Prompt injection attacks: Malicious instructions in user/retrieved content attempting to override policy.
• Jailbreak attempts: Adversarial prompts trying to bypass model safety constraints.
• Temperature: Controls randomness of token sampling.
• Top-k: Limits sampling to the top k highest-probability tokens.
• Top-p (nucleus sampling): Samples from the smallest token set whose cumulative probability exceeds p.
• Beam search: Explores multiple candidate sequences to maximize likelihood.
• Greedy decoding: Chooses the highest-probability token at each step.
• Deterministic vs stochastic inference: Deterministic outputs are repeatable; stochastic outputs vary and can increase creativity.
• Logits manipulation: Adjusting token probabilities (penalties/biases) before sampling.
• Structured output: Constraining responses to schema formats (for example JSON).
• Tool calling (conceptual): Delegating parts of a task to external tools/services.
• Function calling: Structured model output that maps to explicit callable functions.
• Output formatting constraints: Rules for layout, field names, and validation to improve downstream reliability.

Chapter overview

Prompt engineering is interface design for model behavior. In production, quality depends on both prompt structure and decoding policy. This chapter covers practical prompt patterns, inference controls, and security hardening.

Learning objectives

• Apply zero-shot, few-shot, role prompting, and system-level instruction design.
• Select decoding strategies for determinism, creativity, and reliability.
• Enforce structured outputs for downstream tool integration.
• Identify prompt injection and jailbreak attack paths and mitigations.

4.1 Prompt design patterns

Zero-shot and few-shot

Zero-shot is faster to author and maintain. Few-shot improves formatting consistency and task grounding when the model under-specifies expected structure.

Role and system prompting

System instructions define persistent boundaries and style. Role prompts can localize behavior for specific turns or workflows. Treat both as policy hints, not security controls.

Prompt templates

Templates should include:

• task objective,
• input context contract,
• output schema,
• refusal or escalation rules,
• concise examples when needed. Stable templates reduce output variance and simplify testing.

Chain-of-thought usage

Use reasoning scaffolds when complex decomposition is required, but avoid unnecessary verbosity that burns context budget and can expose sensitive reasoning traces in some settings.

4.2 Decoding and generation control

Temperature

Lower values increase determinism; higher values increase diversity and risk of drift.

Top-k and top-p

Both constrain sampling space:

• top-k caps token candidates by rank,
• top-p caps by cumulative probability mass.

Greedy and beam search

Greedy is fast and deterministic. Beam search can improve sequence likelihood but may reduce diversity and increase compute.

Deterministic vs stochastic modes

• Deterministic mode for extraction, classification-style generation, and compliance workflows.
• Stochastic mode for ideation and creative drafting.

Logits manipulation

Frequency and presence penalties can reduce repetition. Bias controls can suppress or encourage specific tokens when used carefully.

4.3 Output control and tool integration

Structured outputs

Use strict schemas (JSON fields, type constraints, enum values). Structured generation lowers parser failures and simplifies monitoring.

Function and tool calling

Separate reasoning from action:

1. model decides whether tool call is needed,
2. tool executes deterministically,
3. model summarizes tool result with evidence.

Formatting constraints

Explicitly require units, field names, ordering, and null behavior for missing data.

4.4 Prompt security

Prompt injection

Untrusted text (user input or retrieved documents) may try to override policy. Defend through layered controls:

• content trust boundaries,
• tool permission checks,
• instruction hierarchy enforcement,
• response filtering and audit logs.

Jailbreak attempts

Attackers exploit ambiguity, role confusion, or hidden channels. Use robust refusal policies, adversarial testing, and incident review loops.

4.5 Practical prompt engineering workflow

1. Define one measurable task outcome.
2. Build minimal prompt template with schema.
3. Evaluate across representative and adversarial samples.
4. Tune decoding settings per task category.
5. Freeze prompt version and track changes like code.

4.6 Failure modes

• Overly long prompts that bury critical constraints.
• Mixing creative decoding settings into deterministic workflows.
• Relying on prompt text alone for security enforcement.
• Shipping templates without regression tests.

Chapter summary

Prompting and decoding are coupled controls. Production reliability comes from explicit contracts, tested templates, and layered safety design, not from clever phrasing alone.

Mini-lab: prompt and decoding benchmark

Goal: produce task-specific prompt baseline for one workflow.

1. Choose task: extraction, classification, or summarization.
2. Implement one zero-shot and one few-shot template.
3. Test each with three decoding configurations.
4. Measure accuracy, format compliance, and refusal correctness.
5. Identify injection-sensitive failure cases.
6. Document final recommended template and decode settings. Deliverable in Notion:

• Benchmark table and final prompt spec with version tag.

Review questions

1. When is few-shot prompting worth its added token cost?
2. How do top-p and temperature interact in practice?
3. Why is role prompting insufficient as a standalone security control?
4. What does a good output schema prevent operationally?
5. When should greedy decoding be preferred over sampling?
6. How can function calling reduce hallucinated tool behavior?
7. What are common signs of prompt injection success?
8. Why should prompts be versioned like code artifacts?
9. How do you separate instruction quality issues from model-limit issues?
10. Which evaluation set is required before shipping prompt updates?

Key terms

Zero-shot, few-shot, role prompt, system prompt, chain-of-thought, temperature, top-k, top-p, greedy decoding, beam search, logits bias, structured output, function calling, prompt injection, jailbreak.

Exam traps

• Assuming lower temperature automatically means higher factuality.
• Confusing prompt elegance with operational robustness.
• Treating prompt attacks as purely NLP issues instead of system security issues.
• Pretraining
• Fine-tuning
• Supervised fine-tuning (SFT)
• Instruction tuning
• Transfer learning
• Dataset curation
• Data preprocessing
• Data augmentation
• Synthetic data generation
• Curriculum learning
• Distributed training
• Data parallelism
• Model parallelism
• Mixed precision training
• Gradient accumulation
• Checkpointing
• Optimizers (Adam, AdamW)
• Learning rate scheduling
• Overfitting vs underfitting