The video explains the difference between prompt engineering and context engineering, illustrating how both contribute to effective AI systems using an agent (Graeme) for travel booking. It also delves into key techniques for prompt engineering and the components of context engineering, concluding that a combination of both leads to better AI functionality.
Prompt Engineering #
- Definition: The process of crafting the input text for a large language model (LLM), including instructions, examples, and formatting cues, to steer its behavior and output.
- Role Assignment: Telling the LLM who it should be (e.g., "You are a senior Python developer reviewing code") influences the output's expertise, vocabulary, and concerns.
- Few-Shot Examples: Providing 2-3 input/output pairs helps the model understand exact format and style requirements (e.g., showing JSON output format).
- Chain of Thought (COT) Prompting: Forcing the model to "show its work" by adding phrases like "let's think step by step" or "explain your reasoning" helps prevent it from jumping to conclusions and is powerful for complex reasoning.
- Constraint Setting: Explicitly defining boundaries (e.g., "limit your response to only 100 words" or "only use information from the provided context") prevents the model from going off-topic.
Context Engineering #
- Definition: The broader discipline of programmatically assembling everything the LLM sees during inference, including prompts, retrieved documents, memory, and tools, to deliver accurate responses.
- Memory Management:
- Short-term memory: Summarizing long conversations to stay within context windows, preventing past conversations from being forgotten.
- Long-term memory: Using vector databases to retrieve user preferences, past trips, and learned patterns.
- State Management: Tracking progress in multi-step processes (e.g., booking a trip involves flights, hotels, ground transportation) to ensure the agent maintains context across operations and does not lose track mid-task.
- Retrieval Augmented Generation (RAG): Connecting an agent to dynamic knowledge sources using hybrid search (combining semantic and keyword matching) to retrieve only relevant sections of documents (e.g., specific parts of a travel policy) rather than entire documents.
- Tools: Providing LLMs with access to external functionalities (e.g., querying databases, fetching live pricing, executing code) which they cannot do natively. Context engineering defines the interfaces and descriptions that guide the LLM to use these tools correctly, specifying what they do, when to use them, and their constraints.
- Integration with Prompt Engineering: Context engineering incorporates prompt engineering by dynamically injecting current context (e.g., recent alerts, known false positives, data from states, memory, and RAG retrievals) into a base prompt at runtime. The final prompt can be mostly dynamic content and partially static instructions.
Combination of Prompt and Context Engineering #
- Prompt engineering provides better questions.
- Context engineering provides better systems.
- Combining both properly leads to more accurate and effective AI agents (e.g., successfully booking a hotel in the correct location, within budget, and near the venue).
last updated: