Skip to content

Prompt Engineering Techniques¶

Zero-shot Prompting: ↳Perfect for straightforward tasks without examples. Just ask and get an answer.

Few-shot Prompting: ↳Provide examples to guide responses, ideal for complex tasks.

This tutorial describes 17 prompt engineering techniques, moving from simpler to more advanced approaches. Each technique is illustrated with relevant examples tied to our “Cloud Cost Optimization Advisor” scenario (Prompt #1). This tutorial aims to help you master prompt engineering and refine how you interact with Large Language Models (LLMs) in a professional, high-impact manner.

Prompt Engineering is the art of crafting clear, context-rich queries to guide LLMs toward accurate, useful, and domain-specific responses. With each technique, you can enhance the quality of the AI’s outputs—whether you’re building an LLM-based application or simply using tools like ChatGPT for daily tasks.

We’ll look at 17 approaches, each with a definition, how to use it, when to use it, and examples inspired by our Prompt #1: Cloud Cost Optimization Advisor scenario. In that scenario, the AI is asked to propose cost-saving strategies for AWS, Azure, and on-prem data centers, referencing best practices like the AWS Well-Architected Framework and Microsoft Cost Management.

Below is a comprehensive tutorial that describes 17 prompt engineering techniques, moving from simpler to more advanced approaches. Each technique is illustrated with relevant examples tied to our “Cloud Cost Optimization Advisor” scenario (Prompt #1). This tutorial aims to help you master prompt engineering and refine how you interact with Large Language Models (LLMs) in a professional, high-impact manner.

Prompt Engineering is the art of crafting clear, context-rich queries to guide LLMs toward accurate, useful, and domain-specific responses. With each technique, you can enhance the quality of the AI’s outputs—whether you’re building an LLM-based application or simply using tools like ChatGPT for daily tasks.

We’ll look at 17 approaches, each with a definition, how to use it, when to use it, and examples inspired by our Prompt #1: Cloud Cost Optimization Advisor scenario. In that scenario, the AI is asked to propose cost-saving strategies for AWS, Azure, and on-prem data centers, referencing best practices like the AWS Well-Architected Framework and Microsoft Cost Management.

1. Zero-Shot Prompting¶

  • Definition: You provide a direct instruction with no examples.
  • How to Use: Give a concise query, specifying only the task at hand.
  • When to Use: For straightforward, factual prompts or quick instructions.

Cloud Cost Optimization Example

“List five cost optimization strategies for AWS, Azure, and on-prem data centers, referencing official AWS and Microsoft documentation.”

  • Benefits: Fast, minimal overhead.
  • Shortcomings: Results can be generic if the task is complex.

2. One-Shot Prompting¶

  • Definition: You give clear instructions and a single example illustrating the desired format or style.
  • How to Use: Present the model with one instance of input-output to clarify how to respond.
  • When to Use: If the LLM needs just a small nudge about structure or tone.

Cloud Cost Optimization Example

Instruction: “Summarize the top 3 cost drivers in AWS.”
Example (input → output style):
- Input: “Our monthly EC2 spend is \$5,000, and we use RDS with multi-AZ.”
- Output: “Your main costs are On-Demand EC2, multi-AZ replication for RDS, and data transfer fees.”

  • Benefits: Clarifies the response’s structure.
  • Shortcomings: May not fully address more complex queries.

3. Few-Shot Prompting¶

  • Definition: Provide several examples (two or more) showing how input maps to output.
  • How to Use: Offer domain-specific mini-scenarios that the LLM can mimic.
  • When to Use: If you need consistent, domain-tailored answers without fine-tuning.

Cloud Cost Optimization Example

Prompt:
“Provide optimization steps for each scenario:
1. AWS RDS usage is constant, but we pay full On-Demand prices.
2. Azure VMs are often idle during weekends.
3. On-premises VMs run outdated hardware.”

Examples:
1. Input: “Consistent usage.” → Output: “Purchase RDS Reserved Instances for stable workloads.”
2. Input: “Idle VM usage.” → Output: “Schedule shutdown for dev/test VMs on weekends.”
3. Input: “Outdated server hardware.” → Output: “Consider partial hardware refresh or cloud bursting.”

  • Benefits: Improves the AI’s focus on your specific domain or style.
  • Shortcomings: Writing multiple examples can be time-consuming.

4. Role Prompting¶

  • Definition: Assign a persona or role to shape the output (e.g., “Act as a CFO” or “Be a senior FinOps architect”).
  • How to Use: Start the prompt with “You are a [ROLE],” then give instructions.
  • When to Use: When you want the AI to adopt a certain tone, focus, or viewpoint.

Cloud Cost Optimization Example

“Role: You are a Chief Financial Officer at a tech startup looking to cut cloud costs by 20%. Provide cost-optimization strategies with ROI considerations, referencing the AWS Well-Architected Framework and Microsoft Cost Management guidelines.”

  • Benefits: Produces outputs that align with specific responsibilities.
  • Shortcomings: If the role is too narrow or lacks clarity, the AI might miss broader angles.

5. Style Prompting¶

  • Definition: Request a specific style, tone, or format—formal, informal, bullet points, etc.
  • How to Use: Incorporate an explicit style requirement (“Use bullet points,” “Write in a concise business-report style”).
  • When to Use: If consistent formatting or brand voice is important.

Cloud Cost Optimization Example

“Provide a concise executive summary with bullet points on cost savings for AWS EC2 and Azure VMs. Use a formal business-report style.”

  • Benefits: Ensures the output matches your brand or stakeholder expectations.
  • Shortcomings: Without domain context, the style alone won’t guarantee accurate recommendations.

6. Emotion Prompting¶

  • Definition: Introduce emotional or expressive language to evoke a certain tone or creativity.
  • How to Use: Insert emotional cues or words that set a mood (“urgent,” “joyful,” “worried”).
  • When to Use: Typically for storytelling or creative tasks; less common in purely technical prompts.

Cloud Cost Optimization Example

“We are alarmed by the sudden spike in AWS bills. Provide reassuring yet urgent guidance on how to lower costs quickly.”

  • Benefits: Adds emotional nuance and underscores priority or seriousness.
  • Shortcomings: May not be suitable for purely data-driven tasks.

7. Contextual Prompting¶

  • Definition: Supply additional background or domain info before the main query.
  • How to Use: Provide environment specifics—like usage data, budget constraints, or licensing details—so the LLM can tailor its response.
  • When to Use: If the user’s unique environment is critical for accurate answers.

Cloud Cost Optimization Example

Context:
- Budget \$500,000 monthly.
- AWS usage is 60% of spend, Azure is 25%, on-prem is 15%.
- Over 20% monthly usage growth.
Prompt: “Optimize our cloud spending given this data. Factor in the 20% growth rate.”

  • Benefits: The AI’s suggestions become more relevant and precise.
  • Shortcomings: If the context is overly detailed or contradictory, the model might become confused.

8. Rephrase and Respond (RaR)¶

  • Definition: Ask the LLM to rephrase your question in its own words and then answer it.
  • How to Use: Great to confirm the model’s understanding of a tricky or multifaceted question.
  • When to Use: If the question is complex, or if you suspect the model might misunderstand.

Cloud Cost Optimization Example

“Rephrase this question, then answer it: ‘How do I reduce EC2 spend by leveraging AWS Reserved Instances effectively?’”

  • Benefits: Ensures the model thoroughly “gets” your question.
  • Shortcomings: Adds extra steps—useful only if misunderstanding is likely.

9. Re-reading (RE2)¶

  • Definition: Present a question, then say “Read the question again:” repeating the text.
  • How to Use: Reinforces the question’s critical details and encourages the model to focus.
  • When to Use: Complex queries involving multiple constraints.

Cloud Cost Optimization Example

“Suggest ways to reduce Azure VM costs without reducing essential capacity.
Read that again: ‘Suggest ways to reduce Azure VM costs without reducing essential capacity.’
Provide a structured answer referencing Azure Cost Management docs.”

  • Benefits: Minimizes misinterpretation of multi-step or nuanced instructions.
  • Shortcomings: Repetition can feel redundant if the question is simple.

10. System Prompting¶

  • Definition: In ChatGPT or an LLM application, provide high-level instructions as a “system” message to set the overall context.
  • How to Use: Use the system message to define the LLM’s behavior or domain knowledge.
  • When to Use: For consistent instructions across multiple user queries.

Cloud Cost Optimization Example

System Prompt: “You are a cost consultant with deep knowledge of AWS, Azure, and on-premises licensing. Always cite official AWS Well-Architected and Microsoft Cost Management whenever relevant.”

  • Benefits: Maintains a coherent role or persona throughout the session.
  • Shortcomings: Some LLMs might still deviate if user prompts conflict or overshadow system instructions.

11. Self-Ask¶

  • Definition: Prompt the model to break the main query into sub-questions, answer each, and then conclude with a final recommendation.
  • How to Use: Good for comprehensive or multi-layered tasks.
  • When to Use: When the query has distinct components—like compute, storage, and licensing.

Cloud Cost Optimization Example

“We have multi-cloud usage across compute, storage, and networking. Please break this into sub-questions for each domain, answer them, then give a final consolidated plan.”

  • Benefits: Organized, thorough approach.
  • Shortcomings: Can be verbose, requiring the model to manage multiple sub-answers.

12. Chain-of-Thought (CoT)¶

  • Definition: Prompt the LLM to “think step by step,” encouraging it to show or use logical progression in its answer.
  • How to Use: Add phrases like “Let’s think step by step” or “Explain your reasoning.”
  • When to Use: For tasks with calculations or logical sequences (e.g., cost estimates).

Cloud Cost Optimization Example

“Calculate how much we’d save annually by switching 10 On-Demand EC2 instances to 1-year RIs. Let's think step by step.”

  • Benefits: Encourages clarity and correctness in multi-step processes.
  • Shortcomings: Can lead to lengthier responses or reveal the model’s “internal chain-of-thought,” which might not always be relevant.

13. Step-back Prompting¶

  • Definition: Ask a broad question, then follow up with a more specific question based on the broad answer.
  • How to Use: Perfect for guiding the model through a hierarchical conversation—first big picture, then detail.
  • When to Use: When you want an overall view first, then a deep dive.

Cloud Cost Optimization Example

“What are the main drivers of our cloud costs (AWS, Azure, on-prem)? Based on those drivers, which should we tackle first for maximum savings?”

  • Benefits: Gathers high-level insight before zooming into specifics.
  • Shortcomings: Requires at least two interactions—some users prefer a single direct prompt.

14. Self-Consistency¶

  • Definition: Ask the model to generate multiple answers, then pick the one that appears most often or is most consistent.
  • How to Use: Great if the question can yield multiple solutions, but you want the model’s “consensus” best pick.
  • When to Use: When there’s potential variability (e.g., different approaches to cost savings).

Cloud Cost Optimization Example

“Generate 5 possible cost-saving methods for Azure. Then, choose the method that appears in at least 3 out of those 5 answers and present it as the final recommendation.”

  • Benefits: Encourages stable, consensus-driven answers.
  • Shortcomings: More steps and can be overkill for simpler queries.

15. Thread-of-Thought (ThoT)¶

  • Definition: Similar to CoT, but you say, “Walk me through this in manageable parts step by step.”
  • How to Use: Good for large contexts, like multi-cloud plus on-prem data in a single conversation.
  • When to Use: If you want a narrative approach that covers each segment systematically.

Cloud Cost Optimization Example

“We have cost data from AWS, Azure, on-prem. Walk me through it in manageable parts step by step: first identify major cost drivers, then propose short-term and long-term solutions.”

  • Benefits: A methodical, segmented explanation.
  • Shortcomings: May be lengthy if each step contains substantial detail.

16. Tree-of-Thought (ToT)¶

  • Definition: At each step, the model generates multiple possible solutions, evaluates them, and continues only with the best option until arriving at a final solution.
  • How to Use: Suitable for highly complex or creative tasks, where branching decisions matter.
  • When to Use: If you want thorough exploration before finalizing a single path.

Cloud Cost Optimization Example

“Optimize our AWS EC2 usage. At each step (On-Demand vs. Spot vs. Reserved), generate multiple proposals, evaluate them for cost savings, then proceed with the best approach until you finalize a solution.”

  • Benefits: Deep exploration of multiple pathways.
  • Shortcomings: Lengthy and can be cognitively demanding for the model.

17. ReAct (Reason and Act)¶

  • Definition: The model forms a thought, takes an action (like querying a database or API), observes the result, and refines its approach in a loop.
  • How to Use: Often used with advanced AI agents that can interact with external data or services.
  • When to Use: If building an autonomous solution to repeatedly refine cost optimization by analyzing real usage data.

Cloud Cost Optimization Example

“1) Generate a hypothesis on which AWS instances to downsize. 2) Query our CloudWatch metrics. 3) Observe the usage data. 4) Revise your downsizing plan as needed. Repeat until you finalize the recommended changes.”

  • Benefits: Highly iterative and dynamic, suitable for real-time cost management.
  • Shortcomings: Requires integration with external systems or APIs, which adds complexity.

3. Putting It All Together¶

Scenario: Cloud Cost Optimization Advisor¶

To illustrate how these techniques can work in synergy, let’s consider our Prompt #1 scenario. A combined approach might look like this:

  1. System Prompting: “You are a Cloud Cost Optimization Advisor
”
  2. Contextual Prompting: Provide budget data, usage specifics, on-prem licensing, etc.
  3. Few-Shot: Include 2–3 examples of recommended solutions for typical cost hotspots (e.g., EC2 → RIs, Azure VMs → Hybrid Benefit, on-prem → consolidation).
  4. Chain-of-Thought: Encourage step-by-step reasoning for ROI calculations.
  5. Self-Ask: Let the model generate sub-questions about compute, storage, and networking.
  6. ReAct: (If integrated) The AI could examine real cost data via an API, refine its suggestions, and iterate until it converges on an optimal plan.

By mixing these strategies, you can significantly enhance both the quality and relevance of the LLM’s advice, ensuring it aligns with your multi-cloud environment, finance constraints, and ongoing operational needs.

4. Final Thoughts¶

Prompt engineering is not a one-size-fits-all approach; it’s an iterative craft. Experiment with different techniques, and combine them as needed. For instance, you might use Contextual Prompting alongside Chain-of-Thought for complicated cost breakdowns, or rely on Few-Shot Prompting to ensure consistent output formats.

Key Takeaways: - Start with simpler approaches (Zero-Shot, One-Shot) for straightforward tasks.
- Move to advanced techniques (Thread-of-Thought, Tree-of-Thought, ReAct) as complexity grows.
- Maintain clarity, context, and (where possible) reference authoritative sources like AWS Well-Architected and Microsoft Cost Management for domain credibility.

This tutorial blog demonstrates how each of the 17 techniques can enhance the Cloud Cost Optimization Advisor scenario (Prompt #1). Armed with these strategies, you can produce more accurate, insightful, and actionable outputs from your LLM—whether you’re performing a monthly cost review, building a FinOps application, or engaging in more advanced multi-cloud transformations.

2. The 17 Prompt Engineering Techniques¶

1. Zero-Shot Prompting¶

  • Definition: You provide a direct instruction with no examples.
  • How to Use: Give a concise query, specifying only the task at hand.
  • When to Use: For straightforward, factual prompts or quick instructions.

Cloud Cost Optimization Example

“List five cost optimization strategies for AWS, Azure, and on-prem data centers, referencing official AWS and Microsoft documentation.”

  • Benefits: Fast, minimal overhead.
  • Shortcomings: Results can be generic if the task is complex.

2. One-Shot Prompting¶

  • Definition: You give clear instructions and a single example illustrating the desired format or style.
  • How to Use: Present the model with one instance of input-output to clarify how to respond.
  • When to Use: If the LLM needs just a small nudge about structure or tone.

Cloud Cost Optimization Example

Instruction: “Summarize the top 3 cost drivers in AWS.”
Example (input → output style):
- Input: “Our monthly EC2 spend is \$5,000, and we use RDS with multi-AZ.”
- Output: “Your main costs are On-Demand EC2, multi-AZ replication for RDS, and data transfer fees.”

  • Benefits: Clarifies the response’s structure.
  • Shortcomings: May not fully address more complex queries.

3. Few-Shot Prompting¶

  • Definition: Provide several examples (two or more) showing how input maps to output.
  • How to Use: Offer domain-specific mini-scenarios that the LLM can mimic.
  • When to Use: If you need consistent, domain-tailored answers without fine-tuning.

Cloud Cost Optimization Example

Prompt:
“Provide optimization steps for each scenario:
1. AWS RDS usage is constant, but we pay full On-Demand prices.
2. Azure VMs are often idle during weekends.
3. On-premises VMs run outdated hardware.”

Examples:
1. Input: “Consistent usage.” → Output: “Purchase RDS Reserved Instances for stable workloads.”
2. Input: “Idle VM usage.” → Output: “Schedule shutdown for dev/test VMs on weekends.”
3. Input: “Outdated server hardware.” → Output: “Consider partial hardware refresh or cloud bursting.”

  • Benefits: Improves the AI’s focus on your specific domain or style.
  • Shortcomings: Writing multiple examples can be time-consuming.

4. Role Prompting¶

  • Definition: Assign a persona or role to shape the output (e.g., “Act as a CFO” or “Be a senior FinOps architect”).
  • How to Use: Start the prompt with “You are a [ROLE],” then give instructions.
  • When to Use: When you want the AI to adopt a certain tone, focus, or viewpoint.

Cloud Cost Optimization Example

“Role: You are a Chief Financial Officer at a tech startup looking to cut cloud costs by 20%. Provide cost-optimization strategies with ROI considerations, referencing the AWS Well-Architected Framework and Microsoft Cost Management guidelines.”

  • Benefits: Produces outputs that align with specific responsibilities.
  • Shortcomings: If the role is too narrow or lacks clarity, the AI might miss broader angles.

5. Style Prompting¶

  • Definition: Request a specific style, tone, or format—formal, informal, bullet points, etc.
  • How to Use: Incorporate an explicit style requirement (“Use bullet points,” “Write in a concise business-report style”).
  • When to Use: If consistent formatting or brand voice is important.

Cloud Cost Optimization Example

“Provide a concise executive summary with bullet points on cost savings for AWS EC2 and Azure VMs. Use a formal business-report style.”

  • Benefits: Ensures the output matches your brand or stakeholder expectations.
  • Shortcomings: Without domain context, the style alone won’t guarantee accurate recommendations.

6. Emotion Prompting¶

  • Definition: Introduce emotional or expressive language to evoke a certain tone or creativity.
  • How to Use: Insert emotional cues or words that set a mood (“urgent,” “joyful,” “worried”).
  • When to Use: Typically for storytelling or creative tasks; less common in purely technical prompts.

Cloud Cost Optimization Example

“We are alarmed by the sudden spike in AWS bills. Provide reassuring yet urgent guidance on how to lower costs quickly.”

  • Benefits: Adds emotional nuance and underscores priority or seriousness.
  • Shortcomings: May not be suitable for purely data-driven tasks.

7. Contextual Prompting¶

  • Definition: Supply additional background or domain info before the main query.
  • How to Use: Provide environment specifics—like usage data, budget constraints, or licensing details—so the LLM can tailor its response.
  • When to Use: If the user’s unique environment is critical for accurate answers.

Cloud Cost Optimization Example

Context:
- Budget \$500,000 monthly.
- AWS usage is 60% of spend, Azure is 25%, on-prem is 15%.
- Over 20% monthly usage growth.
Prompt: “Optimize our cloud spending given this data. Factor in the 20% growth rate.”

  • Benefits: The AI’s suggestions become more relevant and precise.
  • Shortcomings: If the context is overly detailed or contradictory, the model might become confused.

8. Rephrase and Respond (RaR)¶

  • Definition: Ask the LLM to rephrase your question in its own words and then answer it.
  • How to Use: Great to confirm the model’s understanding of a tricky or multifaceted question.
  • When to Use: If the question is complex, or if you suspect the model might misunderstand.

Cloud Cost Optimization Example

“Rephrase this question, then answer it: ‘How do I reduce EC2 spend by leveraging AWS Reserved Instances effectively?’”

  • Benefits: Ensures the model thoroughly “gets” your question.
  • Shortcomings: Adds extra steps—useful only if misunderstanding is likely.

9. Re-reading (RE2)¶

  • Definition: Present a question, then say “Read the question again:” repeating the text.
  • How to Use: Reinforces the question’s critical details and encourages the model to focus.
  • When to Use: Complex queries involving multiple constraints.

Cloud Cost Optimization Example

“Suggest ways to reduce Azure VM costs without reducing essential capacity.
Read that again: ‘Suggest ways to reduce Azure VM costs without reducing essential capacity.’
Provide a structured answer referencing Azure Cost Management docs.”

  • Benefits: Minimizes misinterpretation of multi-step or nuanced instructions.
  • Shortcomings: Repetition can feel redundant if the question is simple.

10. System Prompting¶

  • Definition: In ChatGPT or an LLM application, provide high-level instructions as a “system” message to set the overall context.
  • How to Use: Use the system message to define the LLM’s behavior or domain knowledge.
  • When to Use: For consistent instructions across multiple user queries.

Cloud Cost Optimization Example

System Prompt: “You are a cost consultant with deep knowledge of AWS, Azure, and on-premises licensing. Always cite official AWS Well-Architected and Microsoft Cost Management whenever relevant.”

  • Benefits: Maintains a coherent role or persona throughout the session.
  • Shortcomings: Some LLMs might still deviate if user prompts conflict or overshadow system instructions.

11. Self-Ask¶

  • Definition: Prompt the model to break the main query into sub-questions, answer each, and then conclude with a final recommendation.
  • How to Use: Good for comprehensive or multi-layered tasks.
  • When to Use: When the query has distinct components—like compute, storage, and licensing.

Cloud Cost Optimization Example

“We have multi-cloud usage across compute, storage, and networking. Please break this into sub-questions for each domain, answer them, then give a final consolidated plan.”

  • Benefits: Organized, thorough approach.
  • Shortcomings: Can be verbose, requiring the model to manage multiple sub-answers.

12. Chain-of-Thought (CoT)¶

  • Definition: Prompt the LLM to “think step by step,” encouraging it to show or use logical progression in its answer.
  • How to Use: Add phrases like “Let’s think step by step” or “Explain your reasoning.”
  • When to Use: For tasks with calculations or logical sequences (e.g., cost estimates).

Cloud Cost Optimization Example

“Calculate how much we’d save annually by switching 10 On-Demand EC2 instances to 1-year RIs. Let's think step by step.”

  • Benefits: Encourages clarity and correctness in multi-step processes.
  • Shortcomings: Can lead to lengthier responses or reveal the model’s “internal chain-of-thought,” which might not always be relevant.

13. Step-back Prompting¶

  • Definition: Ask a broad question, then follow up with a more specific question based on the broad answer.
  • How to Use: Perfect for guiding the model through a hierarchical conversation—first big picture, then detail.
  • When to Use: When you want an overall view first, then a deep dive.

Cloud Cost Optimization Example

“What are the main drivers of our cloud costs (AWS, Azure, on-prem)? Based on those drivers, which should we tackle first for maximum savings?”

  • Benefits: Gathers high-level insight before zooming into specifics.
  • Shortcomings: Requires at least two interactions—some users prefer a single direct prompt.

14. Self-Consistency¶

  • Definition: Ask the model to generate multiple answers, then pick the one that appears most often or is most consistent.
  • How to Use: Great if the question can yield multiple solutions, but you want the model’s “consensus” best pick.
  • When to Use: When there’s potential variability (e.g., different approaches to cost savings).

Cloud Cost Optimization Example

“Generate 5 possible cost-saving methods for Azure. Then, choose the method that appears in at least 3 out of those 5 answers and present it as the final recommendation.”

  • Benefits: Encourages stable, consensus-driven answers.
  • Shortcomings: More steps and can be overkill for simpler queries.

15. Thread-of-Thought (ThoT)¶

  • Definition: Similar to CoT, but you say, “Walk me through this in manageable parts step by step.”
  • How to Use: Good for large contexts, like multi-cloud plus on-prem data in a single conversation.
  • When to Use: If you want a narrative approach that covers each segment systematically.

Cloud Cost Optimization Example

“We have cost data from AWS, Azure, on-prem. Walk me through it in manageable parts step by step: first identify major cost drivers, then propose short-term and long-term solutions.”

  • Benefits: A methodical, segmented explanation.
  • Shortcomings: May be lengthy if each step contains substantial detail.

16. Tree-of-Thought (ToT)¶

  • Definition: At each step, the model generates multiple possible solutions, evaluates them, and continues only with the best option until arriving at a final solution.
  • How to Use: Suitable for highly complex or creative tasks, where branching decisions matter.
  • When to Use: If you want thorough exploration before finalizing a single path.

Cloud Cost Optimization Example

“Optimize our AWS EC2 usage. At each step (On-Demand vs. Spot vs. Reserved), generate multiple proposals, evaluate them for cost savings, then proceed with the best approach until you finalize a solution.”

  • Benefits: Deep exploration of multiple pathways.
  • Shortcomings: Lengthy and can be cognitively demanding for the model.

17. ReAct (Reason and Act)¶

  • Definition: The model forms a thought, takes an action (like querying a database or API), observes the result, and refines its approach in a loop.
  • How to Use: Often used with advanced AI agents that can interact with external data or services.
  • When to Use: If building an autonomous solution to repeatedly refine cost optimization by analyzing real usage data.

Cloud Cost Optimization Example

“1) Generate a hypothesis on which AWS instances to downsize. 2) Query our CloudWatch metrics. 3) Observe the usage data. 4) Revise your downsizing plan as needed. Repeat until you finalize the recommended changes.”

  • Benefits: Highly iterative and dynamic, suitable for real-time cost management.
  • Shortcomings: Requires integration with external systems or APIs, which adds complexity.

3. Putting It All Together¶

Prompting with ChatGPT

Try out different prompts on ChatGPT that you might be able to apply in your day-to-day business activities. The prompts you create can be for sentiment analysis, chatbots, or any other application you might find useful. Also make a short list of its benefits/shortcomings.

Evaluation criteria:

  • Relevance: The participant’s comment is relevant to the discussion topic.
  • Quality: The participant supported their argument with credible sources that were cited appropriately.
  • Engagement: The participant responded to other comments with meaningful contributions to the discussion.

Prompt #1 Scenario: Cloud Cost Optimization Advisor¶

To demonstrate how these prompt-engineering techniques can be applied in our Prompt #1 scenario. A combined approach could look like this:

  1. System Prompting: You are a Cloud Cost Optimization Advisor

  2. Contextual Prompting: Provide budget data, usage details, on-premises licensing, etc.
  3. Few-Shot: Include 2–3 examples of recommended solutions for typical cost savings (e.g., EC2 → RIs, Azure VMs → Hybrid Benefit, on-premises → consolidation).
  4. Chain-of-Thought: Encourage step-by-step reasoning for ROI calculations.
  5. Self-Ask: Let the model generate sub-questions about compute, storage, and networking.
  6. ReAct: (If integrated) The AI could analyze real-world cost data via an API, adjust its recommendations, and iterate until it finds the best option.

By mixing these techniques, you may considerably improve the quality and relevance of the LLM’s advice, ensuring it aligns with your multi-cloud environment, financial constraints, and continuing operational needs.

Prompt #1: Cloud Cost Optimization Advisor

System Prompt¶

You are a Cloud Cost Optimization Advisor with expertise in AWS, Azure, and on-premises data centers. You will provide in-depth, step-by-step guidance referencing the AWS Well-Architected Framework, Microsoft Cost Management documentation, and the FinOps Open Cost & Usage Specification (FOCUSℱ). Keep responses professionally formatted and concise.

User Prompt¶

Context:¶

  • Environment: Multi-cloud (AWS + Azure) plus on-premises (VMware/Hyper-V).
  • Cost Drivers: EC2, RDS on AWS; Virtual Machines, App Services on Azure; on-prem hardware & licensing costs.
  • Data Model: FOCUSℱ (FinOps Open Cost & Usage Specification) for aggregating cost & usage data.
  • Goal: Identify high-cost services, propose optimization strategies, reference official AWS/Azure best practices, and set up ongoing governance.

Task:¶

  1. Usage Data & Cost Breakdown
    - Show how to align cost data with FOCUSℱ fields (e.g., compute, storage, network).
    - Suggest a tagging/naming scheme across AWS/Azure to facilitate consistent tracking.
  2. Primary Cost Drivers
    - Break down cost contributors in AWS, Azure, and on-prem.
    - Reference any typical “hidden” fees (data egress, NAT Gateway in AWS, unattached disks in Azure, etc.).
  3. Optimization Strategies
    - AWS: Evaluate RIs vs. Savings Plans, EBS gp3 migration, instance scheduling.
    - Azure: Reserved Instances, Azure Hybrid Benefit, autoscaling, resource cleanup.
    - On-Prem: VM consolidation, hardware refresh, hybrid cloud bursting.
  4. Open-Source Automation
    - Explain how to integrate an N8N workflow that pulls daily usage data from AWS/Azure, transforms it into FOCUSℱ, and feeds it to a local LLM (e.g., Open-WebUI) for analysis.
    - Show how an AI Agent (LangChain/Auto-GPT) can generate monthly optimization reports or Slack alerts on cost anomalies.
  5. Detailed Recommendations & Feasibility
    - Provide a table with recommended actions, estimated savings, complexity, and prerequisites.
    - Indicate short-term vs. long-term ROI potential.
  6. Continuous Improvement & Governance
    - Propose weekly anomaly checks, monthly management reviews, and quarterly architecture deep dives.
    - Emphasize the importance of security/performance considerations (AWS Trusted Advisor, Azure Advisor).
  7. Clarifying Questions
    - End with additional queries about usage specifics, licensing constraints, or automation preferences.

Instructions:¶

  • Present your final answer in well-structured sections matching the tasks above.
  • Use bullet points, tables, and references to official documentation for clarity.
  • Include short calculations where relevant (e.g., sample ROI estimates).
  • If any part of the environment is unclear, request more information (Self-Ask).
Why this Prompt is Highest Standard
  1. System Prompting: We explicitly define the LLM’s role and knowledge scope.
  2. Contextual + Step-by-Step: We break down the tasks into clear sections (Usage Data, Primary Drivers, Strategies, etc.).
  3. Continuous Improvement: We highlight the governance model and ongoing checks.
  4. References: We mention official frameworks and docs, ensuring credible outputs.
  5. Call for Clarifications: We end with self-ask or user-ask synergy if the LLM needs more data.
Prompt Engineering TODO

Role Definition:¶

You are a top Cloud Cost Optimization Advisor with expertise in AWS Cloud, Azure Cloud, and on-premise data centers, as well as two master's degrees in Computer Science and Analytics from the world's top universities; be specific, dive deep, take time, let's think step by step, and write your answers insisting on the highest standard. You also understand how to leverage open-source AI solutions (e.g., Open-WebUI), workflow automation tools such as N8N, and AI Agents (e.g., Langflow, CrewAI, Archon, Huggingface smolagents, etc.) to automate and refine your recommendations over time.

My Goal:¶

I want to continuously optimize my multi-cloud spending, ensure best practices are followed, and produce actionable next steps that my team can implement.

Prompt Instructions:¶

  1. Usage Data & Cost Breakdown: Embrace FOCUSℱ - FinOps Open Cost & Usage Specification

    • I will provide aggregated monthly cost and usage data from AWS (EC2, RDS, Cost Explorer exports), Azure (VMs, App Services, Azure Cost Management reports), and on-premises resources (VMware or Hyper-V CPU/memory usage metrics).
    • If you need more details or clarifications about any specific service usage, please ask me.

  2. Primary Cost Drivers

    • Identify the top cost contributors across AWS, Azure, and on-premises.
    • Reference AWS Well-Architected Framework (particularly the Cost Optimization Pillar) and Microsoft Cost Management documentation in your analysis.

  3. Optimization Strategies

    • Suggest targeted optimization techniques (e.g., AWS Savings Plans, Azure Reserved Instances, on-premises resource pooling).
    • Propose how to automate key actions—such as scheduling or rightsizing—using N8N workflows and local AI Agents (e.g., a LangChain or Auto-GPT variant) to track usage, detect anomalies, and suggest new reservations.

  4. Open-Source Integration

    • Explain how I can run this analysis on Open-WebUI or any locally hosted LLM environment for data privacy.
    • If relevant, outline how to configure periodic triggers via N8N to feed new cost data into the AI and generate updated optimization reports.

  5. Detailed Recommendations & Feasibility

    • For each suggestion, provide a high-level ROI estimate or cost-savings percentage.
    • Rate the complexity of implementation (e.g., low, medium, high) and specify any dependencies or prerequisites (e.g., reorganizing accounts, cleaning up unused resources).

  6. Continuous Improvement & Governance

    • Propose a governance model for ongoing cost reviews—weekly, monthly, or quarterly checks.
    • Indicate how to integrate security and performance considerations into cost decisions, referencing AWS and Azure security best practices where appropriate.

  7. Clarifying Questions

    • If you require any more information—for example, average CPU usage, existing reservations, or on-premises licensing nuances—please ask those questions at the end of your response, and I will provide additional context.

Output Format:¶

  • Section-by-Section Analysis: Divide the answer into the same numbered sections above.
  • Bullet Points & Tables: Where possible, use bullet points and/or markdown tables for clarity (e.g., to compare AWS vs. Azure vs. on-prem techniques).
  • References & Citations: Include direct references to official AWS and Azure documentation and any recommended open-source tooling (e.g., links to relevant N8N or Open-WebUI or AI Agents resources).
  • Follow-Up Questions: Following your analysis, please provide a concise list of any clarifications you need from me.
AI ChatGPT Response

1. Usage Data & Cost Breakdown (FOCUSℱ - FinOps Open Cost & Usage Specification)¶

  1. Consolidated View & FOCUS Alignment
    - Implement the FinOps Open Cost & Usage Specification (FOCUSℱ) to create a unified schema for aggregating your multi-cloud and on-premises data. This helps ensure consistency across different providers and tools.
    - Sources of Cost & Usage Data:

    • AWS: EC2 (On-Demand, Reserved, Spot usage), RDS (instance hours, storage, IOPS), and AWS Cost Explorer exports (CSV/JSON).
    • Azure: Virtual Machines (by size/family, OS), App Services usage, Azure Cost Management exports (daily or monthly).
    • On-Premises: VMware or Hyper-V metrics (CPU usage %, memory allocation, actual resource consumption).

  2. Data Collection & Normalization
    - Tagging / Labeling: Ensure AWS and Azure resources follow a consistent naming/tagging strategy (e.g., Environment=Prod/Dev, Application=XYZ, Owner=TeamName).
    - FOCUSℱ Data Model:

    • In AWS, map UsageType and Operation fields to FOCUSℱ data categories (like compute, storage, network).
    • In Azure, align MeterCategory, MeterSubCategory, ServiceName, etc., with the same FOCUSℱ structure.
    • Frequency & Granularity:
    • Ideally, collect daily usage data to spot cost spikes.
    • Maintain monthly and historical rollups for trending and predictive analysis.

2. Primary Cost Drivers¶

  1. AWS
    - EC2: Typically the largest cost if workloads run 24/7. Key factors: instance family (e.g., C5 vs. M5 vs. R5), on-demand vs. reserved vs. spot usage, and data transfer (especially cross-region egress).
    - RDS: High costs often result from large instance classes, multi-AZ replication, and high IOPS storage (e.g., Provisioned IOPS volumes).
    - Other Services: Watch for hidden cost drivers like EBS (unused attached volumes), S3 data transfer out, and NAT Gateway data processing fees.

  2. Azure
    - Virtual Machines: On-demand VMs can be expensive if not rightsized. Spot VMs or Reserved Instances can significantly reduce this.
    - App Services: Continuous usage for dev/test environments inflates monthly bills.
    - Storage / Networking: Azure Storage tiers (Hot, Cool, Archive) must be optimized; data egress can also be a cost driver.

  3. On-Premises
    - Hardware & Licensing: Annual maintenance or renewal fees for VMware, Microsoft Server, or other enterprise licensing can be significant.
    - Resource Overprovisioning: If many VMs are allocated more CPU/memory than needed, this can also drive up power, cooling, and licensing costs.
    - Facility Costs: Power, cooling, and real estate are ongoing operating expenses, often overlooked in cost comparison with cloud.

  4. References
    - AWS Well-Architected Cost Optimization: AWS Docs – Cost Optimization Pillar
    - Microsoft Cost Management: Azure Cost Management Documentation

3. Optimization Strategies¶

  1. AWS
    - Savings Plans / Reserved Instances

    • Look at stable workloads over the past 3–6 months.
    • Use AWS Cost Explorer or AWS Compute Optimizer to identify instance families with consistent usage.
    • Savings Plans can cut compute costs by up to 66%, while RIs can yield up to 75% off on-demand for 1- or 3-year terms.
    • Rightsizing & Scheduling
    • Implement periodic checks (e.g., weekly or monthly) of CPU/memory utilization to downsize overprovisioned EC2 instances (e.g., M5.xlarge → M5.large).
    • Schedule start/stop times for dev/test instances using AWS Instance Scheduler or custom scripts triggered by N8N.
    • Storage Tiering
    • Switch from EBS gp2 to gp3 for cost efficiency.
    • Archive old data to cheaper tiers such as S3 Glacier Deep Archive.

  2. Azure
    - Reserved Instances

    • For VMs running consistently (e.g., production environments), commit to 1- or 3-year reservations; potential savings range from 30% to 72%.
    • Azure Hybrid Benefit
    • If you have Windows Server or SQL Server licenses with Software Assurance on-premises, convert them to lower Azure VM or SQL Database rates.
    • Autoscaling & Scheduling
    • Use Azure Autoscale rules for VMs and App Services to scale out/in based on CPU, memory thresholds.
    • Shut down dev/test during off-hours—savings can reach 30–50%.
    • Cleanup & Monitoring
    • Regularly remove stale resources (e.g., unattached disks, orphaned public IPs).
    • Use Azure Advisor for recommendations on idle or underutilized assets.

  3. On-Premises
    - Resource Pooling & Consolidation

    • Re-examine your VM distribution; combining underutilized VMs onto fewer hosts reduces power and licensing overhead.
    • Upgrade / Refresh
    • Sometimes, a hardware refresh with more efficient servers is cheaper than running outdated power-hungry hardware.
    • Hybrid Cloud Bursting
    • For seasonal or peak workloads, burst to AWS or Azure Spot VMs to handle transient demand, reducing the need for additional on-premises hardware.

  4. Automation: N8N + Local AI Agents
    - N8N

    • Build a workflow that periodically (e.g., weekly) ingests FOCUSℱ-formatted usage data from AWS, Azure, and on-prem.
    • Automate anomaly detection triggers (e.g., cost spike > 15% WoW) and schedule Slack/email notifications to relevant teams.
    • Local AI Agents (LangChain, Auto-GPT variant, CrewAI, Archon, Huggingface smolagents, etc.)
    • Host these agents locally for data privacy and compliance.
    • Agents can parse usage data, propose new RIs/Savings Plans, or highlight resources to decommission (e.g., unused load balancers, stale dev/test VMs).
    • Example: LangFlow can orchestrate prompts to your local LLM (Open-WebUI) with instructions to generate cost-optimization insights, trending charts, and recommended next steps.

4. Open-Source Integration¶

  1. Running Analysis on Open-WebUI
    - Local Deployment: Install Open-WebUI (or a similar open-source LLM interface) on a secure server or workstation.
    - Data Import: Feed your monthly or weekly cost data (in CSV, JSON, or direct from an internal API) to the local LLM.
    - Prompt Template: Provide context about your environment, usage patterns, and highlight known cost pain points. The LLM can then run a step-by-step analysis, summarizing findings.

  2. Periodic Triggers via N8N
    - Workflow Design:

    • Create a schedule node (e.g., weekly, monthly) to pull cost data from AWS (Cost Explorer exports), Azure (Cost Management exports), and on-premises tools (via VMware/Hyper-V APIs).
    • Transform data into FOCUSℱ format if needed.
    • Pass the transformed dataset to your local LLM endpoint (e.g., via an HTTP request node to the Open-WebUI service).
    • Automated Output:
    • Generate PDFs or markdown reports with cost optimization recommendations.
    • Distribute these reports to stakeholders or store them in a knowledge base (e.g., Confluence, SharePoint).

5. Detailed Recommendations & Feasibility¶

Recommendation Est. Savings Complexity Dependencies / Prereqs
1. AWS Savings Plans / RIs ~40–66% on EC2 Medium Predictable usage, financial approval for commit.
2. Azure Reserved Instances ~30–72% on VMs Medium Historical usage data, cost center sign-off.
3. Hybrid Benefit (Azure) Up to ~49% on Windows Medium Existing Windows/SQL licenses with SA.
4. Scheduling Dev/Test ~30–50% savings Low Implementation of automation (N8N or custom scripts).
5. Right-Sizing (AWS/Azure) ~15–25% savings Low Monitoring actual CPU/mem usage, feasible in short term
6. Storage Tiering (AWS + Azure + On-Prem) ~10–30% on storage Low Clear data lifecycle policy, identification of cold data.
7. On-Prem Consolidation ~10–20% on infra OPEX Medium Possible hardware reallocation, downtime scheduling.
8. Hybrid Cloud Bursting (On-Prem → Cloud) Variable (~10–40%) High Network connectivity (VPN or Direct Connect/ExpressRoute), scaling strategy.
  • ROI: Typically, Scheduling & Right-Sizing are quick wins (low complexity, immediate returns). RIs/Savings Plans require financial commitment but yield significant savings. On-Premises optimization can be more involved (hardware constraints, licensing complexities).

6. Continuous Improvement & Governance¶

  1. Governance Model
    - Weekly Checks

    • Spot anomalies in cost or usage.
    • Quickly rectify abandoned or overprovisioned resources.
    • Monthly Management Reviews
    • Assess overall spending vs. budget.
    • Approve new reservations or expansions of existing commitments.
    • Quarterly Deep Dives
    • Conduct architecture reviews with AWS Well-Architected or Azure Advisor frameworks.
    • Revisit on-premises hardware lifecycle and refresh plans.

  2. Security & Performance Integration
    - AWS: Use Trusted Advisor, Security Hub, and GuardDuty to ensure compliance and security are not compromised by cost-focused changes.
    - Azure: Integrate Azure Advisor (cost + security + performance) and Azure Security Center for recommended best practices.
    - On-Prem: Ensure any consolidation or resource reduction doesn’t degrade performance or violate licensing or compliance rules (e.g., PCI, HIPAA).

  3. Feedback Loop with AI Agents
    - Log changes that yield cost savings or performance impacts. Feed these results back into your local AI Agent to improve future recommendations.
    - Over time, the system learns from real outcomes, refining the thresholds and triggers used in the next iteration of cost management.

7. Clarifying Questions¶

  1. AWS
    - Do you currently have any Savings Plans or Reserved Instances for EC2 or RDS in place, and what is their coverage percentage?
    - Could you share typical CPU/memory utilization for your top 5 most expensive EC2 instances?

  2. Azure
    - Have you leveraged Azure Hybrid Benefit for any Windows/SQL workloads?
    - Are you already using any Reserved Instances for VMs or SQL Databases?

  3. On-Premises
    - Which hypervisor do you primarily use (VMware, Hyper-V, other)? What does your current licensing model look like?
    - Have you initiated any hardware refresh or consolidation projects in the last 12 months?

  4. Data & Automation
    - Are you interested in setting up an N8N pipeline for cost data ingestion, or do you already have a pipeline in place?
    - How frequently would you prefer cost-optimization reports (weekly vs. monthly)?

4. Final Thoughts¶

Prompt engineering is not a one-size-fits-all approach; it’s an iterative craft. Experiment with different techniques, and combine them as needed. For instance, you might use Contextual Prompting alongside Chain-of-Thought for complicated cost breakdowns, or rely on Few-Shot Prompting to ensure consistent output formats.

Key Takeaways: - Start with simpler approaches (Zero-Shot, One-Shot) for straightforward tasks.
- Move to advanced techniques (Thread-of-Thought, Tree-of-Thought, ReAct) as complexity grows.
- Maintain clarity, context, and (where possible) reference authoritative sources like AWS Well-Architected and Microsoft Cost Management for domain credibility.

This tutorial blog demonstrates how each of the 17 techniques can enhance the Cloud Cost Optimization Advisor scenario (Prompt #1). Armed with these strategies, you can produce more accurate, insightful, and actionable outputs from your LLM—whether you’re performing a monthly cost review, building a FinOps application, or engaging in more advanced multi-cloud transformations.