Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
•
0 likes•75 views
In this follow-up session on knowledge and prompt engineering, we will explore structured prompting, chain of thought prompting, iterative prompting, prompt optimization, emotional language prompts, and the inclusion of user signals and industry-specific data to enhance LLM performance. Join EIS Founder & CEO Seth Earley and special guest Nick Usborne, Copywriter, Trainer, and Speaker, as they delve into these methodologies to improve AI-driven knowledge processes for employees and customers alike.
Report
Share
More Related Content
Similar to Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
Similar to Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches (20)
More from Earley Information Science
More from Earley Information Science (20)
Recently uploaded
Recently uploaded (20)
Knowledge and Prompt Engineering Part 2 Focus on Prompt Design Approaches
- 1. www.earley.com WEBINAR WEBINAR The Key to Context: Prompt and Knowledge Engineering - Part 2 Focus on Prompts SETH EARLEY CEO & FOUNDER EARLEY INFORMATION SCIENCE Media Sponsor NICK USBORNE FOUNDER & TRAINER BE MORE HUMAN SANJAY MEHTA PRINCIPAL SOLUTION ARCHITECT EARLEY INFORMATION SCIENCE
- 2. www.earley.com Today’s Speakers Seth@earley.com https://www.linkedin.com/in/sethearley/ 2 Nick Usborne Founder & Trainer Be More Human nick@bemorehuman.ai https://www.linkedin.com/in/ nickusborne/ Seth Earley Founder & CEO Earley Information Science Sanjay Mehta Principal Solution Architect Earley Information Science Sanjay.mehta@earley.com https://www.linkedin.com/in/sanjaymehta/ “I do not know of any books that have such useful and detailed advice on the relationship between data and successful conversational AI systems.” —Tom Davenport, President’s Distinguished Professor at Babson College, Research Fellow at MIT Initiative on the Digital Economy, and author of Only Humans Need Apply and The AI Advantage
- 3. www.earley.com BeforeWe Get Started WE ARE RECORDING SESSION WILL BE 50 MINUTES PLUS 10 MINUTES FOR Q&A YOUR INPUT IS VALUED Link to recording & slides will be sent by email after the webinar Use the Q&A box to submit questions Participate in the polls during the webinar Feedback survey afterward (~1.5 minutes) Thank you to our media partners : CMSWire 3
- 4. www.earley.com About Earley Information Science 4 Proven methodologies to organize information and data. SELL MORE PRODUCT SERVICE CUSTOMERS EFFICIENTLY INNOVATE FASTER 1994 YEAR FOUNDED. Boston HEADQUARTERED. 50+ SPECIALISTS & GROWING.
- 5. www.earley.com It’s all about context 5 • Context windows are growing but there are limitations – adding content by way of example output or adding IP needs to be done in a secure way • Related data sources will further contextualize the vector space • Adding user context as buyer avatar, brand avatar and other user signals and characteristics including voice of end customer and emotion will improve results
- 6. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com PromptApproaches 6 Prompt approaches varies • By task (ex. Summarization, Conversation,Translation, Extraction, Generation etc.) • By model & model size (OpenAI-GPT vs. Mixtral vs. Llama2, etc.) • By mode (text vs. vision vs. voice) • By action (function calling vs fine tuning, code generation). Capability (chatbot vs assistant) Chat is stateless - sends memory back on each request (stateless) completions vs. assistant where standing instructions are provided up front (stateful) & threads – great for repetitive tasks, invoking tools, reducing token duplication and overhead
- 7. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com PromptApproaches 7 • How context is sent ex. Using retrievers for RAG, or Examples • Consider modular prompts to simplify complex tasks such as output of one LLM request goes into the input (prompt of another) ex. multi-modal use case such as vision to text • Accommodate changing context - Dynamic prompts/prompt templates, IncorporatingVolatile Information for bulk requests • Providing examples, zero / multi-shot and chain of thought • Defining output information, structure and information to determine logic/reasoning for diagnosis and debugging • Use LLMs to build prompts
- 8. www.earley.com Structured prompting 8 Design prompts based on use cases, scenarios and the RACE framework… Role -You are an expert on the topic of on-farm dairy equipment Action - Write a series of informational emails for prospects Context - Here is background information on our dairy equipment Examples - Follow the structure, style and voice of these previous, successful informational emails
- 9. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Use Cases as Prompt 9 Use cases inform the knowledge architecture. Libraries of use cases allow for testing functionality, facet identification and baseline metric comparison Role metadata Entity metadata
- 10. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Chain of thought 10 Chain of thought prompting - asking the LLM to work through the problem step by step "Let's think step by step.... Used when complex reasoning is required such as writing code Works for larger models like GPT-4 Is an alternative to providing examples https://arxiv.org/abs/2201.11903
- 11. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Dynamic Prompts 11 "Hello [Customer Name], thank you for contacting us regarding [Issue] on [Date]. Our team is looking into it and will get back to you soon. Meanwhile, can you please provide us with the following details: [Additional Information]?“ Each field and context such as embeddings can populate the template Category Code, Category Name,Brand Code, Brand Name, Series Code, Series Name Original, Series Name New, Embeddings E3107000000,Jumper Bars,MSM1,MISUMI,110400166560,Dedicated Short Bar, -0.04236443713307381, 0.006765271537005901, 0.01923784799873829, ... E3107000000,Jumper Bars,MSM1,MISUMI,110400195710,MKB Series Short Bar, 050873927772045135, -0.0314302034676075, - 0.0084866126999259, …. E3107000000,Jumper Bars,MSM1,MISUMI,110400317670,Short-Circuit Plates for BTDK Series European Style Terminal Blocks, 0.030240707099437714, …. -0.06340941041707993, 0.02191 Ideal for bulk tasks or repeatable use cases that can be templated to achieve scale. - Programmatically or interactively populate elements of the prompt - Pass in dynamic context such as through RAG - When more of a predictable output and approach is required
- 12. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Collaborative Prompting 12 • Collaborative Prompting represents a mindset shift. • Use an AI Model not as a software tool, but as a co-intelligence • Get into conversation, prompt by prompt. • Ask the model to contribute to the conversation. • Example Prompt:“Based on our conversation so far, do you think we’re missing anything important? If so, what?”
- 13. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Empathetic Prompting 13 • Begin with Empathy Mapping - An Empathy Map gives you a picture of what your audience Says,Thinks, Does and Feels. • Empathetic Prompting is about starting with an Empathy Map, building an emotion-first avatar, and then leading with emotion in your very first prompt. Example: Prompting an AI model to write an email to a SaaS prospect who Says she’s excited… but Feels anxious about making a significant change in her company's operations.
- 14. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Empathetic Prompt Example 14 Write an email to a SaaS prospect who is excited about making a significant change in her company's operations “Subject: Excited to hear about your vision for change! I've attached a few resources that you might find helpful: • Case study: [Link to a case study showcasing a similar company achieving success with your solution] • Product brochure: [Link to a brochure outlining the features and benefits of your solution] • Free trial: [Link to a free trial offer, if applicable]” Write an email to a SaaS prospect who Says she’s excited but feels very anxious about making a significant change in her company's operations “Subject: Understanding your concerns and excitement about change Here are some ways we can help you overcome your anxieties: • Personalized onboarding:We'll work closely with you to understand your specific needs and develop a customized implementation plan. • Dedicated customer success manager:You'll have a dedicated point of contact who will be available to answer your questions and provide ongoing support. • Comprehensive training and resources:We offer a variety of training materials and resources to help your team learn and adapt to the new system. • Phased implementation:We can implement the solution in phases, allowing your team to gradually adjust to the changes.”
- 15. www.earley.com • BRAND GUIDELINES • CORPORATE TERMINOLOGY • USER SIGNALS • CORPORATE DATA SOURCES • INDUSTRY SPECIFIC DATA 15 Corporate Standards and User Signals
- 16. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Source Data to be Enriched Original Product Name Original Description Attribute Names Original Catalog Context SCHEMA/ MDM/PIM/ERP Related Artifacts Web Pages/Content Digital Assets (images, diagrams) Existing Search Index Product Sheets, Manuals Knowledge Base Target Audiences / Segments Feed, API, Crawl, Direct Merge by key – series, category or attribute code Unified Document Repository Rules & Governance Standards Bodies (ISO, TC, SC) Restrictions Style Guide Branding Examples (Positive & Negative) Existing Search Rules 3rd Party Intelligence Google Knowledge API & Graph Competitor Websites & Search Results Large Language Models EIS MRO Knowledge Graph User context Signals / Telemetry Search Analytics Clickstream Analytics Transactional Analytics Performance Metrics KPI's Audience / Segment / Profile Prepare/process: normalize, classify, tag Generate Doc Embeddings Create Graph Index Generate Contextual Embeddings (Application, Audience, Behavioral) Generate Query / Phrase / Q&A Embeddings Generate Industry & Organization Specific Ontological Embeddings Create Vector Index Generate LLM Context and Metadata
- 17. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. “THERE’S NO AI WITHOUT IA” Alignment with Knowledge Engineering 17
- 18. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com USER CONTEXT, PROCESS CONTEXT AND CONTENT CONTEXT 18 How do I set up my modem? Where is the installation guide? What does error code 50 mean? Questions andAnswers Need Context
- 19. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Product Manual – HM 2900 Series Modem GenerativeAI Reference Content Requires Context 19 Overview Set up options Settings Model 2960 Error code 50 Settings Model 2970 Error code 56 Installation Troubleshooting Hardware setup VPN requirements Factory settings Technical Specifications I need to install a router. Which router model do you have? Content type = Product Manual Content type = Troubleshooting Content type = Installation Model 2960 I am receiving an error code of 50 OK, here are the installation settings… That error requires the following troubleshooting steps: What context is required? Type of information, installation, product model, error code, etc. Metadata provides context Product name = HM 2900 Series Router Model = 2960 Error code = 50
- 20. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Prompt as Metadata Container 20 “I need to install a model 2960 router, but I am receiving an error code of 50. Please provide the troubleshooting steps to complete my installation” Content type = Product Manual Content type = Troubleshooting Content type = Installation What context is required? Type of information, installation, product model, error code, etc. Prompt metadata provides context Product name = HM 2900 Series Modem Model = 2960 Error code = 50
- 21. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. Knowledge Graphs 21 What is the purpose of a Knowledge Graph? How are linguistics and analytics integrated? Knowledge Graphs connect the LLM to corporate knowledge and data. They are the source of truth for the LLM so that responses are appropriate to enterprise information
- 22. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com Derived Knowledge Graph
- 23. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Prompt Metadata 23 Asking for outputs and then using knowledge to ask additional questions. “What questions should I ask about Generative AI and prompts?” Understanding the Prompt's Purpose: • What is the main goal or desired outcome of this prompt? (e.g., generate creative text formats, translate languages, write different kinds of creative content) • Who is the target audience for the generated output? (e.g., general public, specific professionals, children) • In what context will the output be used? (e.g., marketing materials, educational content, personal entertainment) Crafting an Effective Prompt: • What specific information or keywords must be included in the prompt? • How can I phrase the prompt to ensure clarity and avoid ambiguity? How can you describe the output? (tone, mood, technical depth) How can you describe the audience? What is the expected artefact (plan, program, campaign, contract)? What additional metadata (features) can you provide? What is the use case?
- 24. © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com Summary 24 Develop use case libraries for iterative testing Create reference prompts representing classes of use case Provide cheat sheets and training materials for how users need to ask questions Align prompting with the knowledge architecture to provide clues as to how to ask questions to get content in context
- 25. www.earley.com Getting Started /Next Steps 25 • Identify a business problem that is important to the organization • Ensure adequate funding and executive support • Define scenarios and use cases • Identify content and data needed to support use cases • Gather baseline metrics for supported processes • Build domain model and metadata structures for target use cases • Ingest tagged content into vector store as enriched embeddings • Test against use cases with gold standard of responses • Measure against baselines to show improvement • Shampoo, Rinse, Repeat
- 26. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com www.earley.com KM for AI Readiness Sprint and Proof of Value Plan A KM for AI readiness sprint will quickly identify how the organization can deploy Generative AI to address issues and challenges that arise from knowledge capabilities that have not kept up with growth, acquisitions, changes in the marketplace or changes in products, technologies and the competitive landscape. The sprint consists of the following: • Stakeholder interviews • Education and alignment • Current state maturity • Review of knowledge systems and tools • Generative AI Proof of Value (PoV) plan https://www.earley.com/km-ai-readiness-assessment
- 27. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com www.earley.com Maturity Model for Knowledge Management 1-Unpredictable 2-Aware 3-Competent 4-Synchronized 5-Choreographed Core Collaboration Rudimentary, random , haphazard Intentional, ineffective knowledge capture and codification Practices identified, formal harvesting and promotion of output Integrated into processes with creation, access and reuse mapped Seamless and habitual with collaboration processes integrated with business needs and downstream uses Expertise Location Who you know Word of mouth Key skills Identified and captured Formal expertise directories leveraged Expertise derivation through text analytics, community participation and group interactions Content Curation Haphazard and application limited Pockets of curated content, sub optimized across processes Content aligned with platform and process, governance driven information concierge Workflow driven integration with hybrid tagging and entity extraction processes Value added at each touch point, core metadata flows with content, lifecycles managed , retention enforced Information Architecture IA is Navigation, inconsistent metadata, few standards, poor usability Application and department localized taxonomies Classification structures applied to support dynamic knowledge Multi-channel, device and format independent cross application architecture Upstream supply chain and downstream syndication w/ partner & customer processes Infrastructure Foundational, little to no collaborative tools, CM rudimentary Foundational tools in place but with out of the box deployment Knowledge harvesting integrated with collaboration and ruse Expertise location, community management, intentional knowledge optimization Multiple tools mapped to detailed requirements and use cases with ongoing tuning and enhancement Search Integration Search as “Random Document Generator “ Some tuning of search ranking algorithm with content tagging Integration across structured and unstructured systems for content in context Expertise mapped to search domains and terms, reduced e-discovery risk Search driven integration across platforms, knowledge in task and process context Governance Non existent Initial attempts lead to fiefdoms Repeatable, defined KM governance Integrated, cross functional managed processes Business value driven, enterprise wide deployment Capability Maturity
- 28. Copyright © 2023 Earley Information Science, Inc. All Rights Reserved. Knowledge Management and AI Working Session Topics Topic Overview Goals Questions to Address Key Concepts and Success Criteria Generative AI and knowledge management definitions and success factors. • Level set on key terms and foundational understanding. • How can the organization make use of Generative AI? • What is the role of knowledge? Knowledge in Context Overview of business objectives and use cases for knowledge application • Orient client team to multiple knowledge taxonomies. • Get initial response on which are most important (to client). • Which business priorities must the AI serve first? • Which teams will be most involved? Current Landscape Definition Map knowledge and content by people and systems to identify how business processes are currently supported. • Develop more complete view of scope and scale of systems with knowledge and content. • Map knowledge to process • Develop domain model • Where are key knowledge leverage (usage) points/processes? • Where are key areas of changes re: people and process? PoV Planning Plan and proposal for Generative AI Proof of Value. • Implement knowledge base with AI powered chat front end • What are the costs and needed infrastructure for execution?
- 29. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com www.earley.com The EIS KM and AI Readiness Assessment Can Help Get You There Through a combination of interviews, questionnaires, surveys and working sessions, the EIS KM for AI Readiness Assessment: 1. Educates executives and stakeholders about AI technologies – capabilities and limitations 2. Evaluates business value and target use cases for Generative AI 3. Outlines success factors and metrics 4. Examines critical areas of the enterprise for Generative AI readiness: • Business alignment and process clarity • Knowledge and Data readiness and technology infrastructure • Ongoing governance, decision making and success measures 5. Summarizes the current state in an executive working session designed to identify gaps, set realistic goals and prioritize actions for a Generative AI Proof of Value (PoV) 29 The output of the EIS KM and AI Readiness Assessment is a roadmap for deployment of a Generative AI PoV based on corporate knowledge
- 30. Copyright © 2024 Earley Information Science, Inc. All Rights Reserved. www.earley.com www.earley.com www.earley.com 30 Need Clarity on Retrieval Augmented Generation (RAG)? https://www.earley.com/ama-article Great companion to The AI Powered Enterprise Download now: or you can request a physical reprint
- 31. www.earley.com Contact Seth@earley.com https://www.linkedin.com/in/sethearley/ 31 Seth Earley Founder & CEO Earley Information Science Dave Skrobela Client Partner Managing Director Earley Information Science dave.skrobela@earley.com Sanjay Mehta Principal Solution Architect Earley Information Science Sanjay.mehta@earley.com https://www.linkedin.com/in/sanjaymehta/ www.linkedin.com/in/skrobela/ Nick Usborne Founder & Trainer Be More Human nick@bemorehuman.ai https://www.linkedin.com/in/ nickusborne/ Check out https://bemorehuman.ai/ for training courses
- 32. www.earley.com 32 We Make Information More Useable, Findable, And Valuable Earley Information Science is a professional services firm headquartered in Boston and founded in 1994. With over 50+ specialists and growing, Earley focuses on architecting and organizing data – making it more findable, usable, and valuable. Our proven methodologies are designed to address product data, content assets, customer data, and corporate knowledge bases. We deliver scalable solutions to the world’s leading brands, driving measurable business results.