Construct a generative AI Slack chat assistant utilizing Amazon Bedrock and Amazon Kendra

Regardless of the proliferation of knowledge and information in enterprise environments, workers and stakeholders typically discover themselves trying to find info and struggling to get their questions answered rapidly and effectively. This may result in productiveness losses, frustration, and delays in decision-making.

A generative AI Slack chat assistant may help tackle these challenges by offering a available, clever interface for customers to work together with and procure the knowledge they want. Through the use of the pure language processing and technology capabilities of generative AI, the chat assistant can perceive consumer queries, retrieve related info from numerous information sources, and supply tailor-made, contextual responses.

By harnessing the facility of generative AI and Amazon Net Providers (AWS) providers Amazon Bedrock, Amazon Kendra, and Amazon Lex, this answer offers a pattern structure to construct an clever Slack chat assistant that may streamline info entry, improve consumer experiences, and drive productiveness and effectivity inside organizations.

Why use Amazon Kendra for constructing a RAG software?

Amazon Kendra is a totally managed service that gives out-of-the-box semantic search capabilities for state-of-the-art rating of paperwork and passages. You need to use Amazon Kendra to rapidly construct high-accuracy generative AI functions on enterprise information and supply essentially the most related content material and paperwork to maximise the standard of your Retrieval Augmented Technology (RAG) payload, yielding higher massive language mannequin (LLM) responses than utilizing standard or keyword-based search options. Amazon Kendra presents simple-to-use deep studying search fashions which can be pre-trained on 14 domains and don’t require machine studying (ML) experience. Amazon Kendra can index content material from a variety of sources, together with databases, content material administration programs, file shares, and internet pages.

Additional, the FAQ function in Amazon Kendra enhances the broader retrieval capabilities of the service, permitting the RAG system to seamlessly swap between offering prewritten FAQ responses and dynamically producing responses by querying the bigger information base. This makes it well-suited for powering the retrieval element of a RAG system, permitting the mannequin to entry a broad information base when producing responses. By integrating the FAQ capabilities of Amazon Kendra right into a RAG system, the mannequin can use a curated set of high-quality, authoritative solutions for generally requested questions. This may enhance the general response high quality and consumer expertise, whereas additionally decreasing the burden on the language mannequin to generate these primary responses from scratch.

This answer balances retaining customizations when it comes to mannequin choice, immediate engineering, and including FAQs with not having to take care of phrase embeddings, doc chunking, and different lower-level complexities sometimes required for RAG implementations.

Answer overview

The chat assistant is designed to help customers by answering their questions and offering info on a wide range of matters. The aim of the chat assistant is to be an internal-facing Slack device that may assist workers and stakeholders discover the knowledge they want.

The structure makes use of Amazon Lex for intent recognition, AWS Lambda for processing queries, Amazon Kendra for looking out by way of FAQs and internet content material, and Amazon Bedrock for producing contextual responses powered by LLMs. By combining these providers, the chat assistant can perceive pure language queries, retrieve related info from a number of information sources, and supply humanlike responses tailor-made to the consumer’s wants. The answer showcases the facility of generative AI in creating clever digital assistants that may streamline workflows and improve consumer experiences primarily based on mannequin selections, FAQs, and modifying system prompts and inference parameters.

Structure diagram

The next diagram illustrates a RAG method the place the consumer sends a question by way of the Slack software and receives a generated response primarily based on the information listed in Amazon Kendra. On this submit, we use Amazon Kendra Net Crawler as the information supply and embody FAQs saved on Amazon Easy Storage Service (Amazon S3). See Knowledge supply connectors for a listing of supported information supply connectors for Amazon Kendra.

The step-by-step workflow for the structure is the next:

1. The consumer sends a question akin to What's the AWS Properly-Architected Framework? by way of the Slack app.
2. The question goes to Amazon Lex, which identifies the intent.
3. Presently two intents are configured in Amazon Lex (Welcome and FallbackIntent).
4. The welcome intent is configured to reply with a greeting when a consumer enters a greeting akin to “hello” or “good day.” The assistant responds with “Good day! I may help you with queries primarily based on the paperwork offered. Ask me a query.”
5. The fallback intent is fulfilled with a Lambda operate.
   1. The Lambda operate searches Amazon Kendra FAQs by way of the search_Kendra_FAQ technique by taking the consumer question and Amazon Kendra index ID as inputs. If there’s a match with a excessive confidence rating, the reply from the FAQ is returned to the consumer.

      def search_Kendra_FAQ(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and checks if the query exists within the Kendra FAQs.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: If present in FAQs, returns the reply together with any related hyperlinks. If not, returns False after which calls kendra_retrieve_document operate.
          """
          kendra_client = boto3.consumer('kendra')
          response = kendra_client.question(IndexId=kendra_index_id, QueryText=query, QueryResultTypeFilter="QUESTION_ANSWER")
          for merchandise in response['ResultItems']:
              score_confidence = merchandise['ScoreAttributes']['ScoreConfidence']
              # Taking solutions from FAQs which have a really excessive confidence rating solely
              if score_confidence == 'VERY_HIGH' and len(merchandise['AdditionalAttributes']) > 1:
                  textual content = merchandise['AdditionalAttributes'][1]['Value']['TextWithHighlightsValue']['Text']
                  url = "None"
                  if merchandise['DocumentURI'] != '':
                      url = merchandise['DocumentURI']
                  return (textual content, url)
          return (False, False)

   2. If there isn’t a match with a excessive sufficient confidence rating, related paperwork from Amazon Kendra with a excessive confidence rating are retrieved by way of the kendra_retrieve_document technique and despatched to Amazon Bedrock to generate a response because the context.

      def kendra_retrieve_document(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and retrieves related passages primarily based on default PageSize of 10.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: Returns the context to be despatched to the LLM and doc URIs to be returned as related information sources.
          """
          kendra_client = boto3.consumer('kendra')
          paperwork = kendra_client.retrieve(IndexId=kendra_index_id, QueryText=query)
          textual content = ""
          uris = set()
          if len(paperwork['ResultItems']) > 0:
              for i in vary(len(paperwork['ResultItems'])):
                  score_confidence = paperwork['ResultItems'][i]['ScoreAttributes']['ScoreConfidence']
                  if score_confidence == 'VERY_HIGH' or score_confidence == 'HIGH':
                      textual content += paperwork['ResultItems'][i]['Content'] + "n"
                      uris.add(paperwork['ResultItems'][i]['DocumentURI'])
          return (textual content, uris)

   3. The response is generated from Amazon Bedrock with the invokeLLM technique. The next is a snippet of the invokeLLM technique inside the success operate. Learn extra on inference parameters and system prompts to switch parameters which can be handed into the Amazon Bedrock invoke mannequin request.

      def invokeLLM(query, context, modelId):
          """
          This operate takes within the query from the consumer, together with the Kendra responses as context to generate a solution
          for the consumer on the frontend.
          :param query: The query the consumer is asking that was requested through the frontend enter textual content field.
          :param paperwork: The response from the Kendra doc retrieve question, used as context to generate a greater
          reply.
          :return: Returns the ultimate reply that will probably be offered to the end-user of the applying who requested the unique
          query.
          """
          # Setup Bedrock consumer
          bedrock = boto3.consumer('bedrock-runtime')
          # configure mannequin specifics akin to particular mannequin
          modelId = modelId
      
          # physique of information with parameters that's handed into the bedrock invoke mannequin request
          physique = json.dumps({"max_tokens": 350,
                  "system": "You're a truthful AI assistant. Your purpose is to offer informative and substantive responses to queries primarily based on the paperwork offered. Should you have no idea the reply to a query, you in truth say you have no idea.",
                  "messages": [{"role": "user", "content": "Answer this user query:" + question + "with the following context:" + context}],
                  "anthropic_version": "bedrock-2023-05-31",
                      "temperature":0,
                  "top_k":250,
                  "top_p":0.999})
      
          # Invoking the bedrock mannequin together with your specs
          response = bedrock.invoke_model(physique=physique,
                                          modelId=modelId)
          # the physique of the response that was generated
          response_body = json.masses(response.get('physique').learn())
          # retrieving the particular completion subject, the place you reply will probably be
          reply = response_body.get('content material')
          # returning the reply as a ultimate consequence, which in the end will get returned to the top consumer
          return reply

   4. Lastly, the response generated from Amazon Bedrock together with the related referenced URLs are returned to the top consumer.

   When choosing web sites to index, adhere to the AWS Acceptable Use Coverage and different AWS phrases. Bear in mind that you would be able to solely use Amazon Kendra Net Crawler to index your individual internet pages or internet pages that you’ve authorization to index. Go to the Amazon Kendra Net Crawler information supply information to be taught extra about utilizing the net crawler as a knowledge supply. Utilizing Amazon Kendra Net Crawler to aggressively crawl web sites or internet pages you don’t personal is not thought of acceptable use.

   Supported options

   The chat assistant helps the next options:

   1. Assist for the next Anthropic’s fashions on Amazon Bedrock:
      • claude-v2
      • claude-3-haiku-20240307-v1:0
      • claude-instant-v1
      • claude-3-sonnet-20240229-v1:0
   2. Assist for FAQs and the Amazon Kendra Net Crawler information supply
   3. Returns FAQ solutions provided that the boldness rating is VERY_HIGH
   4. Retrieves solely paperwork from Amazon Kendra which have a HIGH or VERY_HIGH confidence rating
   5. If paperwork with a excessive confidence rating aren’t discovered, the chat assistant returns “No related paperwork discovered”

   Stipulations

   To carry out the answer, it is advisable have following stipulations:

   • Fundamental information of AWS
   • An AWS account with entry to Amazon S3 and Amazon Kendra
   • An S3 bucket to retailer your paperwork. For extra info, see Step 1: Create your first S3 bucket and the Amazon S3 Person Information.
   • A Slack workspace to combine the chat assistant
   • Permission to put in Slack apps in your Slack workspace
   • Seed URLs for the Amazon Kendra Net Crawler information supply
     • You’ll want authorization to crawl and index any web sites offered
   • AWS CloudFormation for deploying the answer sources

   Construct a generative AI Slack chat assistant

   To construct a Slack software, use the next steps:

   1. Request mannequin entry on Amazon Bedrock for all Anthropic fashions
   2. Create an S3 bucket within the us-east-1 (N. Virginia) AWS Area.
   3. Add the AIBot-LexJson.zip and SampleFAQ.csv information to the S3 bucket
   4. Launch the CloudFormation stack within the us-east-1 (N. Virginia) AWS Area.
   5. Enter a Stack identify of your selection
   6. For S3BucketName, enter the identify of the S3 bucket created in Step 2
   7. For S3KendraFAQKey, enter the identify of the SampleFAQs uploaded to the S3 bucket in Step 3
   8. For S3LexBotKey, enter the identify of the Amazon Lex .zip file uploaded to the S3 bucket in Step 3
   9. For SeedUrls, enter as much as 10 URLs for the net crawler as a comma delimited checklist. Within the instance on this submit, we give the publicly accessible Amazon Bedrock service web page because the seed URL
   10. Depart the remaining as defaults. Select Subsequent. Select Subsequent once more on the Configure stack choices
   11. Acknowledge by choosing the field and select Submit, as proven within the following screenshot
       
   12. Look ahead to the stack creation to finish
   13. Confirm all sources are created
   14. Take a look at on the AWS Administration Console for Amazon Lex
       1. On the Amazon Lex console, select your chat assistant ${YourStackName}-AIBot
       2. Select Intents
       3. Select Model 1 and select Take a look at, as proven within the following screenshot
          
       4. Choose the AIBotProdAlias and select Verify, as proven within the following screenshot. If you wish to make modifications to the chat assistant, you need to use the draft model, publish a brand new model, and assign the brand new model to the AIBotProdAlias. Be taught extra about Versioning and Aliases.
          
       5. Take a look at the chat assistant with questions akin to, “Which AWS service has 11 nines of sturdiness?” and “What’s the AWS Properly-Architected Framework?” and confirm the responses. The next desk reveals that there are three FAQs within the pattern .csv file.
          

          _question	_answer	_source_uri
          Which AWS service has 11 nines of sturdiness?	Amazon S3	https://aws.amazon.com/s3/
          What’s the AWS Properly-Architected Framework?	The AWS Properly-Architected Framework allows prospects and companions to assessment their architectures utilizing a constant method and offers steerage to enhance designs over time.	https://aws.amazon.com/structure/well-architected/
          In what Areas is Amazon Kendra accessible?	Amazon Kendra is presently accessible within the following AWS Areas: Northern Virginia, Oregon, and Eire	https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/

       6. The next screenshot reveals the query “Which AWS service has 11 nines of sturdiness?” and its response. You may observe that the response is identical as within the FAQ file and features a hyperlink.
          
       7. Based mostly on the pages you may have crawled, ask a query within the chat. For this instance, the publicly accessible Amazon Bedrock web page was crawled and listed. The next screenshot reveals the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
          
   15. For integration of the Amazon Lex chat assistant with Slack, see Integrating an Amazon Lex V2 bot with Slack. Select the AIBotProdAlias underneath Alias within the Channel Integrations

   Run pattern queries to check the answer

   1. In Slack, go to the Apps part. Within the dropdown menu, select Handle and choose Browse apps.
      
   2. Seek for ${AIBot} in App Listing and select the chat assistant. It will add the chat assistant to the Apps part in Slack. Now you can begin asking questions within the chat. The next screenshot reveals the query “Which AWS service has 11 nines of sturdiness?” and its response. You may observe that the response is identical as within the FAQ file and features a hyperlink.
      
   3. The next screenshot reveals the query, “What's the AWS Properly-Architected Framework?” and its response.
      
   4. Based mostly on the pages you may have crawled, ask a query within the chat. For this instance, the publicly accessible Amazon Bedrock web page was crawled and listed. The next screenshot reveals the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
      
   5. The next screenshot reveals the query, “What's amazon polly?” As a result of there is no such thing as a Amazon Polly documentation listed, the chat assistant responds with “No related paperwork discovered,” as anticipated.
      

   These examples present how the chat assistant retrieves paperwork from Amazon Kendra and offers solutions primarily based on the paperwork retrieved. If no related paperwork are discovered, the chat assistant responds with “No related paperwork discovered.”

   Clear up

   To wash up the sources created by this answer:

   1. Delete the CloudFormation stack by navigating to the CloudFormation console
   2. Choose the stack you created for this answer and select Delete
   3. Verify the deletion by coming into the stack identify within the offered subject. It will take away all of the sources created by the CloudFormation template, together with the Amazon Kendra index, Amazon Lex chat assistant, Lambda operate, and different associated sources.

   Conclusion

   This submit describes the event of a generative AI Slack software powered by Amazon Bedrock and Amazon Kendra. That is designed to be an internal-facing Slack chat assistant that helps reply questions associated to the listed content material. The answer structure consists of Amazon Lex for intent identification, a Lambda operate for fulfilling the fallback intent, Amazon Kendra for FAQ searches and indexing crawled internet pages, and Amazon Bedrock for producing responses. The submit walks by way of the deployment of the answer utilizing a CloudFormation template, offers directions for operating pattern queries, and discusses the steps for cleansing up the sources. General, this submit demonstrates how you can use numerous AWS providers to construct a strong generative AI–powered chat assistant software.

   This answer demonstrates the facility of generative AI in constructing clever chat assistants and search assistants. Discover the generative AI Slack chat assistant: Invite your groups to a Slack workspace and begin getting solutions to your listed content material and FAQs. Experiment with completely different use instances and see how one can harness the capabilities of providers like Amazon Bedrock and Amazon Kendra to reinforce your online business operations. For extra details about utilizing Amazon Bedrock with Slack, check with Deploy a Slack gateway for Amazon Bedrock.

   ---

   Concerning the authors

   Kruthi Jayasimha Rao is a Associate Options Architect with a give attention to AI and ML. She offers technical steerage to AWS Companions in following greatest practices to construct safe, resilient, and extremely accessible options within the AWS Cloud.

   Mohamed Mohamud is a Associate Options Architect with a give attention to Knowledge Analytics. He focuses on streaming analytics, serving to companions construct real-time information pipelines and analytics options on AWS. With experience in providers like Amazon Kinesis, Amazon MSK, and Amazon EMR, Mohamed allows data-driven decision-making by way of streaming analytics.