On this submit, I’ll present you tips on how to use Amazon Bedrock—with its absolutely managed, on-demand API—together with your Amazon SageMaker educated or fine-tuned mannequin.
Amazon Bedrock is a totally managed service that gives a alternative of high-performing basis fashions (FMs) from main AI firms similar to AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon by means of a single API, together with a broad set of capabilities to construct generative AI functions with safety, privateness, and accountable AI.
Beforehand, when you needed to make use of your individual customized fine-tuned fashions in Amazon Bedrock, you both needed to self-manage your inference infrastructure in SageMaker or practice the fashions immediately inside Amazon Bedrock, which requires expensive provisioned throughput.
With Amazon Bedrock Customized Mannequin Import, you should utilize new or present fashions which were educated or fine-tuned inside SageMaker utilizing Amazon SageMaker JumpStart. You may import the supported architectures into Amazon Bedrock, permitting you to entry them on demand by means of the Amazon Bedrock absolutely managed invoke mannequin API.
Resolution overview
On the time of writing, Amazon Bedrock helps importing customized fashions from the next architectures:
- Mistral
- Flan
- Meta Llama 2 and Llama 3
For this submit, we use a Hugging Face Flan-T5 Base mannequin.
Within the following sections, I stroll you thru the steps to coach a mannequin in SageMaker JumpStart and import it into Amazon Bedrock. Then you’ll be able to work together together with your customized mannequin by means of the Amazon Bedrock playgrounds.
Conditions
Earlier than you start, confirm that you’ve got an AWS account with Amazon SageMaker Studio and Amazon Bedrock entry.
When you don’t have already got an occasion of SageMaker Studio, see Launch Amazon SageMaker Studio for directions to create one.
Prepare a mannequin in SageMaker JumpStart
Full the next steps to coach a Flan mannequin in SageMaker JumpStart:
- Open the AWS Administration Console and go to SageMaker Studio.
- In SageMaker Studio, select JumpStart within the navigation pane.
With SageMaker JumpStart, machine studying (ML) practitioners can select from a broad collection of publicly accessible FMs utilizing pre-built machine studying options that may be deployed in just a few clicks.
- Seek for and select the Hugging Face Flan-T5 Base
On the mannequin particulars web page, you’ll be able to overview a brief description of the mannequin, tips on how to deploy it, tips on how to fine-tune it, and what format your coaching knowledge must be in to customise the mannequin.
- Select Prepare to start fine-tuning the mannequin in your coaching knowledge.
Create the coaching job utilizing the default settings. The defaults populate the coaching job with advisable settings.
- The instance on this submit makes use of a prepopulated instance dataset. When utilizing your individual knowledge, enter its location within the Information part, ensuring it meets the format necessities.
- Configure the safety settings similar to AWS Identification and Entry Administration (IAM) position, digital personal cloud (VPC), and encryption.
- Notice the worth for Output artifact location (S3 URI) to make use of later.
- Submit the job to start out coaching.
You may monitor your job by deciding on Coaching on the Jobs dropdown menu. When the coaching job standing reveals as Accomplished, the job has completed. With default settings, coaching takes about 10 minutes.
Import the mannequin into Amazon Bedrock
After the mannequin has accomplished coaching, you’ll be able to import it into Amazon Bedrock. Full the next steps:
- On the Amazon Bedrock console, select Imported fashions beneath Basis fashions within the navigation pane.
- Select Import mannequin.
- For Mannequin identify, enter a recognizable identify to your mannequin.
- Beneath Mannequin import settings, choose Amazon SageMaker mannequin and choose the radio button subsequent to your mannequin.
- Beneath Service entry, choose Create and use a brand new service position and enter a reputation for the position.
- Select Import mannequin.
- The mannequin import will full in about quarter-hour.
- Beneath Playgrounds within the navigation pane, select Textual content.
- Select Choose mannequin.
- For Class, select Imported fashions.
- For Mannequin, select flan-t5-fine-tuned.
- For Throughput, select On-demand.
- Select Apply.
Now you can work together together with your customized mannequin. Within the following screenshot, we use our instance customized mannequin to summarize an outline about Amazon Bedrock.
Clear up
Full the next steps to scrub up your sources:
- When you’re not going to proceed utilizing SageMaker, delete your SageMaker area.
- When you now not need to keep your mannequin artifacts, delete the Amazon Easy Storage Service (Amazon S3) bucket the place your mannequin artifacts are saved.
- To delete your imported mannequin from Amazon Bedrock, on the Imported fashions web page on the Amazon Bedrock console, choose your mannequin, after which select the choices menu (three dots) and choose Delete.
Conclusion
On this submit, we explored how the Customized Mannequin Import function in Amazon Bedrock allows you to use your individual customized educated or fine-tuned fashions for on-demand, cost-efficient inference. By integrating SageMaker mannequin coaching capabilities with the absolutely managed, scalable infrastructure of Amazon Bedrock, you now have a seamless technique to deploy your specialised fashions and make them accessible by means of a easy API.
Whether or not you favor the user-friendly SageMaker Studio console or the flexibleness of SageMaker notebooks, you’ll be able to practice and import your fashions into Amazon Bedrock. This lets you give attention to creating revolutionary functions and options, with out the burden of managing advanced ML infrastructure.
Because the capabilities of huge language fashions proceed to evolve, the power to combine customized fashions into your functions turns into more and more beneficial. With the Amazon Bedrock Customized Mannequin Import function, now you can unlock the complete potential of your specialised fashions and ship tailor-made experiences to your clients, all whereas benefiting from the scalability and cost-efficiency of a totally managed service.
To dive deeper into fine-tuning on SageMaker, see Instruction fine-tuning for FLAN T5 XL with Amazon SageMaker Jumpstart. To get extra hands-on expertise with Amazon Bedrock, try our Constructing with Amazon Bedrock workshop.
In regards to the Writer
Joseph Sadler is a Senior Options Architect on the Worldwide Public Sector staff at AWS, specializing in cybersecurity and machine studying. With private and non-private sector expertise, he has experience in cloud safety, synthetic intelligence, menace detection, and incident response. His various background helps him architect sturdy, safe options that use cutting-edge applied sciences to safeguard mission-critical methods
