configure cross-account mannequin deployment utilizing Amazon Bedrock Customized Mannequin Import

In enterprise environments, organizations usually divide their AI operations into two specialised groups: an AI analysis workforce and a mannequin internet hosting workforce. The analysis workforce is devoted to creating and enhancing AI fashions utilizing mannequin coaching and fine-tuning strategies. In the meantime, a separate internet hosting workforce is liable for deploying these fashions throughout their very own improvement, staging, and manufacturing environments.

With Amazon Bedrock Customized Mannequin Import, the internet hosting workforce can import and serve customized fashions utilizing supported architectures similar to Meta Llama 2, Llama 3, and Mistral utilizing On-Demand pricing. Groups can import fashions with weights in Hugging Face safetensors format from Amazon SageMaker or from Amazon Easy Storage Service (Amazon S3). These imported customized fashions work alongside current Amazon Bedrock basis fashions (FMs) via a single, unified API in a serverless method, assuaging the necessity to handle mannequin deployment and scaling.

Nonetheless, in such enterprise environments, these groups usually work in separate AWS accounts for safety and operational causes. The mannequin improvement workforce’s coaching outcomes, often known as mannequin artifacts, for instance mannequin weights, are sometimes saved in S3 buckets inside the analysis workforce’s AWS account, however the internet hosting workforce must entry these artifacts from one other account to deploy fashions. This creates a problem: how do you securely share mannequin artifacts between accounts?

That is the place cross-account entry turns into vital. With Amazon Bedrock Customized Mannequin Import cross-account assist, we will help you configure direct entry between the S3 buckets storing mannequin artifacts and the internet hosting account. This streamlines your operational workflow whereas sustaining safety boundaries between groups. Considered one of our prospects quotes:

Bedrock Customized Mannequin Import cross-account assist helped AI Platform workforce to simplify the configuration, scale back operational overhead and safe fashions within the authentic location.

– Scott Chang, Principal Engineer, AI Platform at Salesforce

On this information, we stroll you thru step-by-step directions for configuring cross-account entry for Amazon Bedrock Customized Mannequin Import, protecting each non-encrypted and AWS Key Administration Service (AWS KMS) primarily based encrypted eventualities.

Instance state of affairs

For this walkthrough, contemplate two AWS accounts:

• Mannequin Improvement account (111122223333):
  • Shops mannequin artifacts (customized weights and configurations) in an S3 bucket known as model-artifacts-111122223333
  • Optionally encrypts artifacts utilizing AWS KMS buyer managed key kms-cmk-111122223333
• Mannequin Internet hosting account (777788889999):
  • Hosts fashions utilizing Amazon Bedrock Customized Mannequin Import
  • Makes use of a brand new AWS Identification and Entry Administration (IAM) execution function BedrockCMIExecutionRole-777788889999
  • Can optionally encrypt artifacts utilizing AWS KMS key kms-cmk-777788889999

The next determine illustrates this setup, exhibiting how the cross-account entry is configured between the S3 bucket, KMS keys, and Amazon Bedrock Customized Mannequin Import.

To efficiently implement the described state of affairs whereas adhering to the precept of least privilege entry, the next steps should be executed:

1. The Mannequin Improvement account should present entry to the Mannequin Internet hosting account’s IAM function BedrockCMIExecutionRole-777788889999, permitting it to make the most of their S3 bucket and, if relevant, the encryption key, utilizing resource-based insurance policies.
2. The Mannequin Internet hosting account ought to set up an IAM function, similar to BedrockCMIExecutionRole-777788889999. The identity-based insurance policies wanted can be for the Mannequin Improvement S3 bucket and buyer managed keys for decrypting mannequin artifacts, like utilizing kms-cmk-111122223333.
3. The Mannequin Internet hosting account should allow the Amazon Bedrock service to imagine the IAM function BedrockCMIExecutionRole-777788889999, created in step 2, by together with the Amazon Bedrock service as a trusted entity. This IAM function might be utilized by the Mannequin Internet hosting account to provoke the customized mannequin import job.

Conditions

Earlier than you can begin a customized mannequin import job, you might want to fulfill the next stipulations:

1. Should you’re importing your mannequin from an S3 bucket, put together your mannequin information within the Hugging Face weights format. For extra data discuss with Import supply.
2. (Optionally available) Arrange additional safety configurations.

Step-by-step execution

The next part offers the step-by-step execution of the beforehand outlined high-level course of, from the attitude of an administrator managing each accounts:

Step 1: Arrange the S3 bucket coverage (within the Mannequin Improvement account) to allow entry for the Mannequin Internet hosting account’s IAM function:

1. Sign up to the AWS Administration Console for account 111122223333, then entry the Amazon S3 console.
2. On the Normal function buckets view, find model-artifacts-111122223333, the bucket utilized by the mannequin improvement workforce to retailer their mannequin artifacts.
3. On the Permissions tab, choose Edit within the Bucket coverage part, and insert the next IAM resource-based coverage. Make sure to replace the AWS account IDs (proven in crimson) within the coverage together with your data.

   {
       "Model": "2012-10-17",
       "Id": "AllowCrossAccountS3Access",
       "Assertion": [
           {
               "Sid": "cross-account-list-get",
               "Effect": "Allow",
               "Principal": {
    "AWS": "arn:aws:iam::777788889999:root"             },
               "Action": [
                   "s3:ListBucket",
                   "s3:GetObject"
               ],
               "Useful resource": [
    "arn:aws:s3:::model-artifacts-111122223333", "arn:aws:s3:::model-artifacts-111122223333/*"             ],
               "Situation": {
                   "ArnLike": {
    "aws:PrincipalArn": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999*"                 }
               }
           }
       ]
   }

Step 2: Set up an IAM function (within the Mannequin Internet hosting account) and authorize Amazon Bedrock to imagine this function:

1. Sign up to the AWS console for account 777788889999 and launch the IAM console.
2. Within the left navigation pane, choose Insurance policies after which select Create coverage. Inside the Coverage Editor, change to the JSON tab and insert the next identity-based coverage. This coverage is designed for read-only entry, enabling customers or a job to checklist and obtain objects from a specified S3 bucket, however provided that the bucket is owned by account 111122223333. Customise the AWS account ID and S3 bucket title/prefix (proven in crimson) together with your data.

   {
       "Model": "2012-10-17",
       "Assertion": [
           {
               "Sid": "1",
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetObject"
               ],
               "Useful resource": [
    "arn:aws:s3:::model-artifacts-111122223333", "arn:aws:s3:::model-artifacts-111122223333/*"             ],
               "Situation": {
                   "StringEquals": {
     "aws:ResourceAccount": "111122223333"                 }
               }
           }
       ]
   }

3. Select Subsequent, assign the coverage title as BedrockCMIExecutionPolicy-777788889999, and finalize by selecting Create coverage.
4. Within the left navigation pane, select Roles and choose Customized belief coverage because the Trusted entity sort. Insert the next trusted entity coverage, which restricts the function assumption to the Amazon Bedrock service, particularly for mannequin import jobs in account 777788889999 positioned within the US East (N. Virginia) us-east-1 Area. Modify the AWS account ID and Area (proven in crimson) together with your data.

   {
       "Model": "2012-10-17",
       "Assertion": [
           {
               "Sid": "1",
               "Effect": "Allow",
               "Principal": {
                   "Service": "bedrock.amazonaws.com"
               },
               "Action": "sts:AssumeRole",
               "Condition": {
                   "StringEquals": {
    "aws:SourceAccount": "777788889999"                 },
                   "ArnEquals": {
    "aws:SourceArn": "arn:aws:bedrock:us-east-1:777788889999:model-import-job/*"                 }
               }
           }
       ]
   }

5. Select Subsequent and within the Add permissions part, seek for the coverage created within the earlier step BedrockCMIExecutionPolicy-777788889999, choose the checkbox, and proceed by selecting Subsequent.
6. Assign the Function title as BedrockCMIExecutionRole-777788889999, present a Description as “IAM execution function for use by CMI jobs,” and finalize by selecting Create function.

Vital: Should you’re utilizing an AWS KMS encryption key for mannequin artifacts within the Mannequin Improvement account or for imported mannequin artifacts with the Amazon Bedrock managed AWS account, proceed with steps 3 via 5. If not, skip to step 6.

Step 3: Regulate the AWS KMS key coverage (within the Mannequin Improvement account) to permit the Amazon Bedrock CMI execution IAM function to decrypt mannequin artifacts:

1. Transition again to the Mannequin Improvement account and discover the AWS KMS key named kms-cmk-111122223333 within the AWS KMS console. Notice the AWS KMS key Amazon Useful resource Title (ARN).
2. On the Key coverage tab, change to the Coverage view, and incorporate the next resource-based coverage assertion to allow the Mannequin Internet hosting account’s IAM function BedrockCMIExecutionRole-777788889999 to decrypt mannequin artifacts. Revise objects in crimson together with your data.

   {
         "Sid": "Enable use of the important thing by the vacation spot account",
         "Impact": "Enable",
         "Principal": {
    "AWS": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999"       },
         "Motion": [
           "kms:Decrypt",
           "kms:DescribeKey"
         ],
         "Useful resource": "*"
   }

Step 4: Set the AWS KMS key coverage (within the Mannequin Internet hosting account) for the CMI execution IAM function to encrypt and decrypt mannequin artifacts to securely retailer within the Amazon Bedrock AWS account:

1. Return to the Mannequin Internet hosting account and find the AWS KMS key named kms-cmk-777788889999 within the AWS KMS console. Notice the AWS KMS key ARN.
2. Insert the next assertion into the AWS KMS key’s resource-based coverage to allow the BedrockCMIExecutionRole-777788889999 IAM function to encrypt and decrypt mannequin artifacts at relaxation within the Amazon Bedrock managed AWS account. Revise objects in crimson together with your data.

   {
         "Sid": "Enable use of the important thing",
         "Impact": "Enable",
         "Principal": {
    "AWS": "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999"       },
         "Motion": [
           "kms:Encrypt",
           "kms:Decrypt",
           "kms:ReEncrypt*",
           "kms:GenerateDataKey*",
           "kms:DescribeKey"
         ],
         "Useful resource": "*"
   }

Step 5: Modify the CMI execution function’s permissions (within the Mannequin Internet hosting account) to offer entry to encryption keys:

Entry the IAM console and discover the IAM coverage BedrockCMIExecutionPolicy-777788889999. To the present identity-based coverage, append the next statements (exchange the ARNs in crimson with one famous in steps 4 and 5):

{
    "Impact": "Enable",
    "Motion": [
        "kms:Decrypt",
        "kms:DescribeKey"
    ],
 "Useful resource": "arn:aws:kms:us-east-1:111122223333:key/b5b6e052-fb27-4dbb-bf0d-daf3375a9fda" },
{
    "Impact": "Enable",
    "Motion": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
    ],
 "Useful resource": "arn:aws:kms:us-east-1:777788889999:key/6cd5d3bf-3d9b-4d1c-83d5-8df6284435a1" }

Step 6: Provoke the Mannequin import job (within the Mannequin Internet hosting account)

On this step, we execute the mannequin import job utilizing the AWS Command Line Interface (AWS CLI) command. You may also use AWS SDKs or APIs for a similar function. Run the next command out of your terminal session with an IAM person or function that has the mandatory privileges to create a customized mannequin import job. You don’t must explicitly present an ARN or particulars of the CMK utilized by the Mannequin Improvement workforce.

aws bedrock create-model-import-job 
    --job-name "cmi-job-777788889999-01" 
    --imported-model-name "mistral-777788889999-01" 
    --role-arn "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999" 
    --model-data-source "s3DataSource={s3Uri="s3://model-artifacts-111122223333/mistral-model-weights/"}"

When encrypting mannequin artifacts with Amazon Bedrock Customized Mannequin Import, use the --imported-model-kms-key-id flag and specify the ARN of the Mannequin Internet hosting account’s CMK key.

aws bedrock create-model-import-job 
    --job-name "cmi-job-777788889999-04" 
    --imported-model-name "mistral-777788889999-01" 
    --role-arn "arn:aws:iam::777788889999:function/BedrockCMIExecutionRole-777788889999" 
    --model-data-source "s3DataSource={s3Uri="s3://model-artifacts-111122223333/mistral-model-weights/"}" 
    --imported-model-kms-key-id "arn:aws:kms:us-east-1:777788889999:key/6cd5d3bf-3d9b-4d1c-83d5-8df6284435a1" 

Cross-account entry to the S3 bucket utilizing the customized mannequin import job is simply supported via AWS CLI, AWS SDKs, or APIs. Console assist isn’t but obtainable.

Troubleshooting

When IAM coverage misconfigurations forestall a customized mannequin import job, you may encounter an error like:

Amazon Bedrock doesn't have entry to the S3 location (s3://model-artifacts-111122223333/mistral-model-weights). Replace the permissions and check out once more.

To resolve this, manually confirm entry to Mannequin Improvement’s S3 bucket from the Mannequin Internet hosting account by assuming the BedrockCMIExecutionRole-777788889999. Comply with these steps:

Step 1: Establish the present IAM function or person within the CLI with the next and duplicate the ARN from the output:

aws sts get-caller-identity

Step 2: Replace belief relationships. Append the belief coverage of the BedrockCMIExecutionRole-777788889999 to permit the present person or IAM function to imagine this function:

{
    "Impact": "Enable",
    "Principal": {
        "AWS": "arn:aws:sts::777788889999:function/current-user-role"
    },
    "Motion": "sts:AssumeRole"
}

Step 3: Record or copy the S3 bucket contents assuming the Amazon Bedrock Customized Mannequin Import execution function

1. Assume the CMI execution function (exchange the ARN together with your data):

   aws sts assume-role 
       --role-arn "arn:aws:iam::776941257690:function/BedrockCMIExecutionRole-777788889999" 
       --role-session-name "BedrockCMISession"

2. Export the returned momentary credentials as atmosphere variables:

   export AWS_ACCESS_KEY_ID="ASIA..."
   export AWS_SECRET_ACCESS_KEY="..."
   export AWS_SESSION_TOKEN="..."

3. Run instructions to troubleshoot permission points:

   aws s3 ls s3://model-artifacts-111122223333/mistral-model-weights/
   aws s3 cp s3://model-artifacts-111122223333/mistral-model-weights/config.json . 

If errors persist, think about using Amazon Q Developer or discuss with further sources outlined within the IAM Consumer Information.

Cleanup

There isn’t a further cost to import a customized mannequin to Amazon Bedrock (discuss with step 6 within the Step-by-step execution part). Nonetheless, in case your mannequin isn’t in use for inference, and also you need to keep away from paying storage prices (discuss with Amazon Bedrock pricing), delete the imported mannequin utilizing the AWS console or AWS CLI reference or API Reference. For instance (exchange the textual content in crimson together with your imported mannequin title):

aws bedrock delete-imported-model 
    --model-identifier "mistral-777788889999-01"

Conclusion

Through the use of cross-account entry in Amazon Bedrock Customized Mannequin Import, organizations can considerably streamline their AI mannequin deployment workflows.

Amazon Bedrock Customized Mannequin Import is usually obtainable in the present day in Amazon Bedrock within the US East (N. Virginia) us-east-1 and US West (Oregon) us-west-2 AWS Areas. Check with the full Area checklist for future updates. To be taught extra, discuss with the Amazon Bedrock Customized Mannequin Import product web page and Amazon Bedrock pricing web page. Give Amazon Bedrock Customized Mannequin Import a attempt within the Amazon Bedrock console in the present day and ship suggestions to AWS re:Put up for Amazon Bedrock or via your ordinary AWS Assist contacts.

Thanks to our contributors Scott Chang (Salesforce), Raghav Tanaji (Salesforce), Rupinder Grewal (AWS), Ishan Singh (AWS), and Dharinee Gupta (AWS)

In regards to the Authors

Hrushikesh Gangur is a Principal Options Architect at AWS. Based mostly in San Francisco, California, Hrushikesh is an professional in AWS machine studying. As a thought chief within the discipline of generative AI, Hrushikesh has contributed to AWS’s efforts in serving to startups and ISVs construct and deploy AI purposes. His experience extends to varied AWS companies, together with Amazon SageMaker, Amazon Bedrock, and accelerated computing that are essential for constructing AI purposes.

Sai Darahas Akkineni is a Software program Improvement Engineer at AWS. He holds a grasp’s diploma in Laptop Engineering from Cornell College, the place he labored within the Autonomous Techniques Lab with a specialization in laptop imaginative and prescient and robotic notion. At present, he helps deploy giant language fashions to optimize throughput and latency.

Prashant Patel is a Senior Software program Improvement Engineer in AWS. He’s enthusiastic about scaling giant language fashions for enterprise purposes. Previous to becoming a member of AWS, he labored at IBM on productionizing large-scale AI/ML workloads on Kubernetes. Prashant has a grasp’s diploma from NYU Tandon College of Engineering. Whereas not at work, he enjoys touring and enjoying along with his canines.