通过 IAM 角色配置 S3 访问

本主题介绍在使用 helm 安装 Milvus 时如何通过 IAM 角色配置 s3 访问。更多详细信息，请参考 IAM roles。

开始之前

使用 eksctl 创建 EKS 集群时，请启用 OIDC。更多详细信息，请参考 IAM OIDC。
本指南假设您已经在 AWS 上使用 Kubernetes 部署了 Milvus 集群。

将 IAM 角色与 Kubernetes 服务账户关联

创建 AWS S3 存储桶。

阅读存储桶命名规则并在命名 AWS S3 存储桶时遵守命名规则。

milvus_bucket_name="milvus-bucket-$(openssl rand -hex 12)"

aws s3api create-bucket --bucket "$milvus_bucket_name" --region 'us-east-2' --acl private  --object-ownership ObjectWriter --create-bucket-configuration LocationConstraint='us-east-2'


# Output
#
# "Location": "http://milvus-bucket-039dd013c0712f085d60e21f.s3.amazonaws.com/"

为上面创建的存储桶中的对象创建读写 IAM 策略。请将存储桶名称替换为您自己的。

echo '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>/*"
      ]
    }
  ]
}' > milvus-s3-policy.json

aws iam create-policy --policy-name MilvusS3ReadWrite --policy-document file://milvus-s3-policy.json


# Get the ARN from the command output as follows:
# {
#     "Policy": {
#         "PolicyName": "MilvusS3ReadWrite",
#         "PolicyId": "AN5QQVVPM1BVTFlBNkdZT",
#         "Arn": "arn:aws:iam::12345678901:policy/MilvusS3ReadWrite",
#         "Path": "/",
#         "DefaultVersionId": "v1",
#         "AttachmentCount": 0,
#         "PermissionsBoundaryUsageCount": 0,
#         "IsAttachable": true,
#         "CreateDate": "2023-11-16T06:00:01+00:00",
#        "UpdateDate": "2023-11-16T06:00:01+00:00"
#     }
# }    

创建 IAM 角色并将其与 Kubernetes 服务账户关联。将 your-account-id 替换为您的账户 ID。

eksctl create iamserviceaccount --name milvus-s3-access-sa --namespace milvus --cluster milvus-eks-cluster --role-name milvus-s3-access-sa \
    --attach-policy-arn arn:aws:iam::<your-account-id>:policy/MilvusS3ReadWrite --approve

验证角色和服务账户设置

请参考 IAM roles。

确认 IAM 角色的信任策略配置正确。

aws iam get-role --role-name milvus-s3-access-sa --query Role.AssumeRolePolicyDocument
# An example output is as follows
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:default:my-service-account",
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

确认您在前面步骤中附加到角色的策略已附加到角色。

aws iam list-attached-role-policies --role-name milvus-s3-access-sa --query 'AttachedPolicies[].PolicyArn' --output text
# An example output is as follows
arn:aws:iam::12345678901:policy/MilvusS3ReadWrite

查看策略的默认版本。

export policy_arn='arn:aws:iam::12345678901:policy/MilvusS3ReadWrite'
aws iam get-policy --policy-arn $policy_arn
# An example output is as follows
{
    "Policy": {
        "PolicyName": "MilvusS3ReadWrite",
        "PolicyId": "EXAMPLEBIOWGLDEXAMPLE",
        "Arn": "arn:aws:iam::12345678901:policy/MilvusS3ReadWrite",
        "Path": "/",
        "DefaultVersionId": "v2",
        [...]
    }
}

查看策略内容以确保策略包含您的 Pod 需要的所有权限。如有必要，将以下命令中的 1 替换为前面输出中返回的版本。

aws iam get-policy-version --policy-arn $policy_arn --version-id v2
# An example output is as follows
{
    "PolicyVersion": {
        "Document": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [
                        "s3:GetObject",
                        "s3:PutObject",
                        "s3:ListBucket",
                        "s3:DeleteObject"
                    ],
                    "Resource": [
                        "arn:aws:s3:::<bucket-name>",
                        "arn:aws:s3:::<bucket-name>/*"
                    ]
                }
            ]
        },
        [...]
    }
}

确认 Kubernetes 服务账户已用角色进行注释。

kubectl describe serviceaccount milvus-s3-access-sa -n milvus
# An example output is as follows
Name:                milvus-s3-access-sa
Namespace:           milvus
Labels:              app.kubernetes.io/managed-by=eksctl
Annotations:         eks.amazonaws.com/role-arn: arn:aws:iam::12345678901:role/milvus-s3-access-sa
[...]

部署 Milvus

在本指南中，我们将使用 Milvus Helm Charts 部署 Milvus 集群。您可以在这里找到这些图表。

添加 Milvus Helm Chart 仓库。

helm repo add milvus https://zilliztech.github.io/milvus-helm/
helm repo update

准备 Milvus 配置文件 milvus.yaml，并将 <bucket-name> 替换为上面创建的存储桶名称。

cluster:
  enabled: true

service:
  type: LoadBalancer
  port: 19530
  annotations: 
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-name: milvus-service
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip

serviceAccount:
  create: false
  name: milvus-s3-access-sa

minio:
  enabled: false

externalS3:
  enabled: true
  host: "s3.us-east-2.amazonaws.com"
  port: "443"
  useSSL: true
  bucketName: "<bucket-name>"
  useIAM: true
  cloudProvider: "aws"
  iamEndpoint: ""

rootCoordinator:
  replicas: 2
  activeStandby:
    enabled: true
  resources: 
    limits:
      cpu: 1
      memory: 2Gi

indexCoordinator:
  replicas: 2
  activeStandby:
    enabled: true
  resources: 
    limits:
      cpu: "0.5"
      memory: 0.5Gi

queryCoordinator:
  replicas: 2
  activeStandby:
    enabled: true
  resources: 
    limits:
      cpu: "0.5"
      memory: 0.5Gi

dataCoordinator:
  replicas: 2
  activeStandby:
    enabled: true
  resources: 
    limits:
      cpu: "0.5"
      memory: 0.5Gi

proxy:
  replicas: 2
  resources: 
    limits:
      cpu: 1
      memory: 2Gi  

安装 Milvus。

helm upgrade --install milvus-demo milvus/milvus -n milvus -f milvus.yaml

验证安装

请参考验证安装。

开始之前​

将 IAM 角色与 Kubernetes 服务账户关联​

验证角色和服务账户设置​

部署 Milvus​

验证安装​

开始之前

将 IAM 角色与 Kubernetes 服务账户关联

验证角色和服务账户设置

部署 Milvus

验证安装