kubernetes-external-secrets allows you to use external secret management systems, like AWS Secrets Manager, to securely add Secrets in Kubernetes, so Pods can access Secrets normally.
ref:
https://github.com/external-secrets/kubernetes-external-secrets
AWS Secret Manager
For instance, we create a secret named YOUR_SECRET on AWS Secret Manager in the same region as our EKS cluster, using DefaultEncryptionKey as the encryption key. The content of the secret entity look like:
{
"KEY_1": "VALUE_1",
"KEY_2": "VALUE_2",
}We can retrieve the secret value:
aws secretsmanager get-secret-value --profile=perp \
--region ap-northeast-1 \
--secret-id YOUR_SECRETkubernetes-external-secrets
For kubernetes-external-secrets to work properly, it must be granted access to AWS Secrets Manager. To achieve that, we need to create an IAM role for kubernetes-external-secrets' service account.
ref:
https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
Configure Secrets Backends
Create an IAM OIDC provider for the cluster:
eksctl utils associate-iam-oidc-provider --profile=perp \
--region ap-northeast-1 \
--cluster perp-staging \
--approve
aws iam list-open-id-connect-providers --profile=perpref:
https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
Create an IAM policy that allows the role to access all secrets we created on AWS Secret Manager:
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --profile=perp --query "Account" --output text)
cat <<EOF > policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
"Resource": [
"arn:aws:secretsmanager:ap-northeast-1:${AWS_ACCOUNT_ID}:secret:*"
]
}
]
}
EOF
aws iam create-policy --profile=perp \
--policy-name perp-staging-secrets-policy --policy-document file://policy.jsonAttach the above IAM policy to an IAM role, and define AssumeRole for the service account external-secrets-kubernetes-external-secrets which will be created later:
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --profile=perp --query "Account" --output text)
OIDC_PROVIDER=$(aws eks describe-cluster --profile=perp --name perp-staging --region ap-northeast-1 --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
cat <<EOF > trust.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDC_PROVIDER}:aud": "sts.amazonaws.com",
"${OIDC_PROVIDER}:sub": "system:serviceaccount:default:external-secrets-kubernetes-external-secrets"
}
}
}
]
}
EOF
aws iam create-role --profile=perp \
--role-name perp-staging-secrets-role \
--assume-role-policy-document file://trust.json
aws iam attach-role-policy --profile=perp \
--role-name perp-staging-secrets-role \
--policy-arn YOUR_POLICY_ARNref:
https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
https://gist.github.com/lukaszbudnik/f1f42bd5a57430e3c25034200ba44c2e
Deploy kubernetes-external-secrets Controller
helm repo add external-secrets https://external-secrets.github.io/kubernetes-external-secrets/
helm install external-secrets \
external-secrets/kubernetes-external-secrets \
--skip-crds \
--set env.AWS_REGION=ap-northeast-1 \
--set securityContext.fsGroup=65534 \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"='YOUR_ROLE_ARN'
helm listIt would automatically create a service account named external-secrets-kubernetes-external-secrets in Kubernetes.
Deploy ExternalSecret
ExternalSecret app-secrets will generate a Secret object with the same name, and the content would look like:
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: example-secret
spec:
backendType: secretsManager
region: ap-northeast-1
dataFrom:
- YOUR_SECRET
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: example-app
spec:
replicas: 3
selector:
matchLabels:
app: example-app
template:
metadata:
labels:
app: example-app
spec:
containers:
- name: example-app
image: busybox:latest
envFrom:
- secretRef:
name: example-secretkubectl get secret example-secret -o jsonpath="{.data.KEY_1}" | base64 --decoderef:
https://gist.github.com/lukaszbudnik/f1f42bd5a57430e3c25034200ba44c2e
