Kubernetes is the de facto standard of container orchestration (deploying workloads on distributed systems). Google Kubernetes Engine (GKE) is the managed Kubernetes as a Service provided by Google Cloud Platform.<\/p>\n
Currently, GKE is still your best choice compared to other managed Kubernetes services, i.e., Azure Container Service (AKS) and Amazon Elastic Container Service for Kubernetes (EKS).<\/p>\n
ref: You could find the sample project on GitHub. Install Install ref: Some useful tools:<\/p>\n ref: Make sure you use the right Google Cloud Platform account.<\/p>\n Create a regional cluster in You can only get a regional cluster by creating a whole new cluster, Google currently won't allow you to turn an existing cluster into a regional one.<\/p>\n ref: Google Kubernetes Engine clusters running Kubernetes version 1.8+ enable Role-Based Access Control (RBAC) by default. Therefore, you must explicitly provide ref: Delete the cluster. After you delete the cluster, you might also need to manually<\/strong> delete persistent disks (under Compute Engine), load balancers (under Network services) and static IPs (under VPC network) which belong to the cluster on Google Cloud Platform Console.<\/p>\n Create a cluster with preemptible VMs which are much cheaper than regular instances using You might receive ref: You could use Google Cloud Build or any Continuous Integration (CI) service to automatically build Docker images and push them to Google Container Registry.<\/p>\n Furthermore, you need to tag your Docker images appropriately with the registry name format: ref: An example of ref: Of course, you could also manually push Docker images to Google Container Registry.<\/p>\n ref: Moreover, you should always adopt Multi-Stage builds for your Dockerfiles.<\/p>\n ref: No, you should never create Pods directly which are so-called naked Pods. Use Deployment instead.<\/p>\n ref: Pods have the following life cycles (states):<\/p>\n ref: Show information about Pods.<\/p>\n ref: Execute a command in a container.<\/p>\n ref: Tail Pod logs. It is also recommended to use ref: List all Pods on a certain node.<\/p>\n Check resource usage.<\/p>\n ref: Restart Pods.<\/p>\n ref: Completely delete resources.<\/p>\n ref: Create an environment-variable-like ConfigMap.<\/p>\n Load environment variables from a ConfigMap:<\/p>\n Create a file-like ConfigMap.<\/p>\n Mount files from a ConfigMap:<\/p>\n ref: Only mount a single file with ref: It is worth noting that changing ConfigMap or Secret won't trigger re-deploying Deployment. A workaround might be changing the name of ConfigMap every time you change the content of ConfigMap. If you mount ConfigMap as environment variables, you must trigger a re-deployment explicitly.<\/p>\n ref: First of all, Secrets are only base64 encoded, not encrypted<\/strong>.<\/p>\n Encode and decode a Secret value.<\/p>\n Create an environment-variable-like Secret.<\/p>\n Export data (base64-encoded) from a Secret.<\/p>\n ref: Deployments are designed for stateless (or nearly stateless) services. Deployment controls ReplicaSet and ReplicaSet controls Pod.<\/p>\n ref: ref: It is also a best practice to always specify resource limits: ref: Create a Deployment with probes.<\/p>\n Create another Deployment of Celery workers.<\/p>\n The minimum value of ref: If multiple Init Containers are specified for a Pod, those Containers are run one at a time in sequential order. Each must succeed before the next can run. When all of the Init Containers have run to completion, Kubernetes initializes regular containers as usual.<\/p>\n ref: TODO<\/p>\n ref: Yes, you could publish a deployment with The formal way.<\/p>\n The git way.<\/p>\n ref: Simply increase the number of ref: The Horizontal Pod Autoscaler automatically scales the number of pods in a Deployment based on observed CPU utilization, memory usage, or custom metrics. Yes, HPA only applies to Deployments and ReplicationControllers.<\/p>\n ref: You could run some load testing.<\/p>\n ref: There is also Cluster Autoscaler in Google Kubernetes Engine.<\/p>\n ref: TODO<\/p>\n ref: PodDisruptionBudgets are only accounted for with voluntary disruptions, something like a hardware failure will not take PodDisruptionBudget into account. PDB cannot prevent involuntary disruptions from occurring, but they do count against the budget.<\/p>\n Create a PodDisruptionBudget for a stateless application.<\/p>\n Create a PodDisruptionBudget for a multiple-instance stateful application.<\/p>\n ref: Actually, you could also have the similar functionality using Which makes sure that ref: A Service is basically a load balancer of a set of Pods which are selected by labels. Since you can't rely on any Pod's IP which changes every time it creates and destroys, you should always provide a Service as an entry point for your Pods or so-called Microservice.<\/p>\n Typically, containers you run in the cluster are not accessible from the Internet, because they do not have external IP addresses. You must explicitly expose your application by creating a Service or an Ingress.<\/p>\n There are the following Service types:<\/p>\n Create a Service.<\/p>\n ref: After a Service is created,
\nhttps:\/\/kubernetes.io\/<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/<\/a><\/p>\n
\nhttps:\/\/github.com\/vinta\/simple-project-on-k8s<\/a><\/p>\nInstallation<\/h2>\n
gcloud<\/code> to create Kubernetes clusters on Google Cloud Platform.<\/p>\nkubectl<\/code> to interact with any Kubernetes cluster.<\/p>\n$ brew install kubernetes-cli\n# or\n$ gcloud components install kubectl\n$ gcloud components update<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/sdk\/docs\/<\/a>
\nhttps:\/\/kubernetes.io\/docs\/tasks\/tools\/install-kubectl\/<\/a><\/p>\n\n
fubectl<\/code>: https:\/\/github.com\/kubermatic\/fubectl<\/a><\/li>\nk9s<\/code>: https:\/\/github.com\/derailed\/k9s<\/a><\/li>\nstern<\/code>: https:\/\/github.com\/wercker\/stern<\/a><\/li>\nzsh-kubectl-prompt<\/code>: https:\/\/github.com\/superbrothers\/zsh-kubectl-prompt<\/a><\/li>\n<\/ul>\nConcepts<\/h2>\n
Nodes<\/h3>\n
\n
Objects<\/h3>\n
\n
Controllers<\/h3>\n
\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/<\/a>
\nhttps:\/\/kubernetes.io\/docs\/reference\/glossary\/?all=true<\/a><\/p>\nSetup Google Cloud Accounts<\/h2>\n
$ gcloud init\n# or\n$ gcloud config configurations list\n$ gcloud config configurations activate default\n$ gcloud config set project simple-project-198818\n$ gcloud config set compute\/region asia-east1\n$ gcloud config set compute\/zone asia-east1-a\n$ gcloud config list<\/code><\/pre>\nCreate Clusters<\/h2>\n
asia-east1<\/code> region which has 1 node in each of the asia-east1<\/code> zones using --region=asia-east1 --num-nodes=1<\/code>. By default, a cluster only creates its cluster master and nodes in a single compute zone.<\/p>\n# show available OSs and versions of Kubernetes\n$ gcloud container get-server-config\n\n# show available CPU platforms in the desired zone\n$ gcloud compute zones describe asia-east1-a\navailableCpuPlatforms:\n- Intel Skylake\n- Intel Broadwell\n- Intel Haswell\n- Intel Ivy Bridge\n\n$ gcloud container clusters create demo \n--cluster-version=1.11.6-gke.6 \n--node-version=1.11.6-gke.6 \n--scopes=gke-default,cloud-platform,storage-full,compute-ro,pubsub,https:\/\/www.googleapis.com\/auth\/cloud_debugger \n--region=asia-east1 \n--num-nodes=1 \n--enable-autoscaling --min-nodes=1 --max-nodes=10 \n--maintenance-window=20:00 \n--machine-type=n1-standard-4 \n--min-cpu-platform=\"Intel Skylake\" \n--enable-ip-alias \n--create-subnetwork=\"\" \n--image-type=UBUNTU \n--node-labels=custom.kubernetes.io\/fs-type=xfs\n\n$ gcloud container clusters describe demo --region=asia-east1\n\n$ kubectl version\nClient Version: version.Info{Major:\"1\", Minor:\"13\", GitVersion:\"v1.13.3\", GitCommit:\"721bfa751924da8d1680787490c54b9179b1fed0\", GitTreeState:\"clean\", BuildDate:\"2019-02-04T04:48:55Z\", GoVersion:\"go1.11.5\", Compiler:\"gc\", Platform:\"darwin\/amd64\"}\nServer Version: version.Info{Major:\"1\", Minor:\"11+\", GitVersion:\"v1.11.5-gke.5\", GitCommit:\"9aba9c1237d9d2347bef28652b93b1cba3aca6d8\", GitTreeState:\"clean\", BuildDate:\"2018-12-11T02:36:50Z\", GoVersion:\"go1.10.3b4\", Compiler:\"gc\", Platform:\"linux\/amd64\"}\n\n$ kubectl get nodes -o wide<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/sdk\/gcloud\/reference\/container\/clusters\/create<\/a>
\nhttps:\/\/cloud.google.com\/compute\/docs\/machine-types<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/concepts\/regional-clusters<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/how-to\/min-cpu-platform<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/how-to\/alias-ips<\/a><\/p>\n--enable-legacy-authorization<\/code> option to disable RBAC.<\/p>\n
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/how-to\/role-based-access-control<\/a><\/p>\n$ gcloud container clusters delete demo --region=asia-east1<\/code><\/pre>\nCreate Node Pools<\/h2>\n
--preemptible<\/code>.<\/p>\nThe connection to the server x.x.x.x was refused - did you specify the right host or port?<\/code> error while upgrading the cluster which includes adding new node pools.<\/p>\n$ gcloud container node-pools create n1-standard-4-pre \n--cluster=demo \n--node-version=1.11.6-gke.6 \n--scopes=gke-default,storage-full,compute-ro,pubsub,https:\/\/www.googleapis.com\/auth\/cloud_debugger \n--region=asia-east1 \n--num-nodes=1 \n--enable-autoscaling --min-nodes=1 --max-nodes=10 \n--machine-type=n1-standard-4 \n--min-cpu-platform=\"Intel Skylake\" \n--node-labels=custom.kubernetes.io\/scopes-storage-full=true\n--enable-autorepair \n--preemptible\n\n$ gcloud container node-pools list --cluster=demo --region=asia-east1\n\n$ gcloud container operations list<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/sdk\/gcloud\/reference\/container\/node-pools\/create<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/concepts\/preemptible-vm<\/a>
\nhttps:\/\/cloud.google.com\/compute\/docs\/regions-zones\/<\/a><\/p>\nBuild Docker Images<\/h2>\n
region_name.gcr.io\/your_project_id\/your_image_name:version<\/code>.<\/p>\n
\nhttps:\/\/cloud.google.com\/container-builder\/<\/a>
\nhttps:\/\/cloud.google.com\/container-registry\/<\/a><\/p>\ncloudbuild.yaml<\/code>:<\/p>\nsubstitutions:\n _REPO_NAME: simple-api\nsteps:\n- id: pull-image\n name: gcr.io\/cloud-builders\/docker\n entrypoint: \"\/bin\/sh\"\n args: [\n \"-c\",\n \"docker pull asia.gcr.io\/$PROJECT_ID\/$_REPO_NAME:$BRANCH_NAME || true\"\n ]\n waitFor: [\n \"-\"\n ]\n- id: build-image\n name: gcr.io\/cloud-builders\/docker\n args: [\n \"build\",\n \"--cache-from\", \"asia.gcr.io\/$PROJECT_ID\/$_REPO_NAME:$BRANCH_NAME\",\n \"--label\", \"git.commit=$SHORT_SHA\",\n \"--label\", \"git.branch=$BRANCH_NAME\",\n \"--label\", \"ci.build-id=$BUILD_ID\",\n \"-t\", \"asia.gcr.io\/$PROJECT_ID\/$_REPO_NAME:$SHORT_SHA\",\n \"simple-api\/\"\n ]\n waitFor: [\n \"pull-image\",\n ]\nimages:\n - asia.gcr.io\/$PROJECT_ID\/$_REPO_NAME:$SHORT_SHA<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/container-builder\/docs\/build-config<\/a>
\nhttps:\/\/cloud.google.com\/container-builder\/docs\/create-custom-build-steps<\/a><\/p>\n$ gcloud auth configure-docker && \ngcloud config set project simple-project-198818 && \nexport PROJECT_ID=\"$(gcloud config get-value project -q)\"\n\n$ docker build --rm -t asia.gcr.io\/${PROJECT_ID}\/simple-api:v1 simple-api\/\n\n$ gcloud docker -- push asia.gcr.io\/${PROJECT_ID}\/simple-api:v1\n\n$ gcloud container images list --repository=asia.gcr.io\/${PROJECT_ID}<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/container-registry\/docs\/pushing-and-pulling<\/a><\/p>\nFROM python:3.6.8-alpine3.7 AS builder\n\nENV PATH=$PATH:\/root\/.local\/bin\nENV PIP_DISABLE_PIP_VERSION_CHECK=1\n\nWORKDIR \/usr\/src\/app\/\n\nRUN apk add --no-cache --virtual .build-deps \n build-base \n linux-headers \n openssl-dev \n zlib-dev\n\nCOPY requirements.txt .\n\nRUN pip install --user -r requirements.txt && \n find $(python -m site --user-base) -type f -name \"*.pyc\" -delete && \n find $(python -m site --user-base) -type f -name \"*.pyo\" -delete && \n find $(python -m site --user-base) -type d -name \"__pycache__\" -delete\n\n###\n\nFROM python:3.6.8-alpine3.7\n\nENV PATH=$PATH:\/root\/.local\/bin\nENV FLASK_APP=app.py\n\nWORKDIR \/usr\/src\/app\/\n\nRUN apk add --no-cache --virtual .run-deps \n ca-certificates \n curl \n openssl \n zlib\n\nCOPY --from=builder \/root\/.local\/ \/root\/.local\/\nCOPY . .\n\nEXPOSE 8000\n\nCMD [\"uwsgi\", \"--ini\", \"config\/uwsgi.ini\", \"--single-interpreter\", \"--enable-threads\", \"--http\", \":8000\"]<\/code><\/pre>\n
\nhttps:\/\/medium.com\/@tonistiigi\/advanced-multi-stage-build-patterns-6f741b852fae<\/a><\/p>\nCreate Pods<\/h2>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/pods\/pod-overview\/<\/a><\/p>\n\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/pods\/pod-lifecycle\/<\/a><\/p>\nInspect Pods<\/h2>\n
$ kubectl get all\n\n$ kubectl get deploy\n\n$ kubectl get pods\n$ kubectl get pods -l app=simple-api\n$ kubectl get pods\n\n$ kubectl describe pod simple-api-5bbf4dd4f9-8b4c9\n$ kubectl get pod simple-api-5bbf4dd4f9-8b4c9 -o yaml<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#describe<\/a>
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#get<\/a><\/p>\n$ kubectl exec -i -t simple-api-5bbf4dd4f9-8b4c9 -- sh<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#exec<\/a><\/p>\nkubetail<\/code>.<\/p>\n$ kubectl logs simple-api-5bbf4dd4f9-8b4c9 -f\n$ kubectl logs deploy\/simple-api -f\n$ kubectl logs statefulset\/mongodb-rs0 -f\n\n$ kubetail simple-api\n$ kubetail simple-worker\n$ kubetail mongodb-rs0 -c db<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#logs<\/a>
\nhttps:\/\/github.com\/johanhaleby\/kubetail<\/a><\/p>\n$ kubectl describe node gke-demo-default-pool-fb33ac26-frkw\n...\nNon-terminated Pods: (7 in total)\n Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits\n --------- ---- ------------ ---------- --------------- -------------\n default mongodb-rs0-1 2100m (53%) 4 (102%) 4G (30%) 4G (30%)\n default simple-api-84554476df-w5b5g 500m (25%) 1 (51%) 1G (16%) 1G (16%)\n default simple-worker-6495b6b74b-rqplv 500m (25%) 1 (51%) 1G (16%) 1G (16%)\n kube-system fluentd-gcp-v3.0.0-848nq 100m (2%) 0 (0%) 200Mi (1%) 300Mi (2%)\n kube-system heapster-v1.5.3-6447d67f78-7psb2 138m (3%) 138m (3%) 301856Ki (2%) 301856Ki (2%)\n kube-system kube-dns-788979dc8f-5zvfk 260m (6%) 0 (0%) 110Mi (0%) 170Mi (1%)\n kube-system kube-proxy-gke-demo-default-pool-3c058fcf-x7cv 100m (2%) 0 (0%) 0 (0%) 0 (0%)\n...\n\n$ kubectl get pods --all-namespaces -o wide --sort-by=\"{.spec.nodeName}\"<\/code><\/pre>\n$ kubectl top pods\n$ kubectl top nodes<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#top<\/a>
\nhttps:\/\/kubernetes.io\/docs\/tasks\/debug-application-cluster\/<\/a><\/p>\n# you could simply kill Pods which would restart automatically if your Pods are managed by any Deployment\n$ kubectl delete pods -l app=simple-worker\n\n# you could replace a resource by providing a manifest\n$ kubectl replace --force -f simple-api\/<\/code><\/pre>\n
\nhttps:\/\/stackoverflow.com\/questions\/40259178\/how-to-restart-kubernetes-pods<\/a><\/p>\n$ kubectl delete -f simple-api\/ -R\n$ kubectl delete deploy simple-api\n$ kubectl delete deploy -l app=simple,role=worker\n\n# delete a Pod forcefully\n$ kubectl delete pod simple-api-668d465985-886h5 --grace-period=0 --force\n$ kubectl delete deploy simple-api --grace-period=0 --force\n\n# delete all resources under a namespace\n$ kubectl delete daemonsets,deployments,services,statefulset,pvc,pv --all --namespace tick<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#delete<\/a><\/p>\nCreate ConfigMaps<\/h2>\n
kind: ConfigMap\napiVersion: v1\nmetadata:\n name: simple-api\ndata:\n FLASK_ENV: production\n MONGODB_URL: mongodb:\/\/mongodb-rs0-0.mongodb-rs0.default.svc.cluster.local,mongodb-rs0-1.mongodb-rs0.default.svc.cluster.local,mongodb-rs0-3.mongodb-rs0.default.svc.cluster.local\/demo?readPreference=secondaryPreferred&maxPoolSize=10\n CACHE_URL: redis:\/\/redis-cache.default.svc.cluster.local\/0\n CELERY_BROKER_URL: redis:\/\/redis-broker.default.svc.cluster.local\/0\n CELERY_RESULT_BACKEND: redis:\/\/redis-broker.default.svc.cluster.local\/1<\/code><\/pre>\nkind: Deployment\napiVersion: apps\/v1\nmetadata:\n name: simple-api\n labels:\n app: simple-api\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: simple-api\n template:\n metadata:\n labels:\n app: simple-api\n spec:\n containers:\n - name: simple-api\n image: asia.gcr.io\/simple-project-198818\/simple-api:4fc4199\n command: [\"uwsgi\", \"--ini\", \"config\/uwsgi.ini\", \"--single-interpreter\", \"--enable-threads\", \"--http\", \":8000\"]\n envFrom:\n - configMapRef:\n name: simple-api\n ports:\n - containerPort: 8000<\/code><\/pre>\nkind: ConfigMap\napiVersion: v1\nmetadata:\n name: redis-cache\ndata:\n redis.conf: |-\n maxmemory-policy allkeys-lfu\n appendonly no\n save \"\"<\/code><\/pre>\nkind: Deployment\napiVersion: apps\/v1\nmetadata:\n name: redis-cache\n labels:\n app: redis-cache\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: redis-cache\n template:\n metadata:\n labels:\n app: redis-cache\n spec:\n volumes:\n - name: config\n configMap:\n name: redis-cache\n containers:\n - name: redis\n image: redis:4.0.10-alpine\n command: [\"redis-server\"]\n args: [\"\/etc\/redis\/redis.conf\", \"--loglevel\", \"verbose\", \"--maxmemory\", \"1g\"]\n volumeMounts:\n - name: config\n mountPath: \/etc\/redis\n ports:\n - containerPort: 6379<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-pod-configmap\/<\/a><\/p>\nsubPath<\/code>.<\/p>\nkind: Deployment\napiVersion: apps\/v1\nmetadata:\n name: redis-cache\n labels:\n app: redis-cache\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: redis-cache\n template:\n metadata:\n labels:\n app: redis-cache\n spec:\n volumes:\n - name: config\n configMap:\n name: redis-cache\n containers:\n - name: redis\n image: redis:4.0.10-alpine\n command: [\"redis-server\"]\n args: [\"\/etc\/redis\/redis.conf\", \"--loglevel\", \"verbose\", \"--maxmemory\", \"1g\"]\n volumeMounts:\n - name: config\n mountPath: \/etc\/redis\/redis.conf\n subPath: redis.conf\n ports:\n - containerPort: 6379<\/code><\/pre>\n
\nhttps:\/\/github.com\/kubernetes\/kubernetes\/issues\/44815#issuecomment-297077509<\/a><\/p>\n
\nhttps:\/\/github.com\/kubernetes\/kubernetes\/issues\/22368<\/a><\/p>\nCreate Secrets<\/h2>\n
$ echo -n 'YOUR_SECRET_KEY' | base64\nWU9VUl9TRUNSRVRfS0VZ\n\n$ echo 'WU9VUl9TRUNSRVRfS0VZ' | base64 --decode\nYOUR_SECRET_KEY<\/code><\/pre>\nkind: Secret\napiVersion: v1\nmetadata:\n name: simple-api\ndata:\n SECRET_KEY: WU9VUl9TRUNSRVRfS0VZ<\/code><\/pre>\n$ kubectl get secret simple-project-com --export=true -o yaml<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/configuration\/secret\/<\/a><\/p>\nCreate Deployments With Probes<\/h2>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/controllers\/deployment\/<\/a><\/p>\nlivenessProbe<\/code> can be used to determine when an application must be restarted by Kubernetes, while readinessProbe<\/code> can be used to determine when a container is ready to accept traffic.<\/p>\n
\nhttps:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-liveness-readiness-probes\/<\/a><\/p>\nresources.requests<\/code> and resources.limits<\/code>.<\/p>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/configuration\/manage-compute-resources-container\/<\/a><\/p>\nkind: Deployment\napiVersion: apps\/v1\nmetadata:\n name: simple-api\n labels:\n app: simple-api\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: simple-api\n template:\n metadata:\n labels:\n app: simple-api\n spec:\n containers:\n - name: simple-api\n image: asia.gcr.io\/simple-project-198818\/simple-api:4fc4199\n command: [\"uwsgi\", \"--ini\", \"config\/uwsgi.ini\", \"--single-interpreter\", \"--enable-threads\", \"--http\", \":8000\"]\n envFrom:\n - configMapRef:\n name: simple-api\n ports:\n - containerPort: 8000\n livenessProbe:\n exec:\n command: [\"curl\", \"-fsS\", \"-m\", \"0.1\", \"-H\", \"User-Agent: KubernetesHealthCheck\/1.0\", \"http:\/\/127.0.0.1:8000\/health\"]\n initialDelaySeconds: 5\n periodSeconds: 1\n successThreshold: 1\n failureThreshold: 5\n readinessProbe:\n exec:\n command: [\"curl\", \"-fsS\", \"-m\", \"0.1\", \"-H\", \"User-Agent: KubernetesHealthCheck\/1.0\", \"http:\/\/127.0.0.1:8000\/health\"]\n initialDelaySeconds: 3\n periodSeconds: 1\n successThreshold: 1\n failureThreshold: 3\n resources:\n requests:\n cpu: 500m\n memory: 1G\n limits:\n cpu: 1000m\n memory: 1G<\/code><\/pre>\nkind: Deployment\napiVersion: apps\/v1\nmetadata:\n name: simple-worker\nspec:\n replicas: 2\n selector:\n matchLabels:\n app: simple-worker\n template:\n metadata:\n labels:\n app: simple-worker\n spec:\n terminationGracePeriodSeconds: 30\n containers:\n - name: simple-worker\n image: asia.gcr.io\/simple-project-198818\/simple-api:4fc4199\n command: [\"celery\", \"-A\", \"app:celery\", \"worker\", \"--without-gossip\", \"-Ofair\", \"-l\", \"info\"]\n envFrom:\n - configMapRef:\n name: simple-api\n readinessProbe:\n exec:\n command: [\"sh\", \"-c\", \"celery inspect -q -A app:celery -d celery@$(hostname) --timeout 10 ping\"]\n initialDelaySeconds: 15\n periodSeconds: 15\n timeoutSeconds: 10\n successThreshold: 1\n failureThreshold: 3\n resources:\n requests:\n cpu: 500m\n memory: 1G\n limits:\n cpu: 1000m\n memory: 1G<\/code><\/pre>\n$ kubectl apply -f simple-api\/ -R\n$ kubectl get pods<\/code><\/pre>\ntimeoutSeconds<\/code> is 1<\/code> so that you might need to use exec.command<\/code> to run arbitrary shell commands with custom timeout settings.<\/p>\n
\nhttps:\/\/cloudplatform.googleblog.com\/2018\/05\/Kubernetes-best-practices-Setting-up-health-checks-with-readiness-and-liveness-probes.html<\/a><\/p>\nCreate Deployments With InitContainers<\/h2>\n
kind: Service\napiVersion: v1\nmetadata:\n name: gcs-proxy-media-simple-project-com\nspec:\n type: NodePort\n selector:\n app: gcs-proxy-media-simple-project-com\n ports:\n - name: http\n port: 80\n targetPort: 80\n---\nkind: ConfigMap\napiVersion: v1\nmetadata:\n name: google-cloud-storage-proxy\ndata:\n nginx.conf: |-\n worker_processes auto;\n\n http {\n include mime.types;\n default_type application\/octet-stream;\n\n server {\n listen 80;\n\n if ( $http_user_agent ~* (GoogleHC|KubernetesHealthCheck) ) {\n return 200;\n }\n\n root \/usr\/share\/nginx\/html;\n open_file_cache max=10000 inactive=10m;\n open_file_cache_valid 1m;\n open_file_cache_min_uses 1;\n open_file_cache_errors on;\n\n include \/etc\/nginx\/conf.d\/*.conf;\n }\n }\n---\napiVersion: apps\/v1\nkind: Deployment\nmetadata:\n name: gcs-proxy-media-simple-project-com\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: gcs-proxy-media-simple-project-com\n template:\n metadata:\n labels:\n app: gcs-proxy-media-simple-project-com\n spec:\n volumes:\n - name: nginx-config\n configMap:\n name: google-cloud-storage-proxy\n - name: nginx-config-extra\n emptyDir: {}\n initContainers:\n - name: create-robots-txt\n image: busybox\n command: [\"sh\", \"-c\"]\n args:\n - |\n set -euo pipefail\n cat << 'EOF' > \/etc\/nginx\/conf.d\/robots.txt\n User-agent: *\n Disallow: \/\n EOF\n volumeMounts:\n - name: nginx-config-extra\n mountPath: \/etc\/nginx\/conf.d\/\n - name: create-nginx-extra-conf\n image: busybox\n command: [\"sh\", \"-c\"]\n args:\n - |\n set -euo pipefail\n cat << 'EOF' > \/etc\/nginx\/conf.d\/extra.conf\n location \/robots.txt {\n alias \/etc\/nginx\/conf.d\/robots.txt;\n }\n EOF\n volumeMounts:\n - name: nginx-config-extra\n mountPath: \/etc\/nginx\/conf.d\/\n containers:\n - name: http\n image: swaglive\/openresty:gcsfuse\n imagePullPolicy: Always\n args: [\"nginx\", \"-c\", \"\/usr\/local\/openresty\/nginx\/conf\/nginx.conf\", \"-g\", \"daemon off;\"]\n ports:\n - containerPort: 80\n securityContext:\n privileged: true\n capabilities:\n add: [\"CAP_SYS_ADMIN\"]\n env:\n - name: GCSFUSE_OPTIONS\n value: \"--debug_gcs --implicit-dirs --stat-cache-ttl 1s --type-cache-ttl 24h --limit-bytes-per-sec -1 --limit-ops-per-sec -1 -o ro,allow_other\"\n - name: GOOGLE_CLOUD_STORAGE_BUCKET\n value: asia.contents.simple-project.com\n volumeMounts:\n - name: nginx-config\n mountPath: \/usr\/local\/openresty\/nginx\/conf\/nginx.conf\n subPath: nginx.conf\n readOnly: true\n - name: nginx-config-extra\n mountPath: \/etc\/nginx\/conf.d\/\n readOnly: true\n readinessProbe:\n httpGet:\n port: 80\n path: \/\n httpHeaders:\n - name: User-Agent\n value: \"KubernetesHealthCheck\/1.0\"\n timeoutSeconds: 1\n initialDelaySeconds: 5\n periodSeconds: 5\n failureThreshold: 1\n successThreshold: 1\n resources:\n requests:\n cpu: 0m\n memory: 500Mi\n limits:\n cpu: 1000m\n memory: 500Mi<\/code><\/pre>\n$ kubectl exec -i -t simple-api-5968cfc48d-8g755 -- sh (gke_simple-project-198818_asia-east1_demo\/default)\n> curl http:\/\/gcs-proxy-media-simple-project-com\/robots.txt\nUser-agent: *\nDisallow: \/<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/pods\/init-containers\/<\/a>
\nhttps:\/\/blog.percy.io\/tuning-nginx-behind-google-cloud-platform-http-s-load-balancer-305982ddb340<\/a><\/p>\nCreate Deployments With Canary Deployment<\/h2>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/cluster-administration\/manage-deployment\/#canary-deployments<\/a>
\nhttps:\/\/medium.com\/google-cloud\/kubernetes-canary-deployments-for-mere-mortals-13728ce032fe<\/a><\/p>\nRollback A Deployment<\/h2>\n
kubectl apply --record<\/code> and rollback it with kubectl rollout undo<\/code>. However, the simplest way might be just git checkout<\/code> the previous commit and deploy again with kubectl apply<\/code>.<\/p>\n$ kubectl apply -f simple-api\/ -R --record\n$ kubectl rollout history deploy\/simple-api\n$ kubectl rollout undo deploy\/simple-api --to-revision=2<\/code><\/pre>\n$ git checkout b7ed8d5\n$ kubectl apply -f simple-api\/ -R\n$ kubectl get pods<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/controllers\/deployment\/#rolling-back-a-deployment<\/a><\/p>\nScale A Deployment<\/h2>\n
spec.replicas<\/code> and deploy again.<\/p>\n$ kubectl apply -f simple-api\/ -R\n# or\n$ kubectl scale --replicas=10 deploy\/simple-api\n\n$ kubectl get pods<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/reference\/generated\/kubectl\/kubectl-commands#scale<\/a>
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/controllers\/deployment\/#scaling-a-deployment<\/a><\/p>\nCreate HorizontalPodAutoscalers (HPA)<\/h2>\n
kind: HorizontalPodAutoscaler\napiVersion: autoscaling\/v2beta1\nmetadata:\n name: simple-api\nspec:\n scaleTargetRef:\n apiVersion: apps\/v1\n kind: Deployment\n name: simple-api\n minReplicas: 2\n maxReplicas: 20\n metrics:\n - type: Resource\n resource:\n name: cpu\n targetAverageUtilization: 80\n - type: Resource\n resource:\n name: memory\n targetAverageValue: 800M\n---\nkind: HorizontalPodAutoscaler\napiVersion: autoscaling\/v2beta1\nmetadata:\n name: simple-worker\nspec:\n scaleTargetRef:\n apiVersion: apps\/v1\n kind: Deployment\n name: simple-worker\n minReplicas: 2\n maxReplicas: 10\n metrics:\n - type: Resource\n resource:\n name: cpu\n targetAverageUtilization: 80\n - type: Resource\n resource:\n name: memory\n targetAverageValue: 500M<\/code><\/pre>\n$ kubectl apply -f simple-api\/hpa.yaml\n\n$ kubectl get hpa --watch\nNAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE\nsimple-api Deployment\/simple-api 18685952\/800M, 4%\/80% 2 20 3 10m\nsimple-worker Deployment\/simple-worker 122834944\/500M, 11%\/80% 2 10 3 10m<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/tasks\/run-application\/horizontal-pod-autoscale\/<\/a>
\nhttps:\/\/kubernetes.io\/docs\/tasks\/run-application\/horizontal-pod-autoscale-walkthrough\/<\/a><\/p>\n
\nhttps:\/\/medium.com\/@jonbcampos\/kubernetes-horizontal-pod-scaling-190e95c258f5<\/a><\/p>\n$ gcloud container clusters update demo \n--enable-autoscaling --min-nodes=1 --max-nodes=10 \n--node-pool=default-pool<\/code><\/pre>\n
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/concepts\/cluster-autoscaler<\/a><\/p>\nCreate VerticalPodsAutoscalers (VPA)<\/h2>\n
\nhttps:\/\/medium.com\/@Mohamed.ahmed\/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231<\/a><\/p>\nCreate PodDisruptionBudget (PDB)<\/h2>\n
\n
kind: PodDisruptionBudget\napiVersion: policy\/v1beta1\nmetadata:\n name: simple-api\nspec:\n minAvailable: 90%\n selector:\n matchLabels:\n app: simple-api<\/code><\/pre>\nkind: PodDisruptionBudget\napiVersion: policy\/v1beta1\nmetadata:\n name: mongodb-rs0\nspec:\n minAvailable: 2\n selector:\n matchLabels:\n app: mongodb-rs0<\/code><\/pre>\n$ kubectl apply -f simple-api\/pdb.yaml\n$ kubectl apply -f mongodb\/pdb.yaml\n\n$ kubectl get pdb\nNAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE\nmongodb-rs0 2 N\/A 1 48m\nsimple-api 90% N\/A 0 48m<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/pods\/disruptions\/<\/a>
\nhttps:\/\/kubernetes.io\/docs\/tasks\/run-application\/configure-pdb\/<\/a><\/p>\n.spec.strategy.rollingUpdate<\/code>.<\/p>\n\n
maxUnavailable<\/code>: The maximum number of Pods that can be unavailable during the update process.<\/li>\nmaxSurge<\/code>: The maximum number of Pods that can be created over the desired number of Pods.<\/li>\n<\/ul>\ntotal ready Pods >= total desired Pods - maxUnavailable<\/code> and total Pods <= total desired Pods + maxSurge<\/code>.<\/p>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/workloads\/controllers\/deployment\/#writing-a-deployment-spec<\/a>
\nhttps:\/\/cloud.google.com\/kubernetes-engine\/docs\/how-to\/updating-apps<\/a><\/p>\nCreate Services<\/h2>\n
\n
ClusterIP<\/code>: A virtual IP which is only reachable from within the cluster. Also, the default Service type.<\/li>\nNodePort<\/code>: It opens a specific port on all Nodes, and any traffic sent to the specific port on any node is forwarded to the Service.<\/li>\nLoadBalancer<\/code>: It builds on NodePorts<\/code> by additionally configuring the cloud provider to create an external load balancer.<\/li>\nExternalName<\/code>: It maps the service to an external CNAME record, i.e., your MySQL RDS on AWS.<\/li>\n<\/ul>\nkind: Service\napiVersion: v1\nmetadata:\n name: simple-api\nspec:\n type: NodePort\n selector:\n app: simple-api\n ports:\n - name: http\n port: 80\n targetPort: 8000<\/code><\/pre>\ntype: NodePorts<\/code> is enough in most cases; spec.selector<\/code> must match labels defined in the corresponding Deployment as the same as spec.ports.targetPort<\/code> and spec.ports.protocol<\/code>.<\/p>\n$ kubectl apply -f simple-api\/ -R\n\n$ kubectl get svc,endpoints\n\n$ kubespy trace service simple-api\n[ADDED v1\/Service] default\/simple-api\n[ADDED v1\/Endpoints] default\/simple-api\n Directs traffic to the following live Pods:\n - [Ready] simple-api-6b4b4c4bfb-g5dln @ 10.28.1.42\n - [Ready] simple-api-6b4b4c4bfb-h66dg @ 10.28.8.24<\/code><\/pre>\n
\nhttps:\/\/kubernetes.io\/docs\/concepts\/services-networking\/service\/<\/a>
\nhttps:\/\/medium.com\/google-cloud\/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0<\/a><\/p>\nkube-dns<\/code> creates a corresponding DNS A record named your-service.your-namespace.svc.cluster.local<\/code> which resolves to an internal IP in the cluster. In ths case: simple-api.default.svc.cluster.local<\/code>. Headless Services (without a cluster IP) are also assigned a DNS A record which has the same form. Unlike normal Services, this A record directly resolves to