Save greenbacks on Google Container Engine using autoscaling and preemptible VMs






There is an awesome new feature on Google Container Engine (GKE) that lets you combine autoscaling, node pools and preemptible VMs to save big $!


The basic idea is to create a small cluster with an inexpensive VM type that will run 7x24. This primary node can be used for critical services that should not be rescheduled to another pod. A good example would be a Jenkins master server. Here is an example of how to create the cluster:

gcloud alpha container clusters create $CLUSTER \
  --network "default" --num-nodes 1 \
  --machine-type  ${small} --zone $ZONE \
  --disk-size 50

Now here is the money saver trick:  A second node pool is added to the cluster. This node pool is configured to auto-scale from one node up to a maximum. This additional node pool uses preemptible VMs. These are VMs that can be taken away at any time if Google needs the capacity, but in exchange you get dirt cheap images. For example, running a 4 core VM with 15GB of RAM for a month comes in under $30.

This second pool is perfect for containers that can survive a restart or migration to a new node. Jenkins slaves would be a good candidate.

Here is an example of adding the node pool to the cluster you created above:

gcloud alpha container node-pools create $NODEPOOL --cluster $CLUSTER --zone $ZONE \
    --machine-type ${medium} --preemptible --disk-size 50 \
    --enable-autoscaling --min-nodes=1 --max-nodes=4

That node pool will scale down to a single VM if the cluster is not busy, and scale up to a maximum of 4 nodes.

If your VM gets preempted (and it will at least once every 24 hours),  the pods running on that node will be rescheduled onto a new node created by the auto-scaler.

Container engine assigns a label to nodes which you can use for scheduling. For example, to ensure you Jenkins Master does not get put on a preemptible node, you can add the following annotation to your Pod Spec:

apiVersion: v1kind: Podspec:  nodeSelector:    !cloud.google.com/gke-preemptible

apiVersion: v1kind: Podspec:  nodeSelector:    !cloud.google.com/gke-preemptible
nodeSelector:    !cloud.google.com/gke-preemptible

See https://cloud.google.com/container-engine/docs/preemptible-vm for the details.


Comments

Post a Comment

Popular posts from this blog

Introducing ds-operator, the ForgeRock Directory Services Operator for Kubernetes

Image
ForgeRock Directory Services 7.0 was a big achievement for the Grenoble Directory team.  It is the only "Kubernetes native" directory where you can add a new replica using kubectl: kubectl scale sts/ds-idrepo --replicas=3 The 7.0 deployment is assembled using standard Kubernetes primitives such as StatefulSets , Persistent Volume Claims, and Services.  This is all built and orchestrated using Skaffold and Kustomize .  An emerging pattern in the Kubernetes world is the use of Custom Resources and Operators .  Broadly speaking, a custom resource is the declaration of the desired system state, and the operator's job is to observe the current state and bring the system into alignment with the declared state: source: https://blog.container-solutions.com/kubernetes-operators-explained The Kubernetes API server (the thing that responds to your kubectl commands) can be extended to handle new custom types.  A custom resource definition (CRD) describes to the API se...
Read more

OAM R2 REST APIs for Policy Management

Image
Oracle Access Manager 11g R2 provides several new REST APIs. This continues a trend to expose key functionality via Web Services. The OAM Mobile and Social service provides APIs for Authentication, Authorization  and User Profile services.  I will cover those APIs in a future article (have a look here for examples) - but today I want to focus on the  policy management APIs. The Policy Administration API  enables to you to interact with OAM to create a variety of Policy objects such as Application Domains, Resources, AuthN Schemes, and AuthN/AuthZ policies. The policy model is shown below: For example, if you want to retrieve all of the resources in an Application Domain you can perform a GET against the /resource URI: curl -u USER:PASSWORD http://<SERVER>:<PORT>/oam/services/rest/11.1.2.0.0/ssa/policyadmin/resource?appdomain="IAM Suite" Note: The port above is where the OAM Admin Server is deployed (often 7001). It ...
Read more

Automating OpenDJ backups on Kubernetes

Image
Kubernetes   StatefulSets  are designed to run "pet" like services such as databases.   ForgeRock's OpenDJ LDAP server is an excellent fit for StatefulSets as it requires stable network identity and persistent storage. The ForgeOps project contains a Kubernetes Helm chart to deploy DJ to a Kubernetes cluster. Using a StatefulSet, the cluster will auto-provision persistent storage for our pod. We configure OpenDJ to place its backend database on this storage volume. This gives us persistence that survives container restarts, or even restarts of the cluster. As long as we don't delete the underlying persistent volume, our data is safe. Persistent storage is quite reliable, but we typically want additional offline backups. The high level approach to accomplish this is as follows: Configure the OpenDJ container to support scheduled backups to a volume. Configure a Kubernetes volume to store the backups. Create a sidecar container that archives the backups...
Read more