Infinidat Storage for Kubernetes
Before containerization, goods were handled as bulk cargo. Handling multiple pieces led to many delays that in turn made transportation costly and unreliable.
When the first international standard for containers was established, it helped to streamline goods transportation worldwide, and significantly reduced the time, lowered the cost, and significantly improved reliability. When I moved halfway around the world, I used a container to ship all my belongings. I can’t imagine how it could be done without a container.
My Entire House in a Container
Containerization of compute workloads allows similar streamlining of code shipment. A developer can now package an application along with all of the required dependencies into a single software container, which in turn can be shipped and deployed anywhere, whether it is a physical or a virtual host, running on-premises or in the cloud.
Containers are one of the hottest trends in the IT industry and have brought together some solutions for runtime environment management, image repositories, orchestration environments and more. One of these ecosystems, Kubernetes, originally developed at Google is one of the most popular container frameworks.
At INFINIDAT, we noticed the growing interest in containers among our customers. Originally, it was with stateless applications – running web servers, and other similar frontend environments which had less demand for persistent storage. But lately, we’re seeing more customers moving other applications into containerized environments: Kafka, Elastic or MySQL. All of them require reliable, fast storage – lots of it. Petabytes of storage, something we know a thing or three about.
How can an application that requires persistent storage consume it in the containerized environment? Kubernetes introduced a special entity: a pod, which may include one or more tightly-coupled containers and other resources required by the application such as Persistent Volumes (PVs):
Containers and PVs within a Pod on a Kubernetes Worker Node
In earlier Kubernetes versions, a Storage Administrator had to pre-provision static Persistent Volumes (PVs) with predefined attributes – such as size or type of access. The developer had to specify a Persistent Volume Claim (PVC) as part of his pod definition. When the Kubernetes scheduler tried to deploy a pod on one of the worker nodes, it had to match an existing, un-attached PV with the PVC:
Matching an existing, unattached PV with the PVC
While static provisioning is still used, a new concept of StorageClasses and Dynamic Volumes provisioning had been introduced. A StorageClass provides a way for Storage Administrators to describe the profiles of storage available to Developers. Different classes might map to quality-of-service levels, backup policies, or to any other arbitrary policy determined by the Storage Administrators.
Storage classes define a provisioner that determines which volume plugin is used for provisioning PVs. The provisioner would create a PV as requested by the developer via PV claim, offering just-in-time right-size provisioning:
Just-in-Time Right-sized Provisioning
Kubernetes supports several access modes for PVs:
- ReadWriteOnce (RWO) – the volume can be mounted as read-write by a single node
- ReadOnlyMany (ROX) – the volume can be mounted read-only by many nodes
- ReadWriteMany (RWX) – the volume can be mounted as read-write by many nodes
INFINIDAT provisioner supports FC, iSCSI and NFS access to PVs. Each PV’s access modes are set to the specific modes supported by that particular connection:
The INFINIDAT Dynamic Volume Provisioner for Kubernetes is now generally available via code.infinidat.com. It allows our customers to fully leverage InfiniBox functionality in their containerized environments. InfiniBox’s flexible RESTful API allows our provisioner to automatically update PVs when new worker nodes are added or removed. For example, for NFS PVs, NFS exports are updated automatically to ensure secure access to PVs only from the specific cluster nodes.
Following is a sample storage class referring to the InfiniBox NFS PV provisioner:
$ cat nfs_class.yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: ibox-nfs provisioner: infinidat.com/nfs parameters: nfs_networkspace: nas pool_name: k8s nfs_mount_options: hard,rsize=1048576,wsize=1048576 provision_type: THIN nfs_export_permissions: RW ssd_enabled: "true" no_root_squash: "true" reclaimPolicy: Delete
The administrator can create this storage class by executing the following command:
$ kubectl create -f nfs_class.yaml storageclass "ibox-nfs" created
Now the software developer can create a PVC referring to this class:
$ cat nfs_claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: nfs-pvc spec: accessModes: - ReadWriteMany storageClassName: ibox-nfs resources: requests: storage: 5Gi $ kubectl create -f nfs-claim.yaml persistentvolumeclaim "nfs-pvc" created
InfiniBox NFS Storage Class and PVC as shown in Kubernetes UI
When a new PVC is created in the Kubernetes environment referring to the InfiniBox storage class, the provisioner creates a filesystem or a volume with the attributes specified in PVC. Upon the successful creation of PV by the provisioner, Kubernetes will bind it to the PVC and make the storage available for the running pod. When a PVC gets deleted, a PV might also be removed – subject to the reclaim policy.
Our Dynamic PV provisioner can be used with Kubernetes 1.6 or above. INFINIDAT has also joined the Red Hat OpenShift Commons and OpenShift Primed programs to ensure the INFINIDAT k8s persistent volume provisioner is fully supported in RedHat OpenShift environments.
Specifying a PV claim for InfiniBox provisioner using OpenShift UI
Are you running Kubernetes using persistent volumes today? I’ll be glad to hear more about your field experience and discuss how our dynamic provisioning functionality could be improved in the future.