Container Storage Interface

The Container Storage Interface (CSI) is a standard that enables you to use compliant block and file storage systems with containerized workloads. Kubernetes provides CSI support starting with v1.13, allowing a containerized CSI driver to interface with Kubernetes controllers in charge of the following:

Dynamic provisioning and deprovisioning of a volume

Attaching or detaching a volume from a node

Mounting/unmounting a volume from a node

Consumption of both block and mountable volumes

Expanding or resizing volumes

Creating and deleting volume point-in-time snapshots

Provisioning new volumes from a snapshot

Note

For more information about Kubernetes CSI, refer to the Kubernetes CSI documentation.

Starting with Diamanti v2.3.0, the default storage class uses the Diamanti CSI driver. In addition, new static volumes created using dctl commands use the CSI driver by default. You should likewise create all new storage classes in your cluster using the Diamanti CSI driver.

Use the Diamanti CSI Driver

This section provides a series of examples of how to use the Diamanti CSI driver to do the following:

Create a custom StorageClass object
Perform dynamic volume provisioning
Perform static volume provisioning
Create a pod with filesystem volumeMode
Create a pod with block volumeMode
Expand or resize a volume
Create a custom SnapshotClass object
Create a snapshot
Provision a new volume from a snapshot

Important: The example specification files used in this section are available at:

/usr/share/diamanti/manifests/examples/csi/

Create a Custom StorageClass Object

This section provides examples of how to create StorageClass objects that use the Diamanti CSI driver to provision volumes dynamically using the supported parameters.

Note

Volumes with a single plex or single data copy are provisioned at the time of pod creation; these volumes are not provisioned when the PVC object is created. This is known as delayed provisioning. For mirrored volumes and volumes with node selector parameters, volume is provisioned instantly during PVC object creation.

Use the sc-delayed-provisioning.yaml file as an example to create a StorageClass object to allow delayed volume provisioning. The following shows the file contents:

apiVersion: storage.k8s.io/v1 kind: StorageClass
metadata:
  name: csi-sc-delayed-provisioning
provisioner: dcx.csi.diamanti.com
reclaimPolicy: Delete
parameters:
  mirrorCount: "1"
  perfTier: "high"
  fsType: "ext4"
allowVolumeExpansion:true

Similarly, use the sc-instant-provisioning.yaml file as an example to create a StorageClass object to allow instant volume provisioning. The following shows the file contents:

apiVersion: storage.k8s.io/v1 kind: StorageClass
metadata:
  name: csi-sc-instant-provisioning
provisioner: dcx.csi.diamanti.com
reclaimPolicy: Delete
parameters:
  mirrorCount: "1"
  perfTier: "high"
  selectors:'{"node":"node1"}’
  fsType: "ext4"
allowVolumeExpansion: true

Note that the following field is required in the StorageClass specification to allow resizing of a PersistentVolumeClaim by the Diamanti CSI driver:

allowVolumeExpansion: true

Use the following kubectl command to create a StorageClass object using the Diamanti CSI driver:

kubectl create -f <storage-class-specification-file>

For example, use the following command to create a StorageClass object that allows instant volume provisioning:

kubectl create -f sc-instant-provisioning.yaml

You can use the following kubectl commands to display storage class information:

$ kubectl get storageclasses.storage.k8s.io
$ kubectl describe storageclasses.storage.k8s.io csi-sc-delayed-provisioning
$ kubectl describe storageclasses.storage.k8s.io csi-sc-instant-provisioning

Perform Dynamic Volume Provisioning

This section shows an example of how to provision volumes dynamically for the filesystem volumeMode with the Diamanti CSI driver. Note that the StorageClass (default or custom) in the PersistentVolumeClaim specification uses the Diamanti CSI driver to perform the dynamic provisioning of the volume.

Use the following PersistentVolumeClaim specification as an example to provision volumes dynamically. The following shows the contents of the pvc-filesystem-delayedprovisioning.yaml file:

apiVersion: v1
kind:PersistentVolumeClaim
metadata:
  name: csi-pvc-filesystem-delayed-provisioning
  spec:
    accessModes:     - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi
    storageClassName: csi-sc-delayed-provisioning

The following shows the contents of the pvc-block-instant-provisioning.yaml file:

apiVersion: v1
kind:PersistentVolumeClaim
metadata:
  name: csi-pvc-block-instant-provisioning
spec:
  accessModes:     - ReadWriteOnce
  volumeMode: Block
  resources:
    requests:
      storage: 5Gi
  storageClassName: csi-sc-instant-provisioning

Use the following kubectl command to create a PVC using the Diamanti CSI driver:

kubectl create -f <pvc-specification-file>

For example, use the following commands to create the respective PersistentVolumeClaims:

$ kubectl create -f pvc-filesystem-delayed-provisioning.yaml

$ kubectl create -f pvc-block-instant-provisioning.yaml

Perform Static Volume Provisioning

The following example shows how to provision a static volume for filesystem usage with the Diamanti CSI driver.

Create the volume using the following command:

dctl volume create <volume-name> -s <size>

For example:

$ dctl volume create pvc-filesystem-static -s 5Gi

(Optional) Display the specification using the following command:

$ dctl -o json volume describe pvc-filesystem-static | jq -r '.spec'
{
  "type": "Static",
  "size": 5368709120,
  "fsType": "ext4",
  "volumeMode": "Filesystem",
  "volumeUser": "default",
  "perfTier": "best-effort",
  "plexCount": 1,
  "thinProvision": true,
  "allocRatio": 100,
  "driver": "csi"
}

3.(Optional) Display information about the persistent volume using the following command:

$ kubectl get pv pvc-filesystem-static -o json | jq -r '.spec'
{
  "accessModes": [
    "ReadWriteOnce"
  ],
  "capacity": {
    "storage": "5Gi"
  },
  "csi": {
    "driver": "dcx.csi.diamanti.com",
    "fsType": "ext4",
    "volumeAttributes": {
      "name": "pvc-filesystem-static",
      "perfTier": "best-effort"
    },
    "volumeHandle": "pvc-filesystem-static"
  },
  "persistentVolumeReclaimPolicy": "Retain",
  "volumeMode": "Filesystem"
}

Create a PersistentVolumeClaim object using the following command:
```
kubectl create -f <pvc-specification-file>
```

For example:

$ kubectl create -f pvc-filesystem-static.yaml

For reference, the contents of the pvc-test-filesystem.yaml file is as follows:

apiVersion: v1
kind:PersistentVolumeClaim
metadata:
  name: csi-pvc-filesystem-static
spec:
  accessModes:  -ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  volumeName: pvc-filesystem-static
  storageClassName: ""

Display the PersistentVolumeClaim object using the following command:

$ kubectl get pvc
NAME                        STATUS   VOLUME                  CAPACITY   ACCESS MODES   STORAGECLASS   AGE
csi-pvc-filesystem-static   Bound    pvc-filesystem-static   5Gi        RWO                           91m

The following example shows how to provision a static volume for raw block device usage with the Diamanti CSI driver.

Create the volume using the following command:

dctl volume create <volume-name> -b -s <size>

For example:

$ dctl volume create pvc-block-static -b -s 5Gi

(Optional) Display the specification using the following command:

$ dctl -o json volume describe pvc-block-static | jq -r '.spec'
{
  "type": "Static",
  "size": 5368709120,
  "volumeMode": "Block",
  "volumeUser": "default",
  "perfTier": "best-effort",
  "plexCount": 1,
  "thinProvision": true,
  "allocRatio": 100,
  "driver": "csi"
}

(Optional) Display information about the persistent volume using the following command:

$ kubectl get pv pvc-testvol-blk -o json | jq -r '.spec'
{
  "accessModes": [
    "ReadWriteOnce"
  ],
  "capacity": {
    "storage": "5Gi"
  },
  "csi": {
    "driver": "dcx.csi.diamanti.com",
    "volumeAttributes": {
      "name": "pvc-block-static",
      "perfTier": "best-effort"
    },
    "volumeHandle": "pvc-block-static"
  },
  "persistentVolumeReclaimPolicy": "Retain",
  "volumeMode": "Block"
}

Create a persistent volume claim using the following command:
```
kubectl create -f <pvc-specification-file>
```

For example:

$ kubectl create -f pvc-block-static.yaml

pvc-block-static.yaml file is as follows:

apiVersion: v1
kind: PersistentVolumeClaim metadata:
  name: csi-pvc-block-static
spec:
  accessModes: -ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Block
  volumeName: pvc-block-static
  storageClassName: ""

Display the persistent volume claim using the following commands:

$ kubectl get pvc
NAME                   STATUS   VOLUME             CAPACITY   ACCESS MODES   STORAGECLASS   AGE
csi-pvc-block-static   Bound    pvc-block-static   5Gi        RWO                           98m

Create a Pod with Filesystem volumeMode

Note

Before performing the procedure in this section, ensure that you first create the PeristentVolumeClaim included in the pod specification.

Create a pod with the persistent volume claim using the following command:
```
kubectl create -f <pod-specification-file>
```

For example:

$ kubectl create -f pod-fs-static.yaml

pod-fs-static.yaml file is as follows:

apiVersion: v1
kind: Pod
metadata:
  name: csi-pod-filesystem-static
spec:
  containers:
  - name: web-server
    image: nginx
    volumeMounts:
    - name: datavol-filesystem-static
      mountPath: /var/lib/www
  volumes:
    - name: datavol-filesystem-static
      persistentVolumeClaim:
        claimName: csi-pvc-filesystem-static
      readOnly: false

Display the pod information using the following command:

$ kubectl get pod
NAME                        READY   STATUS    RESTARTS   AGE
csi-pod-filesystem-static   1/1     Running   0          71s

Display the mount point within the pod using the following command:

$ kubectl exec -it csi-pod-filesystem-static -- df -kh | grep '/var/ lib/www'
/dev/nvme1n1                       4.9G   21M  4.6G   1% /var/lib/www

Create a Pod with Block volumeMode

Create a pod with the persistent volume claim using the following command:
```
kubectl create -f <pod-specification-file>
```

For example:

$ kubectl create -f pod-blk-static.yaml

pod-blk-static.yaml file is as follows:

apiVersion: v1
kind: Pod
metadata:
  name: csi-pod-block-static
spec:
  containers:
  - name: web-server
    image: nginx
    volumeDevices:
    - devicePath: /dev/vdisk1
      name: datavol-block-static
  volumes:
  - name: datavol-block-static
    persistentVolumeClaim:
      claimName: csi-pvc-block-static
      readOnly: false

Display the pod information using the following command:

$ kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
csi-pod-block-static    1/1     Running   0          73m

Display the raw block device within the pod using the following command:

$ kubectl exec -it csi-pod-block-static -- lsblk /dev/vdisk1 NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT nvme1n1 259:0    0   5G  0 disk

Expand or Resize a Volume

Note

Before performing the procedure in this section, ensure that you first create a storageClass object that uses the Diamanti CSI driver as described in Create a Custom StorageClass Object.

To expand or resize a volume, do the following:

Bring down the pod to allow a volume to be resized offline. Use the following kubectl command:
```
kubectl delete pod <pod-name>
```

For example:

$ kubectl delete pod csi-pod-filesystem-instant-provisioning

Expand the PersistentVolumeClaim using the following command:
```
kubectl patch pvc <pvc-name> -p '{ "spec":
```
{ “resources”: { “requests”: { “storage”: “<new-volume-size>” }}}}’

For example:

$ kubectl patch pvc csi-pvc-filesystem-instant-provisioning
-p '{ "spec": { "resources": { "requests": { "storage": "8Gi" }}}}'

Verify the updated PersistentVolumeClaim resource storage capacity using the following command:
```
kubectl get pvc <pvc-name>
```

For example:

$ kubectl get pvc csi-pvc-filesystem-instant-provisioning

Start the pod using the following command:
```
kubectl create -f <pod-name>
```

For example:

$ kubectl create -f pod-fs-instant-provisioning.yaml

Verify the updated filesystem size within the container using the following command:
```
kubectl exec -it <pod-name> -- /bin/bash
```

Note

After resize the targeted node goes into pending state for some time.

Create a Custom SnapshotClass Object

Use the csi-snapshotclass.yaml file as an example to create a SnapshotClass object. The following shows the file contents:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: default-snapclass
  annotations:
    snapshot.storage.kubernetes.io/is-default-class: "true"
driver: dcx.csi.diamanti.com
deletionPolicy: Delete

Use the following kubectl command to create a snapshot class using the Diamanti CSI driver:

$ kubectl create -f csi-snapshotclass.yaml

You can use the following kubectl commands to display snapshot class information:

$ kubectl get volumesnapshotclasses.snapshot.storage.k8s.io
$ kubectl describe volumesnapshotclasses.snapshot.storage.k8s.io csisnapshotclass

Create a Snapshot

This section provide an example of how to create a snapshot.

Before performing the procedure in this section, ensure that the default SnapshotClass object default-snapclass is already created. You can also optionally create a custom SnapshotClass object using the Kubernetes CLI or Kubernetes API, but the Diamanti default-snapclass generally serves all purposes for creating volume snapshots.

Note

Diamanti offers the default-snapclass for creating snapshots in cases when the snapshot specification does not contain a snapshotClassName field.

Use the following command to verify that the default SnapshotClass object defaultsnapclass is already created and exists.

$ kubectl get volumesnapshotclasses.snapshot.storage.k8s.io defaultsnapclass
  NAME          AGE
  default-snapclass   7h24m

The following shows the default SnapshotClass specification file contents:

$ cat /usr/share/diamanti/manifests/system/csisnapshotclass.yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: default-snapclass
  annotations:
    snapshot.storage.kubernetes.io/is-default-class: "true"
driver: dcx.csi.diamanti.com
deletionPolicy: Delete

To create a snapshot, do the following:

Use the pvc-filesystem-snapshot.yaml file as an example to create a snapshot. The following shows the file contents:

$ cat /usr/share/diamanti/manifests/examples/csi/snapshot/filesystem/pvc-filesystem-snapshot.yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: csi-pvc-filesystem-snapshot
spec:
  volumeSnapshotClassName: csi-snapshotclass
  source:

Create a snapshot with the Diamanti CSI driver using the following kubectl command:
```
kubectl create -f <snapshot-specification-file>
```

For example, use the following command to create a snapshot of a PersistentVolumeClaim:

$ kubectl create -f pvc-filesystem-snapshot.yaml

(Optional) Display snapshot information using the following kubectl commands:

$ kubectl get volumesnapshots.snapshot.storage.k8s.io csi-pvcfilesystem-snapshot

$ kubectl describe volumesnapshots.snapshot.storage.k8s.io csi-pvcfilesystem-snapshot

Provision a New Volume from a Snapshot

Use the pvc-filesystem-newvol-lcv.yaml file as an example to create a new

PersistentVolumeClaim object from a volume snapshot. In the following example, the new PVC is created using a snapshot named csi-pvc-filesystem-snapshot as the data source.

The following shows the file contents:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc-filesystem-lcv
  spec:
    storageClassName: csi-sc-instant-provisioning
    dataSource:
      name: csi-pvc-filesystem-snapshot

kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
accessModes:  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Use the following kubectl command to create a new PersistentVolumeClaim object from a snapshot using the Diamanti CSI driver:

kubectl create -f <specification-file>

You can use the following kubectl commands to display the PersistentVolumeClaim object being created using the snapshot:

$ kubectl get pvc csi-pvc-filesystem-lcv

$ kubectl describe pvc csi-pvc-filesystem-lcv