How to install minio on Kubernetes

How to install minio on Kubernetes

MinIO is a distributed object storage service for high performance, high scale data infrastructures. It is a drop in replacement for AWS S3 in your own environment. It uses erasure coding to provide highly resilient storage that can tolerate failures of upto n/2 nodes. It runs on cloud, container, kubernetes and bare-metal environments. It is simple enough to be deployed in seconds, and can scale to 100s of peta bytes. MinIO is suitable for storing objects such as photos, videos, log files, backups, VM and container images.

MinIO supports distributed mode. In distributed mode, you can pool multiple drives (even on different machines) into a single object storage server.



There are multiple options to deploy MinIO on Kubernetes:

Helm Chart: MinIO Helm Chart offers customizable and easy MinIO deployment with a single command.

YAML File: MinIO can be deployed with yaml files via kubectl.

MinIO-Operator: Operator creates and manages distributed MinIO deployments running on Kubernetes, using CustomResourceDefinitions and Controller.

Helm Chart:

This chart bootstraps MinIO deployment on a Kubernetes cluster using the Helm package manager.


Prerequisites:

Kubernetes 1.4+ with Beta APIs enabled for default standalone mode.
Kubernetes 1.5+ with Beta APIs enabled to run MinIO in distributed mode.
PV provisioner support in the underlying infrastructure.

Installing the Chart
Install this chart using:

$ helm install stable/minio

The command deploys MinIO on the Kubernetes cluster in the default configuration. The configuration section lists the parameters that can be configured during installation.

Example:

[root@master ~]# helm install stable/minio
NAME:   fashionable-bird
LAST DEPLOYED: Mon Nov 11 03:23:20 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                    DATA  AGE
fashionable-bird-minio  1     0s

==> v1/Deployment
NAME                    READY  UP-TO-DATE  AVAILABLE  AGE
fashionable-bird-minio  0/1    1           0          0s

==> v1/PersistentVolumeClaim
NAME                    STATUS   VOLUME  CAPACITY  ACCESS MODES  STORAGECLASS  AGE
fashionable-bird-minio  Pending  gp2     0s        Filesystem

==> v1/Pod(related)
NAME                                     READY  STATUS   RESTARTS  AGE
fashionable-bird-minio-55fd8bd566-zwz5q  0/1    Pending  0         0s

==> v1/Secret
NAME                    TYPE    DATA  AGE
fashionable-bird-minio  Opaque  2     0s

==> v1/Service
NAME                    TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)   AGE
fashionable-bird-minio  ClusterIP  <Cluster IP>  <none>       9000/TCP  0s

==> v1/ServiceAccount
NAME                    SECRETS  AGE
fashionable-bird-minio  1        0s


NOTES:

Minio can be accessed via port 9000 on the following DNS name from within your cluster:
fashionable-bird-minio.default.svc.cluster.local

Release name

An instance of a chart running in a Kubernetes cluster is called a release. Each release is identified by a unique name within the cluster. Helm automatically assigns a unique release name after installing the chart. You can also set your preferred name by:

$ helm install --name my-release stable/minio

Access and Secret keys
By default a pre-generated access and secret key will be used. To override the default keys, pass the access and secret keys as arguments to helm install.

$ helm install --set accessKey=myaccesskey,secretKey=mysecretkey stable/minio

Uninstalling the Chart
Assuming your release is named as fashionable-bird, delete it using the command:

$ helm delete fashionable-bird

The command removes all the Kubernetes components associated with the chart and deletes the release.

Upgrading the Chart
You can use Helm to update MinIO version in a live release. Assuming your release is named as my-release, get the values using the command:

$ helm get values my-release > old_values.yaml

Then change the field image.tag in old_values.yaml file with MinIO image tag you want to use. Now update the chart using

$ helm upgrade -f old_values.yaml my-release stable/minio

Default upgrade strategies are specified in the values.yaml file.

MinIO-Operator: 

Prerequisites

Kubernetes API v1.15 and above.
kubectl configured to refer to relevant Kubernetes cluster.
Create Operator and related resources
To start MinIO-Operator, use :

kubectl create -f https://github.com/minio/minio-operator/blob/master/minio-operator.yaml?raw=true

This will create all relevant resources required for the Operator to work. Here is a list of resources created by above yaml file:

Namespace: Custom namespace for MinIO-Operator. By default it is named as minio-operator-ns.
CustomResourceDefinition: Custom resource definition named as minioinstances.miniocontroller.min.io.
ClusterRole: A cluster wide role for the controller. It is named as minio-operator-role. This is used for RBAC.
ServiceAccount: Service account is used by the custom controller to access the cluster. Account name by default is minio-operator-sa.
ClusterRoleBinding: This cluster wide binding binds the service account minio-operator-sa to cluster role minio-operator-role.
Deployment: Deployment creates a pod using the MinIO-Operator Docker image. This is where the custom controller runs and looks after any changes in custom resource.


Create a MinIO instance
Once MinIO-Operator deployment is running, you can create MinIO instances using the below command

kubectl create -f https://github.com/minio/minio-operator/blob/master/examples/minioinstance.yaml?raw=true

MinIO Kubernetes YAML Files :

Prerequisites
To run this example, you need Kubernetes version >=1.4 cluster installed and running. You will also need to have installed and configured the kubectl command line tool in your path.


MinIO Standalone Server Deployment
The following section describes the process to deploy standalone MinIO server on Kubernetes. The deployment uses the official MinIO Docker image from Docker Hub.

Standalone Quickstart
Run the below commands to get started quickly

kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-pvc.yaml?raw=true
kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-deployment.yaml?raw=true
kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-service.yaml?raw=true

Create Persistent Volume Claim

MinIO needs persistent storage to store objects. If there is no persistent storage, the data stored in MinIO instance will be stored in the container file system and will be destroyed as soon as the container restarts.

Create a persistent volume claim (PVC) to request storage for the MinIO instance. Kubernetes looks out for PVs matching the PVC request in the cluster and binds it to the PVC automatically. Run the following to create the PersistentVolumeClaim:

kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-pvc.yaml?raw=true
persistentvolumeclaim "minio-pv-claim" created

Create MinIO Deployment

A deployment encapsulates replica sets and pods. If a pod goes down, the replication controller makes sure another pod comes up automatically. This way, you won’t need to bother about pod failures and will have a stable MinIO service available. Create the Deployment using the following command:

kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-deployment.yaml?raw=true
deployment "minio-deployment" created

Create MinIO Service

Now that you have a MinIO deployment running, you may either access it internally (within the cluster) or expose it as a Service onto an external (outside of your cluster, for example public internet) IP address, depending on your use case. You can achieve this using Services. There are 3 major service types—the default type is ClusterIP, which exposes a service to connection from inside the cluster. NodePort and LoadBalancer are two types that expose services to external traffic.

In this example, we expose the MinIO Deployment by creating a LoadBalancer service. Create the MinIO service using the following command

kubectl create -f https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/minio-standalone-service.yaml?raw=true
service "minio-service" created
The LoadBalancer service takes couple of minutes to launch. To check if the service was created successfully, run the following command:

kubectl get svc minio-service
NAME            CLUSTER-IP     EXTERNAL-IP       PORT(S)          AGE
minio-service   10.55.248.23   104.199.249.165   9000:31852/TCP   1m

Standalone Resource cleanup

You can cleanup the cluster using

kubectl delete deployment minio && kubectl delete pvc minio-pv-claim && kubectl delete svc minio-service

Create a Bucket :

Once you open the browser in the Welcome page, you'll find a "+" sign. Click on that "+" sign to create a bucket.