MinioをKubernetes上で動かす

Helm Providerを設定します。

provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
    config_context = "docker-for-desktop"
  }
}

Providerを追加したら terraform init を実行します。

MinioのHelmレシピにはこちらを使います。 https://github.com/helm/charts/tree/master/stable/minio

resource "helm_release" "minio" {
  name = "minio"
  chart = "stable/minio"
}

設定を追加したらplan & applyします。

出力

helm_release.minio: Creating...
  chart:            "" => "stable/minio"
  disable_webhooks: "" => "false"
  force_update:     "" => "false"
  keyring:          "" => "/Users/example/.gnupg/pubring.gpg"
  metadata.#:       "" => "<computed>"
  name:             "" => "minio"
  namespace:        "" => "default"
  recreate_pods:    "" => "false"
  reuse:            "" => "false"
  reuse_values:     "" => "false"
  timeout:          "" => "300"
  verify:           "" => "false"
  version:          "" => "2.4.12"
helm_release.minio: Still creating... (10s elapsed)
helm_release.minio: Still creating... (20s elapsed)
helm_release.minio: Still creating... (30s elapsed)
helm_release.minio: Still creating... (40s elapsed)
helm_release.minio: Still creating... (50s elapsed)
helm_release.minio: Creation complete after 51s (ID: minio)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

正常にapplyできたらどのようなpodが起動しているのか確認してみましょう。

kubectl get pods

NAME                     READY     STATUS    RESTARTS   AGE
minio-798f557bbd-rcpj7   1/1       Running   0          3m

minioのpodが1つ作成されています。podの中身も確認してみましょう。

kubectl describe pods minio-798f557bbd-rcpj7

Name:           minio-798f557bbd-rcpj7
Namespace:      default
Node:           docker-for-desktop/192.168.65.3
Start Time:     Mon, 06 May 2019 18:49:29 +0900
Labels:         app=minio
                pod-template-hash=3549113668
                release=minio
Annotations:    <none>
Status:         Running
IP:             10.1.1.156
Controlled By:  ReplicaSet/minio-798f557bbd
Containers:
  minio:
    Container ID:  docker://f64d9399820e24fd8191cbfbe95cafd4849df6201af6b86f07305508b7f7ac90
    Image:         minio/minio:RELEASE.2019-04-09T01-22-30Z
    Image ID:      docker-pullable://minio/minio@sha256:b363f54fc5a64d259d760106ad02c8725999c935f7aeae5348abfc0bed3fef0d
    Port:          9000/TCP
    Host Port:     0/TCP
    Command:
      /bin/sh
      -ce
      /usr/bin/docker-entrypoint.sh minio -C /root/.minio/ server /export
    State:          Running
      Started:      Mon, 06 May 2019 18:49:37 +0900
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:      250m
      memory:   256Mi
    Liveness:   http-get http://:service/minio/health/live delay=5s timeout=1s period=30s #success=1 #failure=3
    Readiness:  http-get http://:service/minio/health/ready delay=5s timeout=1s period=15s #success=1 #failure=3
    Environment:
      MINIO_ACCESS_KEY:  <set to the key 'accesskey' in secret 'minio'>  Optional: false
      MINIO_SECRET_KEY:  <set to the key 'secretkey' in secret 'minio'>  Optional: false
      MINIO_BROWSER:     on
    Mounts:
      /export from export (rw)
      /root/.minio/ from minio-config-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-7x9pp (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  export:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  minio
    ReadOnly:   false
  minio-user:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  minio
    Optional:    false
  minio-config-dir:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  default-token-7x9pp:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-7x9pp
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age              From                         Message
  ----     ------                 ----             ----                         -------
  Warning  FailedScheduling       3m (x5 over 3m)  default-scheduler            pod has unbound PersistentVolumeClaims
  Normal   Scheduled              3m               default-scheduler            Successfully assigned minio-798f557bbd-rcpj7 to docker-for-desktop
  Normal   SuccessfulMountVolume  3m               kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "pvc-34d029bb-6fe4-11e9-aaf8-025000000001"
  Normal   SuccessfulMountVolume  3m               kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "minio-config-dir"
  Normal   SuccessfulMountVolume  3m               kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "minio-user"
  Normal   SuccessfulMountVolume  3m               kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "default-token-7x9pp"
  Normal   Pulling                3m               kubelet, docker-for-desktop  pulling image "minio/minio:RELEASE.2019-04-09T01-22-30Z"
  Normal   Pulled                 3m               kubelet, docker-for-desktop  Successfully pulled image "minio/minio:RELEASE.2019-04-09T01-22-30Z"
  Normal   Created                3m               kubelet, docker-for-desktop  Created container
  Normal   Started                3m               kubelet, docker-for-desktop  Started container

Podは作成されましたがこのままではホスト側からアクセスできません。service.typeをLoadBalancerに変更してアクセス可能にします。

resource "helm_release" "minio" {
  name = "minio"
  chart = "stable/minio"

  set {
    name  = "service.type"
    value = "LoadBalancer"
  }
}

設定を変更したらplan & applyします。

helm_release.minio: Refreshing state... (ID: minio)
helm_release.minio: Modifying... (ID: minio)
  set.#:                "0" => "1"
  set.2699829193.name:  "" => "service.type"
  set.2699829193.value: "" => "LoadBalancer"
helm_release.minio: Modifications complete after 3s (ID: minio)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

serviceの状態を見てみます。

kubectl get svc

NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP      10.96.0.1        <none>        443/TCP          1h
minio        LoadBalancer   10.103.204.219   localhost     9000:30168/TCP   59m

ポート番号の9000が開いていることがわかります。

Webブラウザで http://localhost:9000 でアクセスするとMinioの画面を表示できます。

今回はlocalで起動するだけでしたので Access Key と Secret Key は設定しませんでした。設定しないとデフォルトの値が使われます。使われるデフォルト値は https://github.com/helm/charts/tree/master/stable/minio#configuration の Access Key と Secret Key のところに記載されています。この値は自分で生成した適切な値を使うようにしましょう。

awscliをインストールします。

pip3 install awscli

Requirement already satisfied: awscli in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (1.16.152)
Requirement already satisfied: s3transfer<0.3.0,>=0.2.0 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (0.2.0)
Requirement already satisfied: botocore==1.12.142 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (1.12.142)
Requirement already satisfied: rsa<=3.5.0,>=3.1.2 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (3.4.2)
Requirement already satisfied: PyYAML<=3.13,>=3.10 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (3.13)
Requirement already satisfied: colorama<=0.3.9,>=0.2.5 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (0.3.9)
Requirement already satisfied: docutils>=0.10 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from awscli) (0.14)
Requirement already satisfied: urllib3<1.25,>=1.20; python_version >= "3.4" in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from botocore==1.12.142->awscli) (1.24.2)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from botocore==1.12.142->awscli) (0.9.4)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from botocore==1.12.142->awscli) (2.8.0)
Requirement already satisfied: pyasn1>=0.1.3 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from rsa<=3.5.0,>=3.1.2->awscli) (0.4.5)
Requirement already satisfied: six>=1.5 in /srv/example/var/pyvenvs/py37/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore==1.12.142->awscli) (1.12.0)

一時的にminioを試すだけなので、awscliで使う ACCESS_KEY と SECRET_KEY は環境変数に設定します。

(setenv "AWS_ACCESS_KEY_ID" MINIO-ACCESS-KEY)
(setenv "AWS_SECRET_ACCESS_KEY" MINIO-SECRET-KEY)

awscliを使ってバケットを作成します。

aws --endpoint-url http://localhost:9000 s3 mb s3://testing

make_bucket: testing

バケットの一覧を確認してみます。

aws --endpoint-url http://localhost:9000 s3 ls

2019-05-06 20:03:17 testing

先程作成した testing というバケットが作成されていることがわかります。

• https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html
• https://dev.classmethod.jp/tool/s3-compatible-storage-minio/