K8S原理剖析：存储原理剖析和实践

男娘i 2023-06-27 06:22 101阅读 0赞

# 大纲 #

>  *  K8S存储整体框架及原理解析
>  *  K8S FlexVolume存储扩展机制
>  *  K8S CSI存储扩展机制

# K8S存储整体框架及原理解析 #

## Docker插件机制-架构&评价 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70][]

> **优点：**
> 
>  *  1）不重新编译docker的情况下可以提供针对鉴权、网络、卷等功能的扩展；
>  *  2）基于Http JSON-PRC格式的接口与插件进行交互；
>  *  3）支持容器化和非容器化多种部署形式；
>  *  4）支持插件生命周期管理： 1.52+  docker plugin命令及其API； <1.52  docker volume/network；
>  *  5）支持基于TLS的安全加固。
> 
> **约束：**
> 
>  *  某些插件（认证）的添加需要重启docker daemon

### Docker Volume Plugin列表 ###

<table> 
 <tbody> 
  <tr> 
   <td>名称</td> 
   <td>描述</td> 
   <td>地址</td> 
  </tr> 
  <tr> 
   <td>Azure File Storage<br> plugin</td> 
   <td>Lets you mount Microsoft Azure File Storage shares to Docker containers as volumes using the SMB 3.0 protocol. Learn<br> more.</td> 
   <td>https://github.com/Azure/azurefile<br> dockervolumedriver</td> 
  </tr> 
  <tr> 
   <td>BeeGFS Volume<br> Plugin</td> 
   <td>An open source volume plugin to create persistent volumes in a BeeGFS parallel file system.</td> 
   <td>https://github.com/RedCoolBeans/<br> docker-volume-beegfs</td> 
  </tr> 
  <tr> 
   <td>Blockbridge plugin</td> 
   <td>A volume plugin that provides access to an extensible set of container-based persistent storage options. It supports single<br> and multi-host Docker environments with features that include tenant isolation, automated provisioning, encryption,<br> secure deletion, snapshots and QoS.</td> 
   <td>https://github.com/blockbridge/blo<br> ckbridge-docker-volume</td> 
  </tr> 
  <tr> 
   <td>Contiv Volume<br> Plugin</td> 
   <td>An open source volume plugin that provides multi-tenant, persistent, distributed storage with intent based consumption. It<br> has support for Ceph and NFS.</td> 
   <td>https://github.com/rancher/convoy</td> 
  </tr> 
  <tr> 
   <td>DigitalOcean Block<br> Storage plugin</td> 
   <td>Integrates DigitalOcean’s block storage solution into the Docker ecosystem by automatically attaching a given block storage<br> volume to a DigitalOcean droplet and making the contents of the volume available to Docker containers running on that<br> droplet.</td> 
   <td>https://github.com/omallo/docker<br> volume-plugin-dostorage</td> 
  </tr> 
  <tr> 
   <td>DRBD plugin</td> 
   <td>A volume plugin that provides highly available storage replicated by DRBD. Data written to the docker volume is replicated<br> in a cluster of DRBD nodes.</td> 
   <td>https://www.drbd.org/en/supporte<br> d-projects/docker</td> 
  </tr> 
  <tr> 
   <td>Flocker plugin</td> 
   <td>A volume plugin that provides multi-host portable volumes for Docker, enabling you to run databases and other stateful<br> containers and move them around across a cluster of machines.</td> 
   <td>https://clusterhq.com/docker<br> plugin/</td> 
  </tr> 
  <tr> 
   <td>Fuxi Volume Plugin</td> 
   <td>A volume plugin that is developed as part of the OpenStack Kuryr project and implements the Docker volume plugin API by<br> utilizing Cinder, the OpenStack block storage service.</td> 
   <td>https://github.com/openstack/fuxi</td> 
  </tr> 
  <tr> 
   <td>gce-docker plugin</td> 
   <td>A volume plugin able to attach, format and mount Google Compute persistent-disks.</td> 
   <td>https://github.com/mcuadros/gce<br> docker</td> 
  </tr> 
  <tr> 
   <td>GlusterFS plugin</td> 
   <td>A volume plugin that provides multi-host volumes management for Docker using GlusterFS.</td> 
   <td>https://github.com/calavera/docker<br> -volume-glusterfs</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <tbody> 
  <tr> 
   <td>名称</td> 
   <td>描述</td> 
   <td>地址</td> 
  </tr> 
  <tr> 
   <td>Horcrux Volume<br> Plugin</td> 
   <td>A volume plugin that allows on-demand, version controlled access to your data. Horcrux is an open-source plugin,<br> written in Go, and supports SCP, Minio and Amazon S3.</td> 
   <td>https://github.com/muthu-r/horcrux</td> 
  </tr> 
  <tr> 
   <td>HPE 3Par Volume<br> Plugin</td> 
   <td>A volume plugin that supports HPE 3Par and StoreVirtual iSCSI storage arrays.</td> 
   <td>https://github.com/hpe storage/python-hpedockerplugin/ -</td> 
  </tr> 
  <tr> 
   <td>Infinit volume<br> plugin</td> 
   <td>A volume plugin that makes it easy to mount and manage Infinit volumes using Docker.</td> 
   <td>https://infinit.sh/documentation/dock er/volume-plugin</td> 
  </tr> 
  <tr> 
   <td>IPFS Volume<br> Plugin</td> 
   <td>An open source volume plugin that allows using an ipfs filesystem as a volume.</td> 
   <td>http://github.com/vdemeester/docker -volume-ipfs</td> 
  </tr> 
  <tr> 
   <td>Keywhiz plugin</td> 
   <td>A plugin that provides credentials and secret management using Keywhiz as a central repository.</td> 
   <td>https://github.com/calavera/docker<br> volume-keywhiz</td> 
  </tr> 
  <tr> 
   <td>Local Persist<br> Plugin</td> 
   <td>A volume plugin that extends the default local driver’s functionality by allowing you specify a mountpoint anywhere<br> on the host, which enables the files to always persist, even if the volume is removed via docker volume rm.</td> 
   <td>https://github.com/CWSpear/local<br> persist</td> 
  </tr> 
  <tr> 
   <td>NetApp<br> Plugin(nDVP)</td> 
   <td>A volume plugin that provides direct integration with the Docker ecosystem for the NetApp storage portfolio. The<br> nDVP package supports the provisioning and management of storage resources from the storage platform to Docker<br> hosts, with a robust framework for adding additional platforms in the future.</td> 
   <td>https://github.com/NetApp/netappdv<br> p</td> 
  </tr> 
  <tr> 
   <td>Netshare plugin</td> 
   <td>A volume plugin that provides volume management for NFS 3/4, AWS EFS and CIFS file systems.</td> 
   <td>https://github.com/ContainX/docker<br> volume-netshare</td> 
  </tr> 
  <tr> 
   <td>Nimble Storage<br> Volume Plugin</td> 
   <td>A volume plug-in that integrates with Nimble Storage Unified Flash Fabric arrays. The plug-in abstracts array volume<br> capabilities to the Docker administrator to allow self-provisioning of secure multi-tenant volumes and clones.</td> 
   <td>https://connect.nimblestorage.com/co<br> mmunity/app-integration/docker</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <tbody> 
  <tr> 
   <td>名称</td> 
   <td>描述</td> 
   <td>地址</td> 
  </tr> 
  <tr> 
   <td>OpenStorage Plugin</td> 
   <td>A cluster-aware volume plugin that provides volume management for file and block storage solutions. It implements a<br> vendor neutral specification for implementing extensions such as CoS, encryption, and snapshots. It has example drivers<br> based on FUSE, NFS, NBD and EBS to name a few.</td> 
   <td>https://github.com/libopenstorage/<br> openstorage</td> 
  </tr> 
  <tr> 
   <td>Portworx Volume Plugin</td> 
   <td>A volume plugin that turns any server into a scale-out converged compute/storage node, providing container granular<br> storage and highly available volumes across any node, using a shared-nothing storage backend that works with any docker<br> scheduler.</td> 
   <td>https://github.com/portworx/px<br> dev</td> 
  </tr> 
  <tr> 
   <td>Quobyte Volume Plugin</td> 
   <td>A volume plugin that connects Docker to Quobyte’s data center file system, a general-purpose scalable and fault-tolerant<br> storage platform.</td> 
   <td>https://github.com/quobyte/docker<br> -volume</td> 
  </tr> 
  <tr> 
   <td>REX-Ray plugin</td> 
   <td>A volume plugin which is written in Go and provides advanced storage functionality for many platforms including VirtualBox,<br> EC2, Google Compute Engine, OpenStack, and EMC.</td> 
   <td>https://github.com/emccode/rexray</td> 
  </tr> 
  <tr> 
   <td>Virtuozzo Storage and<br> Ploop plugin</td> 
   <td>A volume plugin with support for Virtuozzo Storage distributed cloud file system as well as ploop devices.</td> 
   <td>https://github.com/virtuozzo/docke r-volume-ploop</td> 
  </tr> 
  <tr> 
   <td>VMware vSphere Storage<br> Plugin</td> 
   <td>Docker Volume Driver for vSphere enables customers to address persistent storage requirements for Docker containers in<br> vSphere environments.</td> 
   <td>https://github.com/vmware/docker<br> volume-vsphere</td> 
  </tr> 
 </tbody> 
</table>

### 支持Kubernetes的平台和存储服务 ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 1][]

## K8S 存储能力-Volume概述 ##

>  *  K8S中的普通Volume提供了在容器中挂卷的能力，它不是独立的K8S资源对象，不能通过k8s去管理（创建、删除等），只能在创建Pod时去引用。
>  *  Pod需要设置卷来源（ spec.volume ） 和挂载点（ spec.containers.volumeMounts ） 两个信息后才可以使用相应的Volume

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 2][]

### K8S 存储能力： In-Tree Volume Plugins ###

> K8S的VolumePlugin提供了插件化扩展存储的机制，分为内置插件（In-Tree Plugins）和外置插件（Out-of-Tree） 两种

<table> 
 <tbody> 
  <tr> 
   <td>名称</td> 
   <td>描述</td> 
  </tr> 
  <tr> 
   <td>awsElasticBlockStore</td> 
   <td>mounts an Amazon Web Services (AWS) EBS Volume (Elastic Block Store)</td> 
  </tr> 
  <tr> 
   <td>azureDisk</td> 
   <td>is used to mount a Microsoft Azure Data Disk into a Pod.</td> 
  </tr> 
  <tr> 
   <td>azureFile</td> 
   <td>is used to mount a Microsoft Azure File Volume (SMB 2.1 and 3.0) into a Pod.</td> 
  </tr> 
  <tr> 
   <td>cephfs</td> 
   <td>allows an existing CephFS volume to be mounted into your pod.</td> 
  </tr> 
  <tr> 
   <td>cinder</td> 
   <td>is used to mount OpenStack Block Storage into a pod.</td> 
  </tr> 
  <tr> 
   <td>configMap</td> 
   <td>The data stored in a ConfigMap object can be referenced in a volume of type configMap and then consumed by containerized applications running in a Pod.</td> 
  </tr> 
  <tr> 
   <td>downwardAPI</td> 
   <td>is used to make downward API data available to applications. It mounts a directory and writes the requested data in plain text files</td> 
  </tr> 
  <tr> 
   <td>emptyDir</td> 
   <td>is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. When a Pod is removed from a node for any reason, the<br> data in the emptyDir is deleted forever.</td> 
  </tr> 
  <tr> 
   <td>fc (fibre channel)</td> 
   <td>allows an existing fibre channel volume to be mounted in a pod</td> 
  </tr> 
  <tr> 
   <td>flocker</td> 
   <td>allows a Flocker dataset to be mounted into a pod.</td> 
  </tr> 
  <tr> 
   <td>gcePersistentDisk</td> 
   <td>mounts a Google Compute Engine (GCE) Persistent Disk into your pod.</td> 
  </tr> 
  <tr> 
   <td>gitRepo</td> 
   <td>mounts an empty directory and clones a git repository into it for your pod to use.</td> 
  </tr> 
  <tr> 
   <td>glusterfs</td> 
   <td>allows a Glusterfs (an open source networked filesystem) volume to be mounted into your pod</td> 
  </tr> 
  <tr> 
   <td>hostPath</td> 
   <td>mounts a file or directory from the host node’s filesystem into your pod.</td> 
  </tr> 
  <tr> 
   <td>iscsi</td> 
   <td>allows an existing iSCSI (SCSI over IP) volume to be mounted into your pod</td> 
  </tr> 
  <tr> 
   <td>local</td> 
   <td>represents a mounted local storage device such as a disk, partition or directory. can only be used as a statically created PersistentVolume.</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <tbody> 
  <tr> 
   <td>名称</td> 
   <td>描述</td> 
  </tr> 
  <tr> 
   <td>nfs</td> 
   <td>allows an existing NFS (Network File System) share to be mounted into your pod</td> 
  </tr> 
  <tr> 
   <td>persistentVolumeClaim</td> 
   <td>is used to mount a PersistentVolume into a pod.</td> 
  </tr> 
  <tr> 
   <td>projected</td> 
   <td>maps several existing volume sources into the same directory.</td> 
  </tr> 
  <tr> 
   <td>portworxVolume</td> 
   <td>can be dynamically created through Kubernetes or it can also be pre-provisioned and referenced inside a Kubernetes pod.</td> 
  </tr> 
  <tr> 
   <td>quobyte</td> 
   <td>allows an existing Quobyte volume to be mounted into your pod.</td> 
  </tr> 
  <tr> 
   <td>rbd</td> 
   <td>allows a Rados Block Device volume to be mounted into your pod.</td> 
  </tr> 
  <tr> 
   <td>scaleIO</td> 
   <td>ScaleIO is a software-based storage platform that can use existing hardware to create clusters of scalable shared block networked storage. The ScaleIO<br> volume plugin allows deployed pods to access existing ScaleIO volumes</td> 
  </tr> 
  <tr> 
   <td>secret</td> 
   <td>is used to pass sensitive information, such as passwords, to pods</td> 
  </tr> 
  <tr> 
   <td>storageos</td> 
   <td>allows an existing StorageOS volume to be mounted into your pod. StorageOS provides block storage to containers, accessible via a file system.</td> 
  </tr> 
  <tr> 
   <td>vsphereVolume</td> 
   <td>used to mount a vSphere VMDK Volume into your Pod.</td> 
  </tr> 
 </tbody> 
</table>

## K8S 存储能力-PersistentVolume ##

> Kubernetes通过Persistent Volume子系统API对管理员和用户提供了存储资源创建和使用的抽象

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 3][]

>  *  FlexVolume: 此Volume Driver允许不同厂商去开发他们自己的驱动来挂载卷到计算节点
>  *  PersistentVolumeClaim： K8提供的资源抽象的Volume Driver，让用户不用关心具体的Volume的实现细节

# K8S FlexVolume存储扩展机制 #

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 4][]

> K8S 1.5引入1.8GA的Out-Of-Tree Volume Plugin：
> 
>  *  driver以二进制命令行形式实现FlexVolume API，以供Controller-Manager和Kubelet调用，对外接口实现容易；
>  *  DaemonSet方式部署确保Master和Node上都会将driver安装到插件目录；
>  *  Docker镜像+yaml配置的交付形式

## Flex Volume Driver部署脚本和配置 ##

> 部署脚本假设所需要的驱动二进制文件，且已经被至于部署镜像的/$DRIVER目录下：启动脚本将二进制driver重名名为.driver，再拷贝到<plugindir>/<vendor~driver>/.driver目录下，接着使用mv将其重命名为driver（确保驱动安装的原子性），最后进入死循环确保容器活着。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 5][]

## Flex Volume CLI API ##

<table> 
 <tbody> 
  <tr> 
   <td>步骤</td> 
   <td>命令</td> 
   <td>描述</td> 
  </tr> 
  <tr> 
   <td>Init</td> 
   <td>&lt;driver executable&gt; init</td> 
   <td>初始化驱动。在Kubelet和Controller-Manager初始化时被调用。若调用成功则需要返回一个展示对应驱动<br> 所支持的FlexVolume能力的map，现在只包含一个必填字段attach，用于表明本驱动是否需要attach和<br> detach操作。为向后兼容该字段一般默认值设为true。</td> 
  </tr> 
  <tr> 
   <td>Attach</td> 
   <td>&lt;driver executable&gt; attach &lt;json options&gt; &lt;node<br> name&gt;</td> 
   <td>将给定规格的卷添加到给定的主机上。若调用成功则返回存储设备添加到该主机的路径。 Kubelet和<br> Controller-Manager都需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Detach</td> 
   <td>&lt;driver executable&gt; detach &lt;mount device&gt; &lt;node<br> name&gt;</td> 
   <td>卸载给定主机上的指定卷。 Kubelet和Controller-Manager都需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Wait for attach</td> 
   <td>&lt;driver executable&gt; waitforattach &lt;mount device&gt;<br> &lt;json options&gt;</td> 
   <td>等待卷被添加到远程节点。若调用成功则将返回设备路径。 Kubelet和Controller-Manager都需要调用该方<br> 法。</td> 
  </tr> 
  <tr> 
   <td>Volume is<br> Attached</td> 
   <td>&lt;driver executable&gt; isattached &lt;json options&gt; &lt;node<br> name&gt;</td> 
   <td>检查卷是否已被添加到节点上。 Kubelet和Controller-Manager都需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Mount device</td> 
   <td>&lt;driver executable&gt; mountdevice &lt;mount dir&gt; &lt;mount<br> device&gt; &lt;json options&gt;</td> 
   <td>将存储设备挂载到一个将被pod使用的全局路径上。 Kubelet需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Unmount device</td> 
   <td>&lt;driver executable&gt; unmountdevice &lt;mount device&gt;</td> 
   <td>将存储设备卸载。 This is called once all bind mounts have been unmounted. Kubelet需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Mount</td> 
   <td>&lt;driver executable&gt; mount &lt;mount dir&gt; &lt;json options&gt;</td> 
   <td>将卷挂载到指定目录。 Kubelet需要调用该方法。</td> 
  </tr> 
  <tr> 
   <td>Unmount</td> 
   <td>&lt;driver executable&gt; unmount &lt;mount dir&gt;</td> 
   <td>将卷进行卸载。 Kubelet需要调用该方法。</td> 
  </tr> 
 </tbody> 
</table>

## K8S CSI存储扩展机制 ##

<table> 
 <tbody> 
  <tr> 
   <td>术语</td> 
   <td>含义</td> 
  </tr> 
  <tr> 
   <td>CO</td> 
   <td>容器编排系统（Container Orchestrator），使用CSI gRPC服务来与插件通信</td> 
  </tr> 
  <tr> 
   <td>RPC</td> 
   <td>远程方法调用（Remote Procedure Call）</td> 
  </tr> 
  <tr> 
   <td>Plugin</td> 
   <td>插件实现，实现CSI服务的gRPC访问端点</td> 
  </tr> 
  <tr> 
   <td>SP</td> 
   <td>存储提供商（Storage Provider），负责提供CSI插件实现</td> 
  </tr> 
  <tr> 
   <td>Volume</td> 
   <td>卷， CO管理的容器可使用的存储单元</td> 
  </tr> 
  <tr> 
   <td>Block Volume</td> 
   <td>块设备卷</td> 
  </tr> 
  <tr> 
   <td>Mounted Volume</td> 
   <td>使用指定文件系统挂载到容器的卷，并显示为容器内的一个目录</td> 
  </tr> 
  <tr> 
   <td>Workload</td> 
   <td>工作负载，是CO任务调度的基本单元，可以是一个或一组容器</td> 
  </tr> 
  <tr> 
   <td>Node</td> 
   <td>用户运行工作负载的主机，从插件的角度通过节点 ID来进行唯一标识</td> 
  </tr> 
  <tr> 
   <td>In-Tree</td> 
   <td>内置的，存在于K8S核心代码仓库内的代码</td> 
  </tr> 
  <tr> 
   <td>Out-Of-Tree</td> 
   <td>外置的，存在于K8S核心代码仓库外的代码</td> 
  </tr> 
  <tr> 
   <td>CSI Volume Plugin</td> 
   <td>一个新的内置卷插件，作为一个适配器来使得外置的第三方CSI卷驱动可以被K8S所使用</td> 
  </tr> 
  <tr> 
   <td>CSI Volume Driver</td> 
   <td>一个外置的CSI兼容的卷插件驱动，可通过K8S卷插件被K8S所使用</td> 
  </tr> 
 </tbody> 
</table>

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 6][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 7][]

## CSI通用架构 ##

> CO通过gRPC与插件交互，每个SP必须实现以下两个plugin：  
> • Node Plugin： 需要运行在使用Volume的Node上，主要负责Volume Mount/Unmount等操作  
> • Controller Plugin：可以运行在任何节点上，主要负责Volume Creation/Deletion、 Attach/Detach等操作

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 8][]

## CO与Plugin的交互： 01.卷的创建和Attach ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 9][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 10][]

> • 卷的Mount系列操作是由Workload的启动所触发的  
> • K8S中volume的全局挂载路径（存储挂载点）格式样例：  
> /var/lib/kubelet/plugins/kubernetes.io/$volume\_plugin/mounts/$volume\_name  
> • K8S中volume的workload挂载路径（软链）格式样例：  
> /var/lib/kubelet/pods/$pod\_id/volumes/$volume\_plugin/$volume\_name

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 11][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 12][]

## 卷的生命周期 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 13][]

## RPC接口集合-Identity ##

> CSI规范定义了3类RPC集合：  
> • Identity Service： Node Plugin和Controller Plugin都需要实现的RPC集合  
> • Controller Service： Controller Plugin需要实现的RPC集合  
> • Node Service： Node Plugin需要实现的RPC集合  
> Identity Service RPC：身份服务RPC允许CO查询插件的功能，健康状况和其他元数据。

![2020010421330767.png][]

## RPC接口集合-Controller ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 14][]

> Controller Service RPC： 控制服务RPC提供卷的创建、删除、 Attach、 Detach、查询等功能，以及卷快照的创建、删除、查询等功能

## RPC接口集合-Node ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 15][]

## K8S CSI架构 ##

> K8S 1.9实现了CSI plugin alpha版本， 1.11版本已升至Beta

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 16][]

> 为了部署一个容器化的第三方CSI volume  
> driver，存储提供商需要执行如下操作：  
> 1. 创建一个实现CSI规范描述的插件功能，并通过Unix套接字来暴露gPRC访问接口的”CSI volume driver” 容器；  
> 2. 结合使用K8S团队提供的帮助容器来部署CSI volume driver，具体需要创建如下两类K8S对象：
> 
>  *  1） StatefulSet：用于与K8S控制器进行交互，实例数1，包含3个容器（ CSI volume driver、 external-attacher 、 externalprovisioner ），需要挂载一个挂载点为/var/lib/csi/sockets/pluginproxy/的emptyDir volume
>  *  2） DaemonSet ：包含2个容器（ CSI volumedriver、 K8S CSI Helper），挂载3个hostpath volume
> 
> 3. 集群管理员为存储系统在K8S集群中部署上述StatefulSet和DaemonSet

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200104211150876.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200104211538535.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200104211721508.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/2020010421192352.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200104212109860.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200104212244266.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200104212628789.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20200104212705889.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20200104212836861.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20200104212910341.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20200104212953107.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 11]: https://img-blog.csdnimg.cn/20200104213110416.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 12]: https://img-blog.csdnimg.cn/20200104213141481.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 13]: https://img-blog.csdnimg.cn/20200104213207555.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[2020010421330767.png]: https://img-blog.csdnimg.cn/2020010421330767.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 14]: https://img-blog.csdnimg.cn/20200104213342430.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 15]: https://img-blog.csdnimg.cn/20200104213608795.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ_size_16_color_FFFFFF_t_70 16]: https://img-blog.csdnimg.cn/20200104213654213.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZseTkxMDkwNQ==,size_16,color_FFFFFF,t_70