【Milvus】记录一次基于milvus-backup做的Milvus备份与恢复

柔情只为你懂 2023-10-13 21:41 91阅读 0赞

#### 文章目录 ####

*  环境
 *  代码准备
 *  备份
 *   *  构建/运行
     *  验证
 *  恢复
 *  遇到的问题

## 环境 ##

milvus：v2.2.4

go：1.20.2 darwin/amd64

milvus-backup：v0.2.2

## 代码准备 ##

https://github.com/zilliztech/milvus-backup/releases

![798eead349a614e326153022ec09567c.png][]

如果你的milvus是2.2.9版本及以上，可以直接下载最新的版本：git clone https://github.com/zilliztech/milvus-backup.git

默认使用的配置文件在config目录下，如果不需要手动指定，直接修改改文件即可，主要修改milvus和minio相关配置即可

![d667d96b2e7b5e0cdda0c6d70563da82.png][]

示例：

# Configures the system log output.
    log:
      level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
      console: true
      file:
        rootPath: "logs/backup.log"
    
    http:
      simpleResponse: true
    
    # milvus proxy address, compatible to milvus.yaml
    milvus:
      address: milvus-t-milvus.milvus.svc.t2.test.xdf.cn
      port: 19530
      authorizationEnabled: false
      # tls mode values [0, 1, 2]
      # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
      tlsMode: 0
      user: xxxx
      password: xxxx
    
    # Related configuration of minio, which is responsible for data persistence for Milvus.
    minio:
      address: minio.openai.svc.t2.test.xdf.cn # Address of MinIO/S3
      port: 80   # Port of MinIO/S3
      accessKeyID: xxxx # accessKeyID of MinIO/S3
      secretAccessKey: xxxx # MinIO/S3 encryption string
      useSSL: false # Access to MinIO/S3 with SSL
      useIAM: false
      cloudProvider: "aws"
      iamEndpoint: ""
    
      bucketName: "milvus-t" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
      rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance
    
      backupBucketName: "milvus-t-backup-all" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath
      backupRootPath: "files" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath
    
    backup:
      maxSegmentGroupSize: 2G

## 备份 ##

### 构建/运行 ###

进入项目根目录，依次执行下面的命令：

go get
    go build

会在根目录下生成可执行文件：

![a0a02c42cebd44ec16d1963e17d6b676.png][]

Zhurunhua-2:milvus-backup-0.2.2 xdf$ ./milvus-backup help
    milvus-backup is a backup tool for milvus.
    
    Usage:
      milvus-backup [flags]
      milvus-backup [command]
    
    Available Commands:
      completion  Generate the autocompletion script for the specified shell
      create      create subcommand create a backup.
      delete      delete subcommand delete backup by name.
      get         get subcommand get backup by name.
      help        Help about any command
      list        list subcommand shows all backup in the cluster.
      restore     restore subcommand restore a backup.
      server      server subcommand start milvus-backup RESTAPI server.
      version     print the version of Milvus backup tool
    
    Flags:
          --config string   config YAML file of milvus (default "backup.yaml")
      -h, --help            help for milvus-backup
    
    Use "milvus-backup [command] --help" for more information about a command.
    Zhurunhua-2:milvus-backup-0.2.2 xdf$

备份需要使用create命令：

![bd367947b33c9b8e7612a5670ee28f60.png][]

示例：指定Collection进行备份

./milvus-backup create --colls tk_td_question_v2 -n tiku_backup

总共19G数据，大概几分钟就完事了

![c721e88ebd28e1515b18a9f14f75aa9e.png][]

### 验证 ###

从minio-console可以看到，对应的备份桶已经创建了，数据也备份好了：

![e135244ab006e96ac91bd404a12f1923.png][]

![ec5c64d7137a7e03ea10a74f3c1d90f7.png][]

binlogs中和源文件一直，meta中会生成一些元数据：

![8f4297aaeef68a911ca30eb14db9e6df.png][]

## 恢复 ##

TODO

## 遇到的问题 ##

我刚开始下载的是最新的源码

1.  this version of sdk is incompatible with server, please downgrade your sdk or upgrade your server

![cf18e28b7266b563fc90c67c06b91dce.png][]

解决：milvus版本过低，与源码所支持的不一致，下载非最新版本的milvus-backup解决。

[798eead349a614e326153022ec09567c.png]: https://img-blog.csdnimg.cn/img_convert/798eead349a614e326153022ec09567c.png
[d667d96b2e7b5e0cdda0c6d70563da82.png]: https://img-blog.csdnimg.cn/img_convert/d667d96b2e7b5e0cdda0c6d70563da82.png
[a0a02c42cebd44ec16d1963e17d6b676.png]: https://img-blog.csdnimg.cn/img_convert/a0a02c42cebd44ec16d1963e17d6b676.png
[bd367947b33c9b8e7612a5670ee28f60.png]: https://img-blog.csdnimg.cn/img_convert/bd367947b33c9b8e7612a5670ee28f60.png
[c721e88ebd28e1515b18a9f14f75aa9e.png]: https://img-blog.csdnimg.cn/img_convert/c721e88ebd28e1515b18a9f14f75aa9e.png
[e135244ab006e96ac91bd404a12f1923.png]: https://img-blog.csdnimg.cn/img_convert/e135244ab006e96ac91bd404a12f1923.png
[ec5c64d7137a7e03ea10a74f3c1d90f7.png]: https://img-blog.csdnimg.cn/img_convert/ec5c64d7137a7e03ea10a74f3c1d90f7.png
[8f4297aaeef68a911ca30eb14db9e6df.png]: https://img-blog.csdnimg.cn/img_convert/8f4297aaeef68a911ca30eb14db9e6df.png
[cf18e28b7266b563fc90c67c06b91dce.png]: https://img-blog.csdnimg.cn/img_convert/cf18e28b7266b563fc90c67c06b91dce.png