【分布式云储存】Springboot微服务接入MinIO实现文件服务

爱被打了一巴掌 2024-02-29 04:17 68阅读 0赞

#### 文章目录 ####

*   *  前言
     *  技术回顾
     *  准备工作
     *   *  申请accessKey\\secretKey
         *  创建数据存储桶
         *  公共资源直接访问测试
     *  接入springboot实现文件服务
     *   *  依赖引入
         *  配置文件
         *  MinIO配置
         *  MinIO工具类
     *  OkHttpSSLSocketClient兼容ssl
     *  静态资源预览解决方案
     *  资源上传预览测试
     *  测试结果

### 前言 ###

上篇博客我们介绍了[分布式云存储MinIO作业环境的搭建][MinIO]，以及分布式云储存MinIO在实际的文件服务中的优势。那么，今天我们就小试牛刀来将MinIO接入我们的微服务项目，实现一个分布式的文件服务器。

### 技术回顾 ###

MinIO 提供高性能、与S3 兼容的对象存储系统，让你自己能够构建自己的私有云储存服务。  
MinIO原生支持 Kubernetes，它可用于每个独立的公共云、每个 Kubernetes 发行版、私有云和边缘的对象存储套件。  
MinIO是软件定义的，不需要购买其他任何硬件，在 GNU AGPL v3 下是 100% 开源的。

### 准备工作 ###

#### 申请accessKey\\secretKey ####

MinIO控制台申请accessKey\\secretKey  
http://your\_hostname:18001~18004/login minio/minio123  
![在这里插入图片描述][c160c66374904397b7c6b71e46ca81de.png]

#### 创建数据存储桶 ####

创建数据存储桶  
![在这里插入图片描述][29bbf5c7703c43059a6eac87b1b31ac4.png]![在这里插入图片描述][f2f30ecfb3e04c03ad16478764c14d30.png]

点击创建的桶我们进行访问策略配置  
![在这里插入图片描述][5a79251cd04248ba9adb1c3bc073d587.png]  
![在这里插入图片描述][984013365c75428fb98133cd577dec26.png]

访问策略有private\\custom\\public  
public 公共的桶，任何人都可以访问资源，直接映射为静态资源，可直接提供预览和下载  
custom 自定义桶，用户根据自身需求定义访问规则  
private 私有的桶，需要授权才能访问

根据一般的生产需求，我们定义一个private，一个custom桶。private保存私有资源，custom保存公共资源并禁止目录访问。  
![在这里插入图片描述][2a253ba6f52c47f88074b23d5085d9e2.png]

private 规则：  
![在这里插入图片描述][d7da0f6c58ac484a8586ac3a963cf3f4.png]

custom 规则：  
![在这里插入图片描述][e10837a1e6244df7a7583d5e5812987e.png]

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "*"
                    ]
                },
                "Action": [
                    "s3:GetBucketLocation",
                    "s3:ListBucketMultipartUploads"
                ],
                "Resource": [
                    "arn:aws:s3:::sacpublic"
                ]
            },
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "*"
                    ]
                },
                "Action": [
                    "s3:AbortMultipartUpload",
                    "s3:DeleteObject",
                    "s3:GetObject",
                    "s3:ListMultipartUploadParts",
                    "s3:PutObject"
                ],
                "Resource": [
                    "arn:aws:s3:::sacpublic/*"
                ]
            }
        ]
    }

#### 公共资源直接访问测试 ####

私有资源代码博文最后测试  
公共资源直接访问测试：  
![在这里插入图片描述][d11c6bbd2c3e4fa99f90106689d46721.png]  
![在这里插入图片描述][7b18d7ba9d354a12a1f5d165161a7b84.png]

### 接入springboot实现文件服务 ###

#### 依赖引入 ####

maven引入依赖

<dependency>
        <groupId>io.minio</groupId>
        <artifactId>minio</artifactId>
        <version>8.2.0</version>
    </dependency>

#### 配置文件 ####

配置文件

minio.url = https://10.10.22.91:9100
    minio.accessKey = fUIXbkBZ9UQTHOOZXNGW
    minio.secretKey = sy1RAgItAOk9pk1gE7FbrPYzsZI87CfpGkuoY0KW
    minio.buckets.public = sacpublic
    minio.buckets.private = sacprivate

注意：接口调用minio url 为nginx映射出来的9000端口  
http://your\_hostname:9000

#### MinIO配置 ####

MinIO配置

#### MinIO工具类 ####

MinIO工具类

/**
     * MinioUtil
     * @author senfel
     * @version 1.0
     * @date 2023/9/14 10:26
     */
    @Component
    @ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true)
    public class MinioUtil {
    
        @Resource
        private MinioClient minioClient;
    
        /**
         * 创建一个桶
         */
        public void createBucket(String bucket) throws Exception {
            boolean found = minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
            if (!found) {
                minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build());
            }
        }
    
        /**
         * 上传一个文件
         */
        public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception {
            String prefix = objectName.substring(objectName.lastIndexOf(".") + 1);
            //静态资源预览解决方案
            //String contentType = ViewContentType.getContentType(prefix);
            ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName)
                    .contentType(contentType)
                    .stream(stream, -1, 10485760).build());
            return objectWriteResponse;
        }
    
        /**
         * 列出所有的桶
         */
        public List<String> listBuckets() throws Exception {
            List<Bucket> list = minioClient.listBuckets();
            List<String> names = new ArrayList<>();
            list.forEach(b -> {
                names.add(b.name());
            });
            return names;
        }
    
        /**
         * 下载一个文件
         */
        public InputStream download(String bucket, String objectName) throws Exception {
            InputStream stream = minioClient.getObject(
                    GetObjectArgs.builder().bucket(bucket).object(objectName).build());
            return stream;
        }
    
        /**
         * 删除一个桶
         */
        public void deleteBucket(String bucket) throws Exception {
            minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucket).build());
        }
    
        /**
         * 删除一个对象
         */
        public void deleteObject(String bucket, String objectName) throws Exception {
            minioClient.removeObject(RemoveObjectArgs.builder().bucket(bucket).object(objectName).build());
        }
    
        /**
         * 复制文件
         *
         * @Param: [sourceBucket, sourceObject, targetBucket, targetObject]
         * @return: void
         * @Date: 2021/11/15
         */
        public void copyObject(String sourceBucket, String sourceObject, String targetBucket, String targetObject) throws Exception {
            this.createBucket(targetBucket);
            minioClient.copyObject(CopyObjectArgs.builder().bucket(targetBucket).object(targetObject)
                    .source(CopySource.builder().bucket(sourceBucket).object(sourceObject).build()).build());
        }
    
        /**
         * 获取文件信息
         *
         * @Param: [bucket, objectName]
         * @return: java.lang.String
         */
        public String getObjectInfo(String bucket, String objectName) throws Exception {
            return minioClient.statObject(StatObjectArgs.builder().bucket(bucket).object(objectName).build()).toString();
        }
    
        /**
         * 生成一个给HTTP GET请求用的presigned URL。浏览器/移动端的客户端可以用这个URL进行下载，即使其所在的存储桶是私有的。
         * @Param: [bucketName, objectName, expires]
         * @return: java.lang.String
         */
        public String getPresignedObjectUrl(String bucketName, String objectName, Integer expires) throws Exception {
            GetPresignedObjectUrlArgs build = GetPresignedObjectUrlArgs
                    .builder().bucket(bucketName).object(objectName).expiry(expires).method(Method.GET).build();
            return minioClient.getPresignedObjectUrl(build);
        }
    
    }

### OkHttpSSLSocketClient兼容ssl ###

对于改解决方案一般生产环境都有固定的域名和匹配的ssl证书，如果也不用https我们代码则不用兼容ssl。但是如果我们配置了不可信任的ssl，这里我们则需要进行ssl兼容方案。

/**
     * MinioConfig
     * @author senfel
     * @version 1.0
     * @date 2023/9/14 11:37
     */
    @Configuration
    @ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true)
    public class MinioConfig {
    
        @Value("${minio.url}")
        private String minioUrl;
        @Value("${minio.accessKey}")
        private String accessKey;
        @Value("${minio.secretKey}")
        private String secretKey;
    
       @Bean
        public MinioClient getMinioClient() {
            OkHttpClient okHttpClient = new OkHttpClient.Builder()
                    .sslSocketFactory(OkHttpSSLSocketClient.getSSLSocketFactory(),OkHttpSSLSocketClient.getX509TrustManager()) // //通过sslSocketFactory方法设置https证书
                    .hostnameVerifier(OkHttpSSLSocketClient.getHostnameVerifier())
                    .build();
            return MinioClient.builder().endpoint(minioUrl).httpClient(okHttpClient)
                    .credentials(accessKey, secretKey).build();
        }
        
    }

/**
     * OkHttpSSLSocketClient
     * @author senfel
     * @version 1.0
     * @date 2023/9/15 10:07
     */
    public class OkHttpSSLSocketClient{
        //获取SSLSocketFactory
        public static SSLSocketFactory getSSLSocketFactory() {
            try {
                SSLContext sslContext = SSLContext.getInstance("SSL");
                sslContext.init(null, getTrustManager(), new SecureRandom());
                return sslContext.getSocketFactory();
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
    
        //获取TrustManager
        private static TrustManager[] getTrustManager() {
            TrustManager[] trustAllCerts = new TrustManager[]{
                    new X509TrustManager() {
                        @Override
                        public void checkClientTrusted(X509Certificate[] chain, String authType) {
                        }
    
                        @Override
                        public void checkServerTrusted(X509Certificate[] chain, String authType) {
                        }
    
                        @Override
                        public X509Certificate[] getAcceptedIssuers() {
                            return new X509Certificate[]{};
                        }
                    }
            };
            return trustAllCerts;
        }
    
        //获取HostnameVerifier，验证主机名
        public static HostnameVerifier getHostnameVerifier() {
            HostnameVerifier hostnameVerifier = (s, sslSession) -> true;
            return hostnameVerifier;
        }
        //X509TrustManager：证书信任器管理类
        public static X509TrustManager getX509TrustManager() {
            X509TrustManager x509TrustManager = new X509TrustManager() {
                //检查客户端的证书是否可信
                @Override
                public void checkClientTrusted(X509Certificate[] chain, String authType) {
    
                }
                //检查服务器端的证书是否可信
                @Override
                public void checkServerTrusted(X509Certificate[] chain, String authType) {
    
                }
    
                @Override
                public X509Certificate[] getAcceptedIssuers() {
                    return new X509Certificate[0];
                }
            };
            return x509TrustManager;
        }
    }

### 静态资源预览解决方案 ###

由于我们使用API上传，minio不能区分我们资源的类型，如果资源类型不对则不能正确提供预览功能，都直接变成为访问资源链接就下载了。

所以，在实际的项目集成中我们在上传资源前需要将资源类型写入请求中，方便minio解析并提供预览和下载功能。

/**
     * 上传一个文件
     */
    public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception {
        String prefix = objectName.substring(objectName.lastIndexOf(".") + 1);
        //静态资源预览解决方案
        String contentType = ViewContentType.getContentType(prefix);
        ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName)
                .contentType(contentType)
                .stream(stream, -1, 10485760).build());
        return objectWriteResponse;
    }

/**
     * ViewContentType
     * @author senfel
     * @version 1.0
     * @date 2023/9/14 17:27
     */
    public enum ViewContentType {
        DEFAULT("default","application/octet-stream"),
        JPG("jpg", "image/jpeg"),
        TIFF("tiff", "image/tiff"),
        GIF("gif", "image/gif"),
        JFIF("jfif", "image/jpeg"),
        PNG("png", "image/png"),
        TIF("tif", "image/tiff"),
        ICO("ico", "image/x-icon"),
        JPEG("jpeg", "image/jpeg"),
        WBMP("wbmp", "image/vnd.wap.wbmp"),
        FAX("fax", "image/fax"),
        NET("net", "image/pnetvue"),
        JPE("jpe", "image/jpeg"),
        RP("rp", "image/vnd.rn-realpix");
        private String prefix;
        private String type;
        public static String getContentType(String prefix){
            if(StringUtils.isEmpty(prefix)){
                return DEFAULT.getType();
            }
            prefix = prefix.substring(prefix.lastIndexOf(".") + 1);
            for (ViewContentType value : ViewContentType.values()) {
                if(prefix.equalsIgnoreCase(value.getPrefix())){
                    return value.getType();
                }
            }
            return DEFAULT.getType();
        }
    
        ViewContentType(String prefix, String type) {
            this.prefix = prefix;
            this.type = type;
        }
    
        public String getPrefix() {
            return prefix;
        }
    
        public String getType() {
            return type;
        }
    }

### 资源上传预览测试 ###

MinIO对于图片等资源可以直接预览，对于excel文档等直接提供下载功能

@SpringBootTest
    @RunWith(SpringJUnit4ClassRunner.class)
    public class CodeDbInfoServiceTests  {
    
        @Resource
        @Lazy
        private MinioUtil minioUtil;
    
    
        /**
         * upload
         * @author senfel
         * @date 2023/9/27 10:00
         * @return void
         */
        @Test
        public void upload() throws Exception {
            FileInputStream fileInputStream = new FileInputStream("C:/Users/dev/Desktop/minio.png");
            ObjectWriteResponse sacprivate = minioUtil.uploadFile(fileInputStream, "sacprivate", "test/minio.png");
            System.err.println(sacprivate.bucket());
        }
    
        /**
         * 获取私库连接
         * @author senfel
         * @date 2023/9/27 10:01
         * @return void
         */
        @Test
        public void getPrivateUrl() throws Exception{
            String url = minioUtil.getPresignedObjectUrl("sacprivate", "test/minio.png", 60);
            System.err.println(url);
        }
    }

### 测试结果 ###

图片成功上传  
![在这里插入图片描述][1d859e0660e74f378fb9e4d17740f4f5.png]

私库直接预览访问失败  
![在这里插入图片描述][667fc86d670546e89eb00b1e22d0516a.png]

获取私库链接过期时间为60s  
![在这里插入图片描述][5df28515843b401fa10f4def60f5e031.png]

访问成功  
![在这里插入图片描述][28d9f53272cb4e27bb4ab2afc1a503b9.png]

60s后再次访问该链接，提示链接已经过期  
![在这里插入图片描述][25563f1e0c044b30b670276899e2d553.png]

至此minio接入并测试完成，对于公开桶的资源直接可任意访问不用获取有效期链接。

写在最后  
Springboot微服务接入MinIO实现文件服务较为简单，我们只要按照官方文档引入依赖调用即可。值得注意的是我们根据需求选择接入ssl兼容方案和静态资源预览功能。当然，minio的private\\custom\\public可以完全实现我们各种需求，可以完全替代市场上付费和笨重的分布式服务。

[MinIO]: http://t.csdn.cn/4S8wO
[c160c66374904397b7c6b71e46ca81de.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/79dc420bd3d54ebebfe30e5189b8377a.png
[29bbf5c7703c43059a6eac87b1b31ac4.png]: https://img-blog.csdnimg.cn/29bbf5c7703c43059a6eac87b1b31ac4.png
[f2f30ecfb3e04c03ad16478764c14d30.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/79e56a55b29d444b87bd9127bebd397d.png
[5a79251cd04248ba9adb1c3bc073d587.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/bfd23d7a1a1247669d852b7c59efc193.png
[984013365c75428fb98133cd577dec26.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/501f895a41664b8cada5bdc5b72a5af1.png
[2a253ba6f52c47f88074b23d5085d9e2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/b543896d6e854f91af898cc827bc622d.png
[d7da0f6c58ac484a8586ac3a963cf3f4.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/523fcad4241442a7a43cf182b6a0af0e.png
[e10837a1e6244df7a7583d5e5812987e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/5391887e12eb4d00bfb6c859502dc17b.png
[d11c6bbd2c3e4fa99f90106689d46721.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/e4a0b35374ab4c10883163a3c470373c.png
[7b18d7ba9d354a12a1f5d165161a7b84.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/7d4cdf8a902f4128b0467904e1e5039c.png
[1d859e0660e74f378fb9e4d17740f4f5.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/55b50b8ef1794cb2a24ff77eee7099fd.png
[667fc86d670546e89eb00b1e22d0516a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/9b50508312fd4ffc903b15650342e384.png
[5df28515843b401fa10f4def60f5e031.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/55118cb74bd347e784ecb8c2461d883f.png
[28d9f53272cb4e27bb4ab2afc1a503b9.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/bfe3c506fcda4ce19feb15986d8fcc03.png
[25563f1e0c044b30b670276899e2d553.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/29/1a5b8dffc5ea40c7aef28326ea3db565.png