【分布式文件存储】MinIO部署及实现文件上传下载_百度小程序

您的位置：上海毫米网络优化公司 > 小程序 > 百度小程序 >

概述

MinIO | 高性能, Kubernetes 原生对象存储

【分布式文件存储】MinIO部署及实现文件上传下载,第1张

MinIO是全球领先的对象存储先锋，目前在全世界有数百万的用户。

高性能，在标准硬件上，读/写速度上高达183GB/秒和171GB/秒，拥有更高的吞吐量和更低的延迟
可扩展性，为对象存储带来了简单的缩放模型，通过添加更多集群可以扩展空间
简单，极简主义是MinIO的指导性设计原则，即可在几分钟内安装和配置
与Amazon S3兼容，亚马逊云的 S3 API（接口协议）是在全球范围内达到共识的对象存储的协议，是全世界内大家都认可的标准

数据安全，使用纠删码来保护数据免受硬件故障和无声数据损坏

MinIO集群部署

基于docker-compose部署入门案例

准备docker-compose.yml

【分布式文件存储】MinIO部署及实现文件上传下载,第2张

version: '3.7'
# 所有容器通用的设置和配置
x-minio-common: &minio-common
  image: minio/minio
  command: server --console-address ":9001" http://minio{1...4}/data
  expose:
    - "9000"
  # environment:
    # MINIO_ROOT_USER: minioadmin
    # MINIO_ROOT_PASSWORD: minioadmin
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
    interval: 30s
    timeout: 20s
    retries: 3
# 启动4个docker容器运行minio服务器实例
# 使用nginx反向代理9000端口，负载均衡, 你可以通过9001、9002、9003、9004端口访问它们的web console
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    ports:
      - "9001:9001"
    volumes:
      - ./data/data1:/data
  minio2:
    <<: *minio-common
    hostname: minio2
    ports:
      - "9002:9001"
    volumes:
      - ./data/data2:/data
  minio3:
    <<: *minio-common
    hostname: minio3
    ports:
      - "9003:9001"
    volumes:
      - ./data/data3:/data
  minio4:
    <<: *minio-common
    hostname: minio4
    ports:
      - "9004:9001"
    volumes:
      - ./data/data4:/data
  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./config/nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

在当前目录下，创建config，创建nginx.conf

【分布式文件存储】MinIO部署及实现文件上传下载,第3张

user  nginx;
worker_processes  auto;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  4096;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;
    # include /etc/nginx/conf.d/*.conf;
    upstream minio {
        server minio1:9000;
        server minio2:9000;
        server minio3:9000;
        server minio4:9000;
    }
    server {
        listen       9000;
        listen  [::]:9000;
        server_name  localhost;
        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;
            proxy_pass http://minio;
        }
    }
}

测试启动

docker-compose up -d

【分布式文件存储】MinIO部署及实现文件上传下载,第4张

访问任意节点

默认账户密码：minioadmin minioadmin

【分布式文件存储】MinIO部署及实现文件上传下载,第5张

【分布式文件存储】MinIO部署及实现文件上传下载,第6张

MinIO用户管理

需要创建一个用户，客户端用户可上传下载权限。

注意：长度最好是32位，否则后续调用API会报错

【分布式文件存储】MinIO部署及实现文件上传下载,第7张

Buckets管理

MinIO uses buckets to organize objects. A bucket is similar to a folder or directory in a filesystem, where each bucket can hold an arbitrary number of objects.

MinIO使用bucket来组织对象。bucket类似于文件系统中的文件夹或目录，其中每个bucket可以容纳任意数量的对象。

【分布式文件存储】MinIO部署及实现文件上传下载,第8张

创建Buckets

注意桶名的命名规范，否则后续调用api创建桶会报错

【分布式文件存储】MinIO部署及实现文件上传下载,第9张

可以在控制台管理文件，上传下载预览删除等

【分布式文件存储】MinIO部署及实现文件上传下载,第10张

Simple Storage Service（Amazon S3）存储桶命名要求 - AWS CloudTrail

【分布式文件存储】MinIO部署及实现文件上传下载,第11张

MinIO客户端

官方接口API文档

Java Client API Reference — MinIO Object Storage for Linux

MinIO支持多种客户端，本文以java为例

【分布式文件存储】MinIO部署及实现文件上传下载,第12张

引入依赖


    io.minio
    minio
    8.5.3

文件上传下载Demo

package pers.kw.minio.demo;
import com.alibaba.fastjson.JSON;
import io.minio.*;
import io.minio.errors.*;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;
public class MinIODemo {
    private static final String accessKey = "PPeCeQbHtFsEcGNxnlmSpbYasxELHoGy";
    private static final String secretKey = "GEMQDMUhQSHaIJCqmxperpESNYCJXvjJ";
    private static final String endpoint = "http://192.168.8.88:9000";
    public static void main(String[] args)
            throws ServerException, InsufficientDataException, ErrorResponseException,
            IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
        String bucket = "test01";
        String objectName = "small125750JkEMk1625374670.jpg";
        downloadFile(bucket, objectName);
    }
    public static void downloadFile(String bucket, String objectName)
            throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
        MinioClient minioClient =
                MinioClient.builder()
                        .endpoint(endpoint)
                        .credentials(accessKey, secretKey)
                        .build();
        //todo 需要关闭流
        try (InputStream inputStream = minioClient.getObject(
                GetObjectArgs.builder()
                        .bucket(bucket)
                        .object(objectName)
                        .build())) {
            // Read data from stream
            byte[] bytes = IOUtils.toByteArray(inputStream);
            String fileName = "22.jpg";
            System.out.println("下载文件：" + bytes.length);
            FileUtils.writeByteArrayToFile(new File("C:\Users\pc\Desktop\" + fileName), bytes);
            System.out.println("文件下载成功");
            IOUtils.closeQuietly(inputStream);
        }
    }
    public static void uploadFile(String bucket) throws IOException, NoSuchAlgorithmException, InvalidKeyException {
        try {
            // Create a minioClient with the MinIO server playground, its access key and secret key.
            MinioClient minioClient =
                    MinioClient.builder()
                            .endpoint(endpoint)
                            .credentials(accessKey, secretKey)
                            .build();
            // Make 'asiatrip' bucket if not exist.
            boolean found =
                    minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
            if (!found) {
                // Make a new bucket called bucket.
                minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build());
            } else {
                System.out.println("Bucket " + bucket + " already exists.");
            }
            // Upload '/home/user/Photos/asiaphotos.zip' as object name 'asiaphotos-2015.zip' to bucket
            // 'asiatrip'.
            String fileName = "C:\Users\pc\Desktop\gateway-server.rar";
            String objectName = "gateway-server_2023058.rar";
            minioClient.uploadObject(
                    UploadObjectArgs.builder()
                            .bucket(bucket)
                            .object(objectName)
                            .filename(fileName)
                            .build());
            System.out.println(
                    fileName + " is successfully uploaded as"
                            + objectName + " to bucket " + bucket + ".");
        } catch (MinioException e) {
            System.out.println("Error occurred: " + e);
            System.out.println("HTTP trace: " + e.httpTrace());
        }
    }
}

调用API碰到的问题

问题一：

java.lang.NoSuchMethodError: okhttp3.RequestBody.create ( BLokhttp3/Medialype:)Lokhttp3/RequestBody;

【分布式文件存储】MinIO部署及实现文件上传下载,第13张

解决: 引入如下依赖


    com.squareup.okhttp3
    okhttp
    4.9.0

问题二：

bucket name does not follow Amazon S3 standards.

命名不符合规则换个名字，最好在代码中增加正则校验

【分布式文件存储】MinIO部署及实现文件上传下载,第14张

问题三：

RequestTimeTooSkewed

客户端时间与服务器时间相差过大导致

【分布式文件存储】MinIO部署及实现文件上传下载,第15张

设置服务器时间

 ntpdate time.windows.com

问题4：

InvalidAccessKeyId

秘钥信息太短

【分布式文件存储】MinIO部署及实现文件上传下载,第16张

青岛中小企业网站制作佛山网站建设多少钱网站建设全过程企业网站制作流程网站制作制作太原网站制作设计

上一篇：14、Nginx---缓存服务

下一篇：【注册表】windows系统注册表常用修改方案