1. 首页
  2. 文章资讯

编译aws并访问minio

29 阅读 0 评论

Aws

amazon (S3) 是一个公开的服务，Web 应用程序开发人员可以使用它存储数字资产，包括图片、视频、音乐和文档。 S3 提供一个 RESTful API 以编程方式实现与该服务的交互.

MinIO是兼容AWS SDK,所以可以通过aws访问minio文件系统。

指导文档：https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/getting-started.html

Aws安装
Windows系统
vcpkg install aws-sdk-cpp 命令行在线安装

需要先安装vcpkg，参看https://blog.csdn.net/baidu_16370559/article/details/140697821?spm=1001.2014.3001.5501

在powershell输入下面命令

vcpkg search aws-sdk-cpp

vcpkg install aws-sdk-cpp[s3]:x64-windows

报错，不能通过

源码安装：

参看https://docs.aws.amazon.com/zh_cn/sdk-for-cpp/v1/developer-guide/setup-windows.html

前提要求：

Install CMake (minimum version 3.13) ，可以在powershell 输入cmake --version 查看版本，这里cmake 的版本是

  1. 源码下载：GitHub - aws/aws-sdk-cpp: AWS SDK for C++ 有2种办法
  1. 直接从git下载zip文件

这里下载的版本是aws-sdk-cpp-1.11.370.zip.  ctr/aws-crt-cpp目录为空，需要手动从git下载.

使用的aws-crt-cpp版本是v0.27.4,此时aws-crt-cpp目录的ctr目录下的各个文件夹也是空的。需要手动下载ctr目录下各文件夹的源码

  1. 在powershell通过git下装

前提条件是：安装git

在powershell 输入 git clone --recurse-submodules GitHub - aws/aws-sdk-cpp: AWS SDK for C++ 这样通过git下载。这种办法只能下载main分支的代码。

这种办法不一定会把所有的文件都下载，也可能需要手动下载依赖库。

  1. mkdir sdk_build  cd sdk_build
  2. Generate build files

cmake ..\aws-sdk-cpp-1.11.370 -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="s3" -DCMAKE_PREFIX_PATH="E:\code\aws\aws\aws-sdk-cpp-1.11.370\install" -DENABLE_TESTING=OFF

  1. cmake --build  --config=Debug
  2. cmake --install  --config=Debug

注意：步骤3过程中，使用powershell编译出错.改成cmake-gui进行编译。

ALL_BUILD右键，生成

单击INSTALL,右键，仅用于项目，仅生成INSTALL

Windows系统使用aws

VS2019

直接使用aws,编译测试demo会报错。

解决方法

官方文档有说到，要添加USE_IMPORT_EXPORT和

USE_WINDOWS_DLL_SEMANTICS两个宏，所以这个问题是符号导出的问题

QT

解决办法：

在pro文件加上预定义

DEFINES +=USE_IMPORT_EXPORT

DEFINES +=USE_WINDOWS_DLL_SEMANTICS

Linux系统
vcpkg install aws-sdk-cpp 命令行在线安装

  * aws-c-auth:x64-linux@0.7.22

  * aws-c-cal:x64-linux@0.6.15

  * aws-c-common:x64-linux@0.9.21

  * aws-c-compression:x64-linux@0.2.18

  * aws-c-event-stream:x64-linux@0.4.2

  * aws-c-http:x64-linux@0.8.2

  * aws-c-io:x64-linux@0.14.9

  * aws-c-mqtt:x64-linux@0.10.4

  * aws-c-s3:x64-linux@0.5.10

  * aws-c-sdkutils:x64-linux@0.1.16

  * aws-checksums:x64-linux@0.1.18

  * aws-crt-cpp:x64-linux@0.26.12

    aws-sdk-cpp[core,dynamodb,kinesis,s3]:x64-linux@1.11.352

  * s2n:x64-linux@1.4.16

没通过，应该因为需要依赖太多第三方库，直接在线安装很多时候下载不成功

源码安装

下载地址：GitHub - aws/aws-sdk-cpp: AWS SDK for C++

参考：Build the AWS SDK for C++ on Linux/macOS - AWS SDK for C++

前提：

GNU Compiler Collection (GCC) 4.9 or later, or

Install CMake (minimum version 3.13

额外的要求：

sudo apt-get install libcurl4-openssl-dev libssl-dev uuid-dev zlib1g-dev libpulse-dev

安装步骤：

  1. 下载源码

 源码地址：GitHub - aws/aws-sdk-cpp: AWS SDK for C++  

 有2种办法

  1. 直接从git下载zip文件

这里下载的版本是aws-sdk-cpp-1.11.370.zip. 如果在windows系统已经下载好源码，可以直接拿来用。aws-crt-cpp版本是v0.27.4。此时aws-crt-cpp目录的ctr目录下的各个文件夹也是空的。需要手动下载ctr目录下各文件夹的源码

  1. 在shell 通过git下装

前提条件是：安装git

git clone --recurse-submodules GitHub - aws/aws-sdk-cpp: AWS SDK for C++

注意事项：ctr/aws-crt-cpp目录下的很多文件可能拉不下来，需要手动从git下载.

  1. mkdir sdk_build   cd sdk_build
  2. Generate build files

这里我使用的是cmake ../aws-sdk-cpp -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/root/software/aws/aws/install/ -DBUILD_ONLY="s3" -DENABLE_TESTING=OFF

  1. make -j4
  2. make install

注意事项：

我这里A.直接从git下载zip文件，编译通过后。调用aws库，会出现libaws-c-s3.so库缺少

libcrypto.so.3。因为我的电脑openssl的版本只有libcrypto.so.1.版本太低了。需要升版libcrypto。

升版方法参考：编译openssl文章。具体操作是从Downloads | Library下载源码编译安装，然后设置软连接，同时把openssl的lib库加载环境变量。

客户端使用aws

要求

  1. 最开始必须调用Aws::InitAPI
  2. 在程序关闭前必须调用Aws::ShutdownAPI
  3. 其他的aws的api在Aws::InitAPI和Aws::ShutdownAPI之间调用

QT通过aws访问minio

pro配置

INCLUDEPATH += $$PWD/install/include

DEPENDPATH += $$PWD/install/include

DESTDIR=$$PWD/bin

 LIBS += -L$$PWD/install/lib/   -laws-c-auth -laws-c-cal\

-laws-c-common -laws-c-compression -laws-c-event-stream -laws-checksums -laws-c-http -laws-c-io\

-laws-c-mqtt -laws-cpp-sdk-core -laws-crt-cpp -laws-c-s3 -laws-c-sdkutils -ls2n -laws-cpp-sdk-s3

代码编写

参看：https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/examples-s3-objects.html

#ifndef MINIOSERVICE_H

#define MINIOSERVICE_H

#include <QMutex>

#include <QString>

#include <QStringList>

#include "aws/s3/S3Client.h"

#include "aws/core/Aws.h"

#include "aws/core/auth/AWSCredentialsProvider.h"

#include "aws/s3/model/PutObjectRequest.h"

#include "aws/s3/model/GetObjectRequest.h"

#include "aws/s3/model/CreateBucketRequest.h"

#include "aws/s3/model/DeleteObjectRequest.h"

#include "aws/s3/model/DeleteBucketRequest.h"

#include "aws/s3/model/ListObjectsRequest.h"

using namespace std;

using namespace Aws::S3;

using namespace Aws::S3::Model;

struct RetStatus

{

    int nRetCode = 0;

    QString msg ="成功";

};

class MinioService

{

public:

    MinioService();

    ~MinioService();

public:

    //单例

    static MinioService * GetInstance();

    //释放

    static void DeleteInstance();

    /**

     * @brief LoginMinio 登录minio文件系统

     * @return 错误码

     */

    RetStatus LoginMinio();

    RetStatus CreateBucket(QString Bucketname);

    RetStatus GetBuckets(QStringList& bucketsname);

    RetStatus GetBucketFile(QString Bucketname,QStringList& fileList);

    /**

     * @brief DownLoadFile 从minio文件系统下载文件

     * @param strBucket   桶的名称

     * @param strObject   需下装的内容

     * @param strSavePath  下载到本地的路径

     * @return   错误码

     */

    RetStatus DownLoadFile(QString strBucket,QString strObject,QString strSavePath);

    /**

     * @brief UpLoadFile 把本地文件上传到minio文件系统

     * @param strBucket  桶的名称

     * @param strObject  存放到minio的文件路径

     * @param strFilePath 本地文件路径

     * @return

     */

    RetStatus UpLoadFile(QString strBucket,QString strObject,QString strFilePath);

    RetStatus DeleteFile(QString strBucket,QString strObject);

    // // 删除Bucket，一定要先删除文件，再删除Bucket

    RetStatus DeleteBucket(QString strBucket);

private:

    static MinioService *m_pInstance;

    static QMutex m_mutex;

    S3Client* m_client;

    Aws::SDKOptions m_options;

};

#endif // MINIOSERVICE_H

#include "minioservice.h"

#include <QFile>

MinioService *MinioService::m_pInstance = nullptr;

QMutex MinioService::m_mutex;

MinioService::MinioService()

{

    m_client = nullptr;

    m_options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Debug;

    Aws::InitAPI(m_options);

    Aws::Client::ClientConfiguration cfg;

    cfg.endpointOverride = "127.0.0.1:3003";

    cfg.scheme = Aws::Http::Scheme::HTTP;

    cfg.verifySSL = false;

    Aws::Auth::AWSCredentials cred("minioadmin", "minioadmin");

    m_client = new S3Client(cred, cfg, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Always, false,

    Aws::S3::US_EAST_1_REGIONAL_ENDPOINT_OPTION::NOT_SET);

}

MinioService::~MinioService()

{

    Aws::ShutdownAPI(m_options);

    if(m_client)

    {

        delete m_client;

        m_client = nullptr;

    }

}

MinioService *MinioService::GetInstance()

{

    if(nullptr == m_pInstance)

    {

        m_mutex.lock();

        if( nullptr == m_pInstance)

        {

            m_pInstance =  new MinioService();

        }

        m_mutex.unlock();

    }

    return m_pInstance;

}

void MinioService::DeleteInstance()

{

    m_mutex.lock();

    if(m_pInstance)

    {

        delete m_pInstance;

        m_pInstance = nullptr;

    }

    m_mutex.unlock();

}

RetStatus MinioService::LoginMinio()

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    auto response = m_client->ListBuckets();

    if (response.IsSuccess())

    {

        Reture.nRetCode = 0;

        Reture.msg = QString("登录minio文件系统成功.");

        return Reture;

    }

    else

    {

        Reture.nRetCode = 1;

        Reture.msg = "登录失败";

    }

    return Reture;

}

RetStatus MinioService::CreateBucket(QString Bucketname)

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    Aws::S3::Model::CreateBucketRequest create_bucket_request;

    create_bucket_request.SetBucket(Bucketname.toStdString());

    create_bucket_request.SetCreateBucketConfiguration({});

    auto create_bucket_outcome = m_client->CreateBucket(create_bucket_request);

    if (create_bucket_outcome.IsSuccess())

    {

        Reture.nRetCode = 0;

        Reture.msg = "创建Bucket成功。";

        return  Reture;

    }

    else

    {

        string strErr = create_bucket_outcome.GetError().GetMessage();

        Reture.nRetCode = 2;

        Reture.msg = "创建Bucket失败：" + QString::fromStdString(strErr) ;

    }

    return Reture;

}

RetStatus MinioService::GetBuckets(QStringList& bucketsname)

{

    bucketsname.clear();

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    auto response = m_client->ListBuckets();

    if (response.IsSuccess())

    {

        auto buckets = response.GetResult().GetBuckets();

        for(auto bucket :buckets)

        {

            bucketsname.append(QString::fromStdString(bucket.GetName()));

        }

        Reture.nRetCode = 0;

        Reture.msg = "获取buckets成功。";

    }

    else

    {

        Reture.nRetCode = 3;

        string strErr = response.GetError().GetMessage();

        Reture.msg = "获取buckets不成功:" + QString::fromStdString(strErr) ;

    }

    return Reture;

}

RetStatus MinioService::GetBucketFile(QString Bucketname, QStringList &fileList)

{

    fileList.clear();

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    Aws::S3::Model::Bucket temps{};

    std::string stringK = Bucketname.toStdString();

    temps.SetName(stringK);

    /// 查找MINIO相关数据

    Aws::S3::Model::ListObjectsRequest objects_request;

    objects_request.WithBucket(temps.GetName());   /// 设置桶路径

    auto list_objects_outcome = m_client->ListObjects(objects_request);  /// 判断存储桶及相关连接是否有效

    if (list_objects_outcome.IsSuccess())

    {

        Aws::Vector<Aws::S3::Model::Object> object_list = list_objects_outcome.GetResult().GetContents();

        for (auto  s3_object : object_list)

        {

            QString filename =QString::fromStdString(s3_object.GetKey());

            fileList.append(filename);

        }

        Reture.nRetCode = 0;

        Reture.msg = "获取文件列表成功。";

    }

    else

    {

        Reture.nRetCode = 8;

        string strErr = list_objects_outcome.GetError().GetMessage();

        Reture.msg = "获取文件列表失败:" + QString::fromStdString(strErr) ;

    }

    return Reture;

}

RetStatus MinioService::DownLoadFile(QString strBucket, QString strObject, QString strSavePath)

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    Aws::S3::Model::GetObjectRequest object_request;

    object_request.WithBucket(strBucket.toStdString().c_str()).WithKey(strObject.toStdString().c_str());

    auto get_object_outcome = m_client->GetObject(object_request);

    if (get_object_outcome.IsSuccess())

    {

        Aws::IOStream &local_file = get_object_outcome.GetResult().GetBody();

        std::istreambuf_iterator<char> begin(local_file);

        std::istreambuf_iterator<char> end;

        std::string buffer(begin, end); // 读取整个文件内容

        QFile file_(strSavePath);

        if(file_.open(QIODevice::ReadWrite | QIODevice::Truncate))//以截取方式打开文件,文件原有的内容全部被删除

        {

            file_.write(QByteArray::fromStdString(buffer));

            file_.close();

            Reture.nRetCode = 0;

            Reture.msg = "下载文件成功。";

        }

        else

        {

            Reture.nRetCode = 4;

            Reture.msg = "下载文件失败:创建本地保存文件失败";

        }

    }

    else

    {

        string strErr = get_object_outcome.GetError().GetMessage() ;

        Reture.nRetCode = 5;

        Reture.msg = "下载文件失败:" + QString::fromStdString(strErr);

    }

    return  Reture;

}

RetStatus MinioService::UpLoadFile(QString strBucket, QString strObject, QString strFilePath)

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    QFile file(strFilePath);

    bool bOpen = file.open(QIODevice::ReadOnly);

    Aws::S3::Model::PutObjectRequest putObjectRequest;

    putObjectRequest.WithBucket(strBucket.toStdString().c_str()).WithKey(strObject.toStdString().c_str());

    std::shared_ptr<Aws::IOStream> input_data = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream");

    *input_data << file.readAll().toStdString();

    file.close();

    putObjectRequest.SetBody(input_data);

    auto putObjectResult = m_client->PutObject(putObjectRequest);

    if (putObjectResult.IsSuccess())

    {

        Reture.nRetCode = 0;

        Reture.msg = "上传文件成功";

    }

    else

    {

        string strErr = putObjectResult.GetError().GetMessage() ;

        Reture.nRetCode = 6;

        Reture.msg = "上传文件失败:" + QString::fromStdString(strErr);

    }

    return  Reture;

}

RetStatus MinioService::DeleteFile(QString strBucket, QString strObject)

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    Aws::S3::Model::DeleteObjectRequest delete_object_request;

    delete_object_request.WithBucket(strBucket.toStdString().c_str()).WithKey(strObject.toStdString().c_str());

    auto delete_object_outcome = m_client->DeleteObject(delete_object_request);

    if (delete_object_outcome.IsSuccess())

    {

        Reture.nRetCode = 0;

        Reture.msg = "删除文件成功";

    }

    else

    {

        string strErr = delete_object_outcome.GetError().GetMessage() ;

        Reture.nRetCode = 7;

        Reture.msg = "删除文件失败:" + QString::fromStdString(strErr);

    }

    return Reture;

}

RetStatus MinioService::DeleteBucket(QString strBucket)

{

    RetStatus Reture;

    if(m_client == nullptr)

    {

        Reture.nRetCode = -1;

        Reture.msg = "创建连接客户端不成功。";

        return  Reture;

    }

    Aws::S3::Model::DeleteBucketRequest delete_bucket_request;

    delete_bucket_request.SetBucket(strBucket.toStdString().c_str());

    auto delete_bucket_outcome = m_client->DeleteBucket(delete_bucket_request);

    if (delete_bucket_outcome.IsSuccess())

    {

        Reture.nRetCode = 0;

        Reture.msg = "删除bucket成功";

    }

    else

    {

        string strErr = delete_bucket_outcome.GetError().GetMessage() ;

        Reture.nRetCode = 7;

        Reture.msg = "删除文件失败:" + QString::fromStdString(strErr);

    }

    return Reture;

}

本站资源均来自互联网，仅供研究学习，禁止违法使用和商用，产生法律纠纷本站概不负责！如果侵犯了您的权益请与我们联系！

转载请注明出处： 免费源码网-免费的源码资源网站 » 编译aws并访问minio

点赞(0) 打赏

评论列表 共有 0 条评论

暂无评论

微信公众账号

微信扫一扫加关注

 发表
评论 返回
顶部