Tried using Dropbox API

Hello! I'm PanKUN.

This time, it's about file management for my custom GameEngine.

Are you all using Git LFS? It's convenient, isn't it? But there are probably many free users among individuals. By the way, my custom GameEngine is created based on C++ and supports DirectX12, DirectX11, converts various model files to unique model data, etc...

Well, in short, I want to load various functions, so the custom GameEngine (BreadEditor) has a directory called ExternalLibrary.

I introduce static libraries, but when I thought about how to manage these files, the first thing that came up was the use of cloud services.

Here, I would like to talk about the method and the points I cared about.

Overview

The rough flow is as follows.

• Create a Dropbox account
• Create an app on Dropbox
• Create an access token for the relevant app
• Create transmission/reception processing with Dropbox in javascript based on node.js
• Create batch files, shell scripts

I will also list the rough directory structure of BreadEditor.

BreadEditor
├── .git
├── Bat
│   ├── js
│   ├── json
│   └── *.bat
├── BreadEditor
├── Data
├── Library
│   ├── BreadLibrary
│   └── ExternalLibrary
└── project.sln

Preparation

Create a Dropbox account

You can do it from here. There is nothing special to mention, but if you use it regularly, it might be better to create an application-specific account to prevent accidents.

Create an app on Dropbox

Create an app from this page. I'll paste the URL of the create button here. If you want to go in one shot, use this.

Explanation of the input items below.

1. Choose an API: If you use it personally, "Dropbox API" should be fine.
2. Choose the type of access you need: Whether the app you create is for a specific folder only or the whole thing. The specific folder only is recommended.
3. Name your app: As you like.
4. Press Create app.

[! NOTE] If you have already created an app, you can touch it from "App console" in the 3-dot barcode on the Dropbox Developer page.

Create Access Token

Set the necessary permissions and press the Generated access token button. Note down the output token somewhere. (Since it is temporary, it is necessary to output it every time work is done. Sensationally, the expiration date is about half a day)

Environment Construction

Create various things under the directory where you work. (This time the Bat directory)

• bat
  • data
    • Dropbox.json
  • js
    • DropboxDownload.js
    • DropboxUpload.js
  • Download.bat
  • Update.bat

Open a terminal under the js directory and execute the following. (Assuming node.js npm is installed)

npm install --save dropbox npm install --save isomorphic-fetch

Create Upload Script

I'll attach the code for now. First from DropboxUpload.js

const fs = require("fs");
const path = require("path");
const Dropbox = require("dropbox");
const accessToken = "{取得したアクセストークン}";
const dbx = new Dropbox.Dropbox({ accessToken });

function DropboxUpload(dirPath, remotePath) {
  const files = fs.readdirSync(dirPath, {
    withFileTypes: true,
  });

  files.forEach((element) => {
    if (element.isDirectory()) {
      dbx.filesCreateFolderV2({
        path: remotePath + element.name + "/",
        autorename: false,
      });
      DropboxUpload(
        dirPath + "/" + element.name,
        remotePath + element.name + "/"
      );
    } else if (element.isFile()) {
      if (path.extname(element.name) == ".{アップしたい拡張子}") {
        const uploadFile = fs.readFileSync(dirPath + "/" + element.name, {
          encoding: "binary",
        });

        const response = dbx.filesUpload({
          path: remotePath + element.name,
          contents: uploadFile,
          mode: "overwrite",
        });
      }
    }
  });
}
DropboxUpload("{作業ディレクトリパス}", "/");

Below is a simple explanation of the code. DropboxUpload is processed recursively. The first argument "dirPath : string" is the directory path you want to synchronize with the local cloud storage. The second argument "remotePath : string" is the remote (cloud storage path.)

The reference path of Dropbox starts with "/". This points directly under the directory managed by the created app.

readdirSync is done with recursive: false this time. The reason is that Dropbox needs to create directories from time to time. It is also the reason why I am processing recursively myself.

If the path obtained by readdirSync is a directory, lower the hierarchy and do the same processing. If it is a file, perform upload.

Create Download Script

Next, DropboxDownload.js

const fs = require("fs");
const Dropbox = require("dropbox");
const fetch = require("isomorphic-fetch");
const accessToken = "{取得したアクセストークン}";
const dbx = new Dropbox.Dropbox({ accessToken: accessToken, fetch: fetch });
const targetDir = "{ダウンロード先ローカルパス}";

function DropboxDownload(originPath) {
  dbx.filesListFolder({ path: originPath }).then(function (response) {
    response.result.entries.forEach((element) => {
      if (element[".tag"] == "folder") {
        if (!fs.existsSync(targetdir + element.path_display)) {
          fs.mkdirSync(targetdir + element.path_display);
        }

        DropboxDownload(element.path_display);
      } else if (element[".tag"] == "file") {
        dbx
          .filesDownload({ path: element.path_display })
          .then(function (response) {
            fs.writeFile(
              targetdir + response.result.path_display,
              response.result.fileBinary,
              "binary",
              function (err) {
                console.error({ err });
              }
            );
          });
      }
    });
  });
}
DropboxDownload("");

Below is a simple explanation of the code. Since it just drops it, there are fewer things to consider than uploading.

The basic idea here is also the same as up. Sweep the elements returned from fileListFolder and recurse if it is a directory. If it is a file, download it is the base.

Create Batch File

Enter the command to execute each javascript in Download.bat, Upload.bat created directly under the Bat directory.

At this time, if you want to change the directory, change the extension, or support changing the token, put it in the argument and support it on the corresponding javascript side.

To get the argument of node.js is process.argv[]. Since it is stored from the first command, the first element should be node and the second element should be execution javascript file name. So you can get the argument from process.argv[2].

Stuck Points

I will talk about the conclusion. Official is the best. From the middle, I gave up on shortening by looking at third-party articles and tried to look at the official one.

The reason is unknown whether the NDK has been updated or what, but there is no single article attaching code that can be used by copy-paste.

For example, response of dbx.filesListFolder().then(function(response)) in DropboxDownload.js, which will be used in the later filesDownload, was described as response.fileBinary. (Such a thing does not exist.)

There are various other things, but what I want to say is that this article is no exception, keep third parties as a reference and read the official properly.

Finally

This time it was about synchronizing some files in the project using cloud storage services. Now you can manage files for free!!

As for the distinction from gitLFS, I think you should judge based on the following two points.

• Do you want to manage versions?
• Is it necessary to keep it? (Even if it is not, it is generated at compile time, etc.)

As an author, I don't want to manage versions but use it to use multiple work environments. Well, if you are pouring money into GitLFS, it is unnecessary.

If you have a dedicated server at home, you don't need this kind of thing, so I feel like it's a method taken by people who are financially tight.

Reference Materials

https://qiita.com/Ella_Engelhardt/items/c33f08b6b427eab8b310