Linux Unlimited Storage Folder using G Suite (Beta)

Building a simple folder on your Linux server with unlimited cloud storage behind it. I will explain the basic concepts used and some guidelines on how to make it all work.

Basic concept

You will have /mnt/gdrivemount

And /mnt/gdrivewrite

And then UnionFS to combine the two into /mnt/gdriveunionfs/

Now, because Google Cloud Drive API limits the amount of requests per seconds, it is suggested that you add a cache into path to help limit these requests, so the /mnt/gdrivemount changes to

To be able to control what and when files are actually moved from your disk storage into the cloud storage, you will control the move to cloud process.

Services and Technologies Used

G Suite Business - gsuite.google.com ( refergsuite.app.goo.gl/TDdP )
Google Domains - domains.google.com
Google Drive - drive.google.com
Google Cloud Platform - console.cloud.google.com
Rclone - rclone.org
UnionFS - unionfs.filesystems.org

G Suite Business

The Business subscription for US$10 or €8 /user/month says it will only provide 1Tb/user if less than 5 users in the account, but many users have reported this to be incorrect. many users are able to load >100Tb of data into this subscription.

(If you are a new subscription, please use this referral link refergsuite.app.goo.gl/TDdP and I will get a couple of coins for the referral at no additional cost to you.)

You will have to subscribe and activate a subscription, they should throw in a 14 days eval.

Google Cloud Platform

You will also have to activate the Google Cloud Drive API and get an Oath 2.0 Client ID to use with Rclone.

PS: This process will offer you some free credits with Google Cloud Platform. I suggest you do not activate this at this time. Do not waste the trial period if you are not going to use it at this time.

Google Drive vs Google Team Drive

Sharing a Google Drive folder with someone else, their contribution into this folder goes against their own storage limits if they fall outside the G Suite account. Sharing a Team Drive with someone else, anything contributed into the Team Drive stays in the G Suite account.


Rclone

Rclone is the Rsync for the Cloud, but also has fuse mount and cache support. They also have a very strong encryption option, but the discussion about when of if to encrypt is for another post.

The Rclone setup process should now contain 2 remotes.

1. Google Drive - This is the Google Drive or Google Team Drive information. Make sure you use the "API Client ID" and "API Client Secret" that you created in the Google Cloud Platform Console.
2. Rclone Cache - Reference the remote name used for your Google Drive in 1. Just ignore and leave blank the Plex questions asked during the cache setup.

Unionfs

Unionsfs is available on most modern Linux distributions as an installable package. It takes values to then set the input folders and then the combined folder. During this you are able to set which is read-only and which is read-write.

Command samples

mkdir /mnt/gdrivemount
mkdir /mnt/gdrivewrite
mkdir /mnt/gdriveunionfs

rclone config create gdriveremote drive \
   --drive-client-id=X \
   --drive-client-secret=X

rclone config create gdrivecache cache \
   remote gdriveremote: \
   chunk_total_size 32G

rclone mount \
   -v --debug-fuse \
   --allow-other \
   --attr-timeout=1s \
   --buffer-size=500M \
   --bwlimit=10M \
   --checkers=16 \
   --dir-cache-time=60m \
   --drive-use-trash \
   --cache-chunk-size=10M \
   --cache-info-age=60m \
   --cache-tmp-wait-time 60m \
   --cache-workers=8 \
   --cache-writes \
   --no-modtime \
   --stats=0 \
   --umask 002 \
   gdrivecache:/ /mnt/gdrivemount

rclone move -v /mnt/gdrivewrite gdriveremote:

unionfs \
   -o cow,max_files=32768 \
   -o allow_other,use_ino,suid,dev,nonempty \
   /mnt/gdrivewrite=RW:/mnt/gdriveremote=RO \
   /mnt/gdriveunionfs

fusermount -u /mnt/gdrivemount