v0.91 Show
Paths are specified as Drive paths may be as deep as required, e.g. ConfigurationThe initial setup for drive involves getting a token from Google drive which you need to do in your browser. Here is an example of how to make a remote called
This will guide you through an interactive setup process:
See the remote setup docs for how to set it up on a machine with no Internet browser available. Note that rclone runs a webserver on your local machine to collect the token as returned from Google if using web browser to automatically authenticate. This only runs from the moment it opens your browser to the moment you get back the verification code. This is on You can then use it like this, List directories in top level of your drive
List all the files in your drive
To copy a local directory to a drive directory called backup
ScopesRclone allows you to select which scope you would like for rclone to use. This changes what type of token is granted to rclone. The scopes are defined here. The scope are driveThis is the default scope and allows full access to all files, except for the Application Data Folder (see below). Choose this one if you aren't sure. drive.readonlyThis allows read only access to all files. Files may be listed and downloaded but not uploaded, renamed or deleted. drive.fileWith this scope rclone can read/view/modify only those files and folders it creates. So if you uploaded files to drive via the web interface (or any other means) they will not be visible to rclone. This can be useful if you are using rclone to backup data and you want to be sure confidential data on your drive is not visible to rclone. Files created with this scope are visible in the web interface. drive.appfolderThis gives rclone its own private area to store files. Rclone will not be able to see any other files on your drive and you won't be able to see rclone's files from the web interface either. drive.metadata.readonlyThis allows read only access to file names only. It does not allow rclone to download or upload data, or rename or delete files or directories. Root folder IDThis option has been moved to the advanced
section. You can set the Normally you will leave this blank and rclone will determine the correct root to use itself. However you can set this to restrict rclone to a specific folder hierarchy or to access data within the "Computers" tab on the drive web interface (where files from Google's Backup and Sync desktop program go). In order to do this you will
have to find the So if the folder you want rclone to use has a URL which looks like NB folders under the "Computers" tab seem to be read only (drive gives a 500 error) when using rclone. There doesn't appear to be an API to discover the folder IDs of the "Computers" tab - please contact us if you know otherwise! Note also that rclone can't access any data under the "Backups" tab on the google drive web interface yet. Service Account supportYou can set up rclone with Google Drive in an unattended mode, i.e. not tied to a specific end-user Google account. This is useful when you want to synchronise files onto machines that don't have actively logged-in users, for example build machines. To use a Service Account instead of OAuth2 token flow, enter the path to your Service Account credentials at the Use case - Google Apps/G-suite account and individual DriveLet's say that you are the administrator of a Google Apps (old) or G-suite account. The goal is to store data on an individual's Drive account, who IS a member of the domain. We'll call the domain example.com, and the user . There's a few steps we need to go through to accomplish this: 1. Create a service account for example.com
2. Allowing API access to example.com Google Drive
3. Configure rclone, assuming a new install
4. Verify that it's working
Note: in case you configured a specific root folder on gdrive and rclone is unable to access the contents of that folder when using
Shared drives (team drives)If you want to configure the remote to point to a Google Shared Drive (previously known as Team Drives) then answer This will fetch the list of Shared Drives from google and allow you to configure which one you want to use. You can also type in a Shared Drive ID if you prefer. For example:
--fast-listThis remote supports It does this by combining multiple This works by combining many
These can now be combined into a single request:
The implementation of In tests, these batch requests were up to 20x faster than the regular method. Running the following command against different sized folders gives:
small folder (220 directories, 700 files):
large folder (10600 directories, 39000 files):
Modified timeGoogle drive stores modification times accurate to 1 ms. Restricted filename charactersOnly Invalid UTF-8 bytes will be replaced, as they can't be used in JSON strings. In contrast to other backends, RevisionsGoogle drive stores revisions of files. When you upload a change to an existing file to google drive using rclone it will create a new revision of that file. Revisions follow the standard google policy which at time of writing was
Deleting filesBy default rclone will send all files to the trash when deleting files. If deleting them permanently is required then use the ShortcutsIn March 2020 Google introduced a new feature in Google Drive called drive shortcuts (API). These will (by September 2020) replace the ability for files or folders to be in multiple folders at once. Shortcuts are files that link to other files on Google Drive somewhat like a symlink in unix, except they point to the underlying file data (e.g. the inode in unix terms) so they don't break if the source is renamed or moved about. Be default rclone treats these as follows. For shortcuts pointing to files:
For shortcuts pointing to folders:
The rclone backend command can be used to create shortcuts. Shortcuts can be completely ignored with the Emptying trashIf you wish to empty your trash you can use the Note that Google Drive takes some time (minutes to days) to empty the trash even though the command returns within a few seconds. No output is echoed, so there will be no confirmation even using -v or -vv. Quota informationTo view your current quota you can use the Import/Export of google documentsGoogle documents can be exported from and uploaded to Google Drive. When rclone
downloads a Google doc it chooses a format to download depending upon the When choosing a format, rclone runs down the list provided in order and chooses the first file format the doc can be exported as from the list. If the file can't be exported to a format on the formats list, then rclone will choose a format from the default list. If you prefer an archive copy then
you might use Note that rclone adds the extension to the google doc, so if it is called When importing files into Google Drive, rclone will convert all files with an extension in The conversion
must result in a file with the same extension when the Here are some examples for allowed and prohibited conversions.
This limitation can be disabled by specifying Here are the possible export extensions with their corresponding mime types. Most of these can also be used for importing, but there more that are not listed here. Some of these additional ones might only be available when the operating system provides the correct MIME type entries. This list can be changed by Google Drive at any time and might not represent the currently available conversions.
Google documents can also be exported as link files. These files will open a browser window for the Google Docs website of that document when opened. The link file extension has to be specified as a
Standard optionsHere are the Standard options specific to drive (Google Drive). --drive-client-idGoogle Application Client Id Setting your own is recommended. See https://rclone.org/drive/#making-your-own-client-id for how to create your own. If you leave this blank, it will use an internal key which is low performance. Properties:
--drive-client-secretOAuth Client Secret. Leave blank normally. Properties:
--drive-scopeScope that rclone should use when requesting access from drive. Properties:
--drive-service-account-fileService Account Credentials JSON file path. Leave blank normally. Needed only if you want use SA instead of interactive login. Leading Properties:
--drive-alternate-exportDeprecated: No longer needed. Properties:
Advanced optionsHere are the Advanced options specific to drive (Google Drive). --drive-tokenOAuth Access Token as a JSON blob. Properties:
--drive-auth-urlAuth server URL. Leave blank to use the provider defaults. Properties:
--drive-token-urlToken server url. Leave blank to use the provider defaults. Properties:
--drive-root-folder-idID of the root folder. Leave blank normally. Fill in to access "Computers" folders (see docs), or for rclone to use a non root folder as its starting point. Properties:
--drive-service-account-credentialsService Account Credentials JSON blob. Leave blank normally. Needed only if you want use SA instead of interactive login. Properties:
--drive-team-driveID of the Shared Drive (Team Drive). Properties:
--drive-auth-owner-onlyOnly consider files owned by the authenticated user. Properties:
--drive-use-trashSend files to the trash instead of deleting permanently. Defaults to true, namely sending files to the trash. Use Properties:
--drive-copy-shortcut-contentServer side copy contents of shortcuts instead of the shortcut. When doing server side copies, normally rclone will copy shortcuts as shortcuts. If this flag is used then rclone will copy the contents of shortcuts rather than shortcuts themselves when doing server side copies. Properties:
--drive-skip-gdocsSkip google documents in all listings. If given, gdocs practically become invisible to rclone. Properties:
--drive-skip-checksum-gphotosSkip MD5 checksum on Google photos and videos only. Use this if you get checksum errors when transferring Google photos or videos. Setting this flag will cause Google photos and videos to return a blank MD5 checksum. Google photos are identified by being in the "photos" space. Corrupted checksums are caused by Google modifying the image/video but not updating the checksum. Properties:
--drive-shared-with-meOnly show files that are shared with me. Instructs rclone to operate on your "Shared with me" folder (where Google Drive lets you access the files and folders others have shared with you). This works both with the "list" (lsd, lsl, etc.) and the "copy" commands (copy, sync, etc.), and with all other commands too. Properties:
--drive-trashed-onlyOnly show files that are in the trash. This will show trashed files in their original directory structure. Properties:
--drive-starred-onlyOnly show files that are starred. Properties:
--drive-formatsDeprecated: See export_formats. Properties:
--drive-export-formatsComma separated list of preferred formats for downloading Google docs. Properties:
--drive-import-formatsComma separated list of preferred formats for uploading Google docs. Properties:
--drive-allow-import-name-changeAllow the filetype to change when uploading Google docs. E.g. file.doc to file.docx. This will confuse sync and reupload every time. Properties:
--drive-use-created-dateUse file created date instead of modified date. Useful when downloading data and you want the creation date used in place of the last modified date. WARNING: This flag may have some unexpected consequences. When uploading to your drive all files will be overwritten unless they haven't been modified since their creation. And the inverse will occur while downloading. This side effect can be avoided by using the "--checksum" flag. This feature was implemented to retain photos capture date as recorded by google photos. You will first need to check the "Create a Google Photos folder" option in your google drive settings. You can then copy or move the photos locally and use the date the image was taken (created) set as the modification date. Properties:
--drive-use-shared-dateUse date file was shared instead of modified date. Note that, as with "--drive-use-created-date", this flag may have unexpected consequences when uploading/downloading files. If both this flag and "--drive-use-created-date" are set, the created date is used. Properties:
--drive-list-chunkSize of listing chunk 100-1000, 0 to disable. Properties:
--drive-impersonateImpersonate this user when using a service account. Properties:
--drive-upload-cutoffCutoff for switching to chunked upload. Properties:
--drive-chunk-sizeUpload chunk size. Must a power of 2 >= 256k. Making this larger will improve performance, but note that each chunk is buffered in memory one per transfer. Reducing this will reduce memory usage but decrease performance. Properties:
--drive-acknowledge-abuseSet to allow files which return cannotDownloadAbusiveFile to be downloaded. If downloading a file returns the error "This file has been identified as malware or spam and cannot be downloaded" with the error code "cannotDownloadAbusiveFile" then supply this flag to rclone to indicate you acknowledge the risks of downloading the file and rclone will download it anyway. Properties:
--drive-keep-revision-foreverKeep new head revision of each file forever. Properties:
--drive-size-as-quotaShow sizes as storage quota usage, not actual size. Show the size of a file as the storage quota used. This is the current version plus any older versions that have been set to keep forever. WARNING: This flag may have some unexpected consequences. It is not recommended to set this flag in your config - the recommended usage is using the flag form --drive-size-as-quota when doing rclone ls/lsl/lsf/lsjson/etc only. If you do use this flag for syncing (not recommended) then you will need to use --ignore size also. Properties:
--drive-v2-download-min-sizeIf Object's are greater, use drive v2 API to download. Properties:
--drive-pacer-min-sleepMinimum time to sleep between API calls. Properties:
--drive-pacer-burstNumber of API calls to allow without sleeping. Properties:
--drive-server-side-across-configsAllow server-side operations (e.g. copy) to work across different drive configs. This can be useful if you wish to do a server-side copy between two different Google drives. Note that this isn't enabled by default because it isn't easy to tell if it will work between any two configurations. Properties:
--drive-disable-http2Disable drive using http2. There is currently an unsolved issue with the google drive backend and HTTP/2. HTTP/2 is therefore disabled by default for the drive backend but can be re-enabled here. When the issue is solved this flag will be removed. See: https://github.com/rclone/rclone/issues/3631 Properties:
--drive-stop-on-upload-limitMake upload limit errors be fatal. At the time of writing it is only possible to upload 750 GiB of data to Google Drive a day (this is an undocumented limit). When this limit is reached Google Drive produces a slightly different error message. When this flag is set it causes these errors to be fatal. These will stop the in-progress sync. Note that this detection is relying on error message strings which Google don't document so it may break in the future. See: https://github.com/rclone/rclone/issues/3857 Properties:
--drive-stop-on-download-limitMake download limit errors be fatal. At the time of writing it is only possible to download 10 TiB of data from Google Drive a day (this is an undocumented limit). When this limit is reached Google Drive produces a slightly different error message. When this flag is set it causes these errors to be fatal. These will stop the in-progress sync. Note that this detection is relying on error message strings which Google don't document so it may break in the future. Properties:
--drive-skip-shortcutsIf set skip shortcut files. Normally rclone dereferences shortcut files making them appear as if they are the original file (see the shortcuts section). If this flag is set then rclone will ignore shortcut files completely. Properties:
--drive-skip-dangling-shortcutsIf set skip dangling shortcut files. If this is set then rclone will not show any dangling shortcuts in listings. Properties:
--drive-resource-keyResource key for accessing a link-shared file. If you need to access files shared with a link like this
Then you will need to use the first part "XXX" as the "root_folder_id" and the second part "YYY" as the "resource_key" otherwise you will get 404 not found errors when trying to access the directory. See: https://developers.google.com/drive/api/guides/resource-keys This resource key requirement only applies to a subset of old files. Note also that opening the folder once in the web interface (with the user you've authenticated rclone with) seems to be enough so that the resource key is no needed. Properties:
--drive-encodingThe encoding for the backend. See the encoding section in the overview for more info. Properties:
Backend commandsHere are the commands specific to the drive backend. Run them with
The help below will explain what arguments each command takes. See the backend command for more info on how to pass options and arguments. These can be run on a running backend using the rc command backend/command. getGet command for fetching the drive config parameters
This is a get command which will be used to fetch the various drive config parameters Usage Examples:
Options:
setSet command for updating the drive config parameters
This is a set command which will be used to update the various drive config parameters Usage Examples:
Options:
shortcutCreate shortcuts from files or directories
This command creates shortcuts from files or directories. Usage:
In the first example this creates a shortcut from the "source_item" which can be a file or a directory to the "destination_shortcut". The "source_item" and the "destination_shortcut" should be relative paths from "drive:" In the second example this creates a shortcut from the "source_item" relative to "drive:" to the "destination_shortcut" relative to "drive2:". This may fail with a permission error if the user authenticated with "drive2:" can't read files from "drive:". Options:
drivesList the Shared Drives available to this account
This command lists the Shared Drives (Team Drives) available to this account. Usage:
This will return a JSON list of objects like this
With the -o config parameter it will output the list in a format suitable for adding to a config file to make aliases for all the drives found and a combined drive.
Adding this to the rclone config file will cause those team drives to be accessible with the aliases shown. Any illegal characters will be substituted with "_" and duplicate names will have numbers suffixed. It will also add a remote called AllDrives which shows all the shared drives combined into one directory tree. untrashUntrash files and directories
This command untrashes all the files and directories in the directory passed in recursively. Usage: This takes an optional directory to trash which make this easier to use via the API.
Use the -i flag to see what would be restored before restoring it. Result:
copyidCopy files by ID
This command copies files by ID Usage:
It copies the drive file with ID given to the path (an rclone path which will be passed internally to rclone copyto). The ID and path pairs can be repeated. The path should end with a / to indicate copy the file as named to this directory. If it doesn't end with a / then the last path component will be used as the file name. If the destination is a drive backend then server-side copying will be attempted if possible. Use the -i flag to see what would be copied before copying. exportformatsDump the export formats for debug purposes
importformatsDump the import formats for debug purposes
LimitationsDrive has quite a lot of rate limiting. This causes rclone to be limited to transferring about 2 files per second only. Individual files may be transferred much faster at 100s of MiB/s but lots of small files can take a long time. Server side copies are also subject to a separate rate limit. If you see User rate limit exceeded errors, wait at least 24 hours and retry. You can disable server-side copies with Limitations of Google DocsGoogle docs will appear as size -1 in This is because rclone can't find out the size of the Google docs without downloading them. Google docs will transfer correctly with However an unfortunate consequence of this is that you may not be able to download Google docs using Duplicated filesSometimes, for no reason I've been able to track down, drive will duplicate a file that rclone uploads. Drive unlike all the other remotes can have duplicated files. Duplicated files cause problems with the syncing and you will see messages in the log about duplicates. Use Note that this isn't just a problem with rclone, even Google Photos on Android duplicates files on drive sometimes. Rclone appears to be re-copying files it shouldn'tThe most likely cause of this is the duplicated file issue above - run This can also be caused by a delay/caching on google drive's end when comparing directory listings. Specifically with team drives used in combination with --fast-list. Files that were uploaded recently may not appear on the directory list sent to rclone when using --fast-list. Waiting a moderate period of time between attempts (estimated to be approximately 1 hour) and/or not using --fast-list both seem to be effective in preventing the problem. Making your own client_idWhen you use rclone with Google drive in its default configuration you are using rclone's client_id. This is shared between all the rclone users. There is a global rate limit on the number of queries per second that each client_id can do set by Google. rclone already has a high quota and I will continue to make sure it is high enough by contacting Google. It is strongly recommended to use your own client ID as the default rclone ID is heavily used. If you have multiple services running, it is recommended to use an API key for each service. The default Google quota is 10 transactions per second so it is recommended to stay under that number as if you use more than that, it will cause rclone to rate limit and make things slower. Here is how to create your own Google Drive client ID for rclone:
Be aware that, due to the "enhanced security" recently introduced by Google, you are theoretically expected to "submit your app for verification" and then wait a few weeks(!) for their response; in practice, you can go right ahead and use the client ID and client secret with rclone, the only issue will be a very scary confirmation screen shown when you connect via your browser for rclone to be able to get its token-id (but as this only happens during the remote configuration, it's not such a big deal). (Thanks to @balazer on github for these instructions.) Sometimes, creation of an OAuth consent in Google API Console fails due to an error message “The request failed because changes to one of the field of the resource is not supported”. As a convenient workaround, the necessary Google Drive API key can be created on the Python Quickstart page. Just push the Enable the Drive API button to receive the Client ID and Secret. Note that it will automatically create a new project in the API Console. How do I find my Google Drive client ID and secret key?How to get Google Client ID and Client Secret?. Go to the Google Developers Console.. Click Select a project ➝ New Project ➝ the Create button.. Enter your Project name ➝ click the Create button.. Click OAuth consent screen in the left side menu ➝ choose User Type ➝ click the Create button.. Should Google client ID be kept secret?Storing and Displaying the Client ID and Secret
Because these are essentially equivalent to a username and password, you should not store the secret in plain text, instead only store an encrypted or hashed version, to help reduce the likelihood of the secret leaking.
What is client ID and secret ID?The Client ID is a public identifier of your application. The Client Secret is confidential and should only be used to authenticate your application and make requests to LinkedIn's APIs.
How do I get Google OAuth Credentials client ID and secret?In the Google Cloud console, go to Menu menu > APIs & Services > Credentials. ... . Click Create Credentials > OAuth client ID.. Click Application type > Chrome app.. In the "Name" field, type a name for the credential. ... . In the "Application ID" field, enter your app's unique 32-character ID string. ... . Click Create. ... . Click OK.. |