Best ways to copy files between two block devices

There are several options on how to transfer data from one volume to another. In this article we will cover two that should cover most use cases.

Option 1: dd to create a 'carbon copy' of volume

CARELESS USAGE OF dd CAN RESULT IN DATA LOSS dd overwrites the contents of the output file/device, please always double check that the if= points to input file/device, of= points to output file/device and those files/devices are not confused with other files/devices.

You can see the device path for each volume attached to instance in storage tab of selected instance

To create a carbon copy of input device and write it to other device/file simply run:

dd if=/path/to/input of=/path/to/output bs=96M status=progress

If we were intending to clone OS-viJ5Hd2x to Volume-2th9Aa6j (see the above picture), the command would be:

dd if=/dev/vda of=/dev/vdb bs=96M status=progress

Option 2: rclone to copy folders and files between two mounted volumes

See Attaching a block volume to know how to mount volume to system

In this guide we will focus on copying to volumes that are mounted to system, therefore it will be considered local from rclone's point of view. You can also use

rclone config

to configure remote storage device to use with rclone. See rclone docs for more information.

First, install rclone, a powerful utility for copying files locally and from/to remote disks.

apt install rclone

Then the command to copy the files would be:

rclone copy --links --progress --metadata --ignore-checksum --log-file=/tmp/rclone-$( date -I ) --exclude="/excluded_dir/" SRC DST

Change SRC folder you want to copy, change DST to destination folder where files from SRC should go to and adapt the --exclude flags to what you want to exclude. In case you need maximum performance you can add some flags to the above command and adapt their values to your setup:

--transfers=8 --checkers=1 --multi-thread-streams=8 --multi-thread-cutoff 256Mi

Read more about these options at rclone flags.