Having made a slight cock-up, I want to use rsync to move all the files
that exist on one server but not the other into a separate heirarchy.

More fully:

I have two identical servers, the second of which rsyncs itself from the
other overnight. This morning, due to an oversight, I put a load of
files into a directory structure that were intended to go into a
subfolder of it. What I want to do is rsync from the second server to
the first and move files that only exist on the first into a directory
tree elsewhere on the server, so that then I can move that extraneous
tree back into the originally intended subfolder. Something like this:

rsync [usual options] --backupdir=/LocalRoot/Elsewhere rsync://user@second/BackUp/TBD/ /LocalRoot/BackUp/TBD/

mv /LocalRoot/Elsewhere /LocalRoot/BackUp/TBD/

However, when I tried the first rsync step with the -n switch to see
what would most likely happen, nothing happened until I added a --delete option, then a load of 'deleting ...' messages for the relevant files
were printed, but I don't want to actually delete them, I want to move
them to their corresponding position in a new & separate tree under
Elsewhere.

I presume that the delete messages mean that the extra files will
actually be deleted not moved, so how can I get them to be moved?

--

Fake news kills!

I may be contacted via the contact address given on my website: www.macfh.co.uk

--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21, Java Jive wrote:

Having made a slight cock-up, I want to use rsync to move all the files
that exist on one server but not the other into a separate heirarchy.

More fully:

I have two identical servers, the second of which rsyncs itself from the other overnight. This morning, due to an oversight, I put a load of
files into a directory structure that were intended to go into a
subfolder of it. What I want to do is rsync from the second server to
the first and move files that only exist on the first into a directory
tree elsewhere on the server, so that then I can move that extraneous
tree back into the originally intended subfolder. Something like this:

rsync doesn't 'move'.

*IF* you've already moved the relevant files from /src/wrongdir to /src/correctdir ; then you can rsync with "--delete" to delete those
files on /dest/ that are no longer in /src/ (i.e. 'got moved from
/src/wrong/ to /src/correct/').

I would definitely do the --dry-run, and read it *very* closely, to make
sure the rsync isn't removing more than I expect.


--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21 14:49, Dan Purgert wrote:

On 2025-10-21, Java Jive wrote:

Having made a slight cock-up, I want to use rsync to move all the files
that exist on one server but not the other into a separate heirarchy.

More fully:

I have two identical servers, the second of which rsyncs itself from the
other overnight. This morning, due to an oversight, I put a load of
files into a directory structure that were intended to go into a
subfolder of it. What I want to do is rsync from the second server to
the first and move files that only exist on the first into a directory
tree elsewhere on the server, so that then I can move that extraneous
tree back into the originally intended subfolder. Something like this:

rsync doesn't 'move'.

*IF* you've already moved the relevant files from /src/wrongdir to /src/correctdir ; then you can rsync with "--delete" to delete those
files on /dest/ that are no longer in /src/ (i.e. 'got moved from
/src/wrong/ to /src/correct/').

I would definitely do the --dry-run, and read it *very* closely, to make
sure the rsync isn't removing more than I expect.

Yes, after posting I did a test along the following lines which proved
the files would be deleted, not moved ...

mkdir 1temp 2temp elsewhere
touch 1temp/test
touch 2temp/extratest

rsync -a --backup-dir=elsewhere 1temp/ 2temp/

ls -al 1temp
.
..
test
ls -al 2temp
.
..
test
ls -al elsewhere
.
..

:-(

So I think it's going to have to be something like ...

1) Move the directory with the extra files to 'Elsewhere'
2) Recreate the original using rsync from the second server
3) Compare the two deleting files from Elsewhere that exist in the original

Tedious, but should work.

--

Fake news kills!

I may be contacted via the contact address given on my website: www.macfh.co.uk


--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21, Java Jive wrote:

On 2025-10-21 14:49, Dan Purgert wrote:

On 2025-10-21, Java Jive wrote:

Having made a slight cock-up, I want to use rsync to move all the files
that exist on one server but not the other into a separate heirarchy.

More fully:

I have two identical servers, the second of which rsyncs itself from the >>> other overnight. This morning, due to an oversight, I put a load of
files into a directory structure that were intended to go into a
subfolder of it. What I want to do is rsync from the second server to
the first and move files that only exist on the first into a directory
tree elsewhere on the server, so that then I can move that extraneous
tree back into the originally intended subfolder. Something like this:

rsync doesn't 'move'.

*IF* you've already moved the relevant files from /src/wrongdir to
/src/correctdir ; then you can rsync with "--delete" to delete those
files on /dest/ that are no longer in /src/ (i.e. 'got moved from
/src/wrong/ to /src/correct/').

I would definitely do the --dry-run, and read it *very* closely, to make
sure the rsync isn't removing more than I expect.

Yes, after posting I did a test along the following lines which proved
the files would be deleted, not moved ...

mkdir 1temp 2temp elsewhere
touch 1temp/test
touch 2temp/extratest

rsync -a --backup-dir=elsewhere 1temp/ 2temp/

ls -al 1temp
.
..
test
ls -al 2temp
.
..
test
ls -al elsewhere
.
..

:-(

rsync cannot "backup" something it doesn't know about or isn't
changing... likewise your command shouldn't have deleted "extratest"

dan@framework:~/rsync$ mkdir 1temp 2temp elsewhere
dan@framework:~/rsync$ touch 1temp/test
dan@framework:~/rsync$ touch 2temp/extratest
dan@framework:~/rsync$ rsync -av --backup-dir=elsewhere 1temp/ 2temp/
sending incremental file list
../
test

sent 116 bytes received 38 bytes 308.00 bytes/sec
total size is 0 speedup is 0.00

dan@framework:~/rsync$ ls -la 1temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la 2temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 extratest
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la elsewhere
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..


Now, if we add in '--delete' ...


dan@framework:~/rsync$ rsync -av --delete --backup-dir=elsewhere 1temp/
2temp/ sending incremental file list
deleting extratest

sent 70 bytes received 25 bytes 190.00 bytes/sec
total size is 0 speedup is 0.00
dan@framework:~/rsync$ ls -la 1temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la 2temp
total 12
drwxr-xr-x 3 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
drwxr-xr-x 2 dan dan 4096 Oct 21 11:33 elsewhere
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la elsewhere
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
dan@framework:~/rsync$ ls -la 2temp/elsewhere/
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:33 .
drwxr-xr-x 3 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 extratest

NOTE -- the deleted file went into "2temp/elsewhere", because of the
behavior of --backup-dir:

Note that if you specify a relative path, the backup directory
will be relative to the destination directory,




--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21 16:37, Dan Purgert wrote:

On 2025-10-21, Java Jive wrote:

On 2025-10-21 14:49, Dan Purgert wrote:

On 2025-10-21, Java Jive wrote:

Having made a slight cock-up, I want to use rsync to move all the files >>>> that exist on one server but not the other into a separate heirarchy.

More fully:

I have two identical servers, the second of which rsyncs itself from the >>>> other overnight. This morning, due to an oversight, I put a load of
files into a directory structure that were intended to go into a
subfolder of it. What I want to do is rsync from the second server to >>>> the first and move files that only exist on the first into a directory >>>> tree elsewhere on the server, so that then I can move that extraneous
tree back into the originally intended subfolder. Something like this: >>>

rsync doesn't 'move'.

*IF* you've already moved the relevant files from /src/wrongdir to
/src/correctdir ; then you can rsync with "--delete" to delete those
files on /dest/ that are no longer in /src/ (i.e. 'got moved from
/src/wrong/ to /src/correct/').

I would definitely do the --dry-run, and read it *very* closely, to make >>> sure the rsync isn't removing more than I expect.

Yes, after posting I did a test along the following lines which proved
the files would be deleted, not moved ...

mkdir 1temp 2temp elsewhere
touch 1temp/test
touch 2temp/extratest

rsync -a --backup-dir=elsewhere 1temp/ 2temp/

ls -al 1temp
.
..
test
ls -al 2temp
.
..
test
ls -al elsewhere
.
..

:-(

rsync cannot "backup" something it doesn't know about or isn't
changing... likewise your command shouldn't have deleted "extratest"

dan@framework:~/rsync$ mkdir 1temp 2temp elsewhere
dan@framework:~/rsync$ touch 1temp/test
dan@framework:~/rsync$ touch 2temp/extratest
dan@framework:~/rsync$ rsync -av --backup-dir=elsewhere 1temp/ 2temp/
sending incremental file list
./
test

sent 116 bytes received 38 bytes 308.00 bytes/sec
total size is 0 speedup is 0.00

dan@framework:~/rsync$ ls -la 1temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la 2temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 extratest
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la elsewhere
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..

My mistake, in my post above I didn't transcribe the process directly
because for other reasons I'd already logged out of the server. Below I
do so, but get the same result ...

Now, if we add in '--delete' ...

dan@framework:~/rsync$ rsync -av --delete --backup-dir=elsewhere 1temp/ 2temp/ sending incremental file list
deleting extratest

sent 70 bytes received 25 bytes 190.00 bytes/sec
total size is 0 speedup is 0.00
dan@framework:~/rsync$ ls -la 1temp
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la 2temp
total 12
drwxr-xr-x 3 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
drwxr-xr-x 2 dan dan 4096 Oct 21 11:33 elsewhere
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 test
dan@framework:~/rsync$ ls -la elsewhere
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:31 .
drwxr-xr-x 5 dan dan 4096 Oct 21 11:31 ..
dan@framework:~/rsync$ ls -la 2temp/elsewhere/
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 11:33 .
drwxr-xr-x 3 dan dan 4096 Oct 21 11:31 ..
-rw-r--r-- 1 dan dan 0 Oct 21 11:31 extratest

NOTE -- the deleted file went into "2temp/elsewhere", because of the
behavior of --backup-dir:

Note that if you specify a relative path, the backup directory
will be relative to the destination directory,

Unfortunately, this doesn't happen with this rsync on these QNAP servers whether or not I specify an absolute path to elsewhere:

[user@server Test]# mkdir 1temp 2temp elsewhere
[user@server Test]# touch 1temp/test
[user@server Test]# touch 2temp/extratest
[user@server Test]# rsync -av --backup-dir=elsewhere --delete 1temp/ 2temp/ sending incremental file list
2./
deleting extratest
test

sent 81 bytes received 34 bytes 230.00 bytes/sec
total size is 0 speedup is 0.00
[user@server Test]# ls -al 1temp
total 8
drwxr-xr-x 2 user user 4096 2025-10-21 17:28 ./
drwxrwx--- 8 user user 4096 2025-10-21 17:27 ../
-rw-r--r-- 1 user user 0 2025-10-21 17:28 test
[user@server Test]# ls -al 2temp
total 8
drwxr-xr-x 2 user user 4096 2025-10-21 17:28 ./
drwxrwx--- 8 user user 4096 2025-10-21 17:27 ../
-rw-r--r-- 1 user user 0 2025-10-21 17:28 test
[user@server Test]# ls -al elsewhere
total 8
drwxr-xr-x 2 user user 4096 2025-10-21 17:27 ./
drwxrwx--- 8 user user 4096 2025-10-21 17:27 ../
[user@server Test]# mkdir 2temp/elsewhere
[user@server Test]# touch 2temp/extratest
[user@server Test]# rsync -av --backup-dir=elsewhere --delete 1temp/ 2temp/ sending incremental file list
../
deleting elsewhere/
deleting extratest

sent 42 bytes received 15 bytes 114.00 bytes/sec
total size is 0 speedup is 0.00
[user@server Test]# ls -al elsewhere
total 8
drwxr-xr-x 2 user user 4096 2025-10-21 17:27 ./
drwxrwx--- 8 user user 4096 2025-10-21 17:27 ../
[user@server Test]# touch 2temp/extratest
[user@server Test]# rsync -av --backup-dir=/LocalPath/elsewhere --delete 1temp/ 2temp/
sending incremental file list
../
deleting extratest

sent 46 bytes received 15 bytes 122.00 bytes/sec
total size is 0 speedup is 0.00
[user@server Test]# ls -al /LocalPath/elsewhere
total 8
drwxr-xr-x 2 user user 4096 2025-10-21 17:27 ./
drwxrwx--- 8 user user 4096 2025-10-21 17:27 ../

--

Fake news kills!

I may be contacted via the contact address given on my website: www.macfh.co.uk


--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21, Java Jive wrote:

On 2025-10-21 16:37, Dan Purgert wrote:

rsync cannot "backup" something it doesn't know about or isn't
changing... likewise your command shouldn't have deleted "extratest"

My mistake, in my post above I didn't transcribe the process directly because for other reasons I'd already logged out of the server. Below I
do so, but get the same result ...

[...]

Unfortunately, this doesn't happen with this rsync on these QNAP servers whether or not I specify an absolute path to elsewhere:

[user@server Test]# mkdir 1temp 2temp elsewhere
[user@server Test]# touch 1temp/test
[user@server Test]# touch 2temp/extratest
[user@server Test]# rsync -av --backup-dir=elsewhere --delete 1temp/ 2temp/

Could be a version thing, yeah. Here's what I'm working with:

dan@framework:~/rsync$ rsync --version
rsync version 3.2.7 protocol version 32
[...more stuff...]


For ref, did the exact commands you posted, just in case my previous
test somehow handled the backup-dir because I did rsync without delete
first.

dan@framework:~/rsync$ mkdir 1temp 2temp
dan@framework:~/rsync$ touch 1temp/test
dan@framework:~/rsync$ touch 2temp/extratest

(NB: edited command to deal with line-wrapping)

dan@framework:~/rsync$ rsync -av --backup-dir=elsewhere --delete \
1temp/ 2temp/
sending incremental file list
deleting extratest
../
test

sent 116 bytes received 51 bytes 334.00 bytes/sec
total size is 0 speedup is 0.00
dan@framework:~/rsync$ ls -al 1temp/*
-rw-r--r-- 1 dan dan 0 Oct 21 15:32 1temp/test
dan@framework:~/rsync$ ls -al 2temp/*
-rw-r--r-- 1 dan dan 0 Oct 21 15:32 2temp/test

2temp/elsewhere:
total 8
drwxr-xr-x 2 dan dan 4096 Oct 21 15:33 .
drwxr-xr-x 3 dan dan 4096 Oct 21 15:32 ..
-rw-r--r-- 1 dan dan 0 Oct 21 15:32 extratest


There might also be the issue that the testing is using zero-byte files,
that has caused me heartache in the past.

Realistically though, as I understand your actual situation, this series
of commands is closer to what happened, right?

(1) Created some new source directory (and files), then ran rsync dan@framework:~/rsync$ mkdir -p src/oops dest
dan@framework:~/rsync$ touch src/oops/somefile.txt
dan@framework:~/rsync$ rsync -av src/ dest
sending incremental file list
oops/
oops/somefile.txt

sent 157 bytes received 39 bytes 392.00 bytes/sec
total size is 0 speedup is 0.00

(2) Realized there was a mistake, corrected the source tree... dan@framework:~/rsync$ mkdir src/correct
dan@framework:~/rsync$ mv src/oops/somefile.txt src/correct/somefile.txt dan@framework:~/rsync$ rmdir src/oops


(3)need to re-sync with the destination, and cleanup the wrong files.
Dry-run to make sure stuff we expect to happen will happen...

(NB: edited the command to deal with line-wrapping)

dan@framework:~/rsync$ rsync -av --backup-dir=backupdir --delete \
--dry-run src/ dest
sending incremental file list
deleting oops/somefile.txt
deleting oops/
../
correct/
correct/somefile.txt

sent 130 bytes received 56 bytes 372.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)

(4) dry-run looked good, so run without that switch.

(NB: Edited the command to deal with line-wrapping)

dan@framework:~/rsync$ rsync -av --backup-dir=backupdir --delete \
src/ dest
sending incremental file list
deleting oops/somefile.txt
deleting oops/
../
correct/
correct/somefile.txt

sent 170 bytes received 76 bytes 492.00 bytes/sec
total size is 0 speedup is 0.00

(5) checked the result
dan@framework:~/rsync$ ls -al src
total 12
drwxr-xr-x 3 dan dan 4096 Oct 21 15:42 .
drwxr-xr-x 4 dan dan 4096 Oct 21 15:42 ..
drwxr-xr-x 2 dan dan 4096 Oct 21 15:42 correct
dan@framework:~/rsync$ ls -al dest/
total 16
drwxr-xr-x 4 dan dan 4096 Oct 21 15:42 .
drwxr-xr-x 4 dan dan 4096 Oct 21 15:42 ..
drwxr-xr-x 3 dan dan 4096 Oct 21 15:43 backupdir
drwxr-xr-x 2 dan dan 4096 Oct 21 15:42 correct


--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)

On 2025-10-21 20:52, Dan Purgert wrote:

Realistically though, as I understand your actual situation, this series
of commands is closer to what happened, right?

Things were rather more complicated than that ...

As I have posted a number of times in the past, I have been scanning
many documents from two branches of my family prior to placing them in
one of Scotland's public document archives. This work is nearing
completion of its third stage, there will be one more stage before it is complete.

Because the current stage is near completion, I wanted to reclaim disk
space on my Windows PC, some of which is taken up by interim
post-processing files deleted into the RecycleBin, but did not want to
lose them entirely just yet by emptying the bin.

I could just have dragged them out of the bin onto a server folder, but
that way I'd have lost the directory structure they were originally part
of, whereas I wanted to recreate the files in their original place in
the directory tree, but not have them muddled up with the final versions
on the data drive, D:, and also, if there were more than one version of
a particular file to be restored, restoring them rather than dragging
and dropping them would number extra versions appropriately.

So I gave the data drive a new drive letter and rebooted, then in a
Command console typed ...

NET USE D: \\Server\Share\Path

.... and restored all the files from the RecycleBin. Then I did ...

NET USE D: /D

.... and restored the data drive to being D:, and rebooted again.

Trouble was, I'd unwittingly left the last directory off the path I gave
to the above command, thus giving myself a new problem which was the
subject of this thread. However, I've solved it now, in a similar
manner to that suggested previously.

--

Fake news kills!

I may be contacted via the contact address given on my website: www.macfh.co.uk


--- MBSE BBS v1.1.1 (Linux-x86_64)
* Origin: A noiseless patient Spider (2:250/1@fidonet)