Properly escaping forward slash in bash script for usage with sed
I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync.
Currently I'm stuck at prefixing the files by their parent folder. I found out how to prefix lines using sed and how to escape using sed, but I'm having troubles combining those two.
This is how far I got:
source="/my/source/folder/"
target="/my/target/folder/"
escaped=`echo "$source" | sed -e 's/[\/&]/\\//g'`
du `rsync -ahnv $source $target | tail -n +2 | head -n -3 | sed "s/^/$escaped/"` | awk '{i+=$1} END {print i}'This is the output I get from bash -x myscript.sh
+ source=/my/source/folder/
+ target=/my/target/folder
++ echo /my/source/folder/
++ sed -e 's/[\/&]/\//g'
+ escaped=/my/source/folder/
+ awk '{i+=$1} END {print i}'
++ rsync -ahnv /my/source/folder/ /my/target/folder/
++ sed 's/^//my/source/folder//'
++ head -n -3
++ tail -n +2
sed: -e expression #1, char 8: unknown option to `s'
+ du
80268Any ideas on how to properly escape would be highly appreciated.
2 Answers
You don't actually need to bother with escaping the /. From the GNU sed manual:
The
/characters may be uniformly replaced by any other single character within any givenscommand.
For example:
echo 'foobar' | sed -e 's#foo#bar#'gives the output
barbarIf you are trying to tally up the data, you will likely want to skip over any possible subdirectories. This may not be applicable, but it's always best to avoid any weirdness. Here is my source directory. These files were all created with touch, so they take up 0 space.
$ find source
source
source/5832
source/5832/5832_1.png
source/5832/5832_2.png
source/5831
source/5831/5831_1.png
source/5830
source/5830/5830_3.png
source/5830/5830_1.png
source/5830/5830_2.pngYou shouldn't need to escape anywhere near as much as you are doing. The sed block in here will do the following:
1d; delete the first line of 'sending incremental file list'
/^$/,$d; delete the blank line that signifies end of file list and everything after it.
//$/d delete any line ending in a '/', which would be the directories
Note that you can escape the slashes with backslashes (to answer the actual question).
$ du $(rsync -ahnv source target | sed '1d;/^$/,$d;/\/$/d')
0 source/5830/5830_1.png
0 source/5830/5830_2.png
0 source/5830/5830_3.png
0 source/5831/5831_1.png
0 source/5832/5832_1.png
0 source/5832/5832_2.pngAnd for extra bonus points, you can even tally up the results:
$ du -s $(rsync -ahnv source target | sed '1d;/^$/,$d;/\/$/d') | awk '{x+=$1;print $0} END {print "sum: "x}'
0 source/5830/5830_1.png
0 source/5830/5830_2.png
0 source/5830/5830_3.png
0 source/5831/5831_1.png
0 source/5832/5832_1.png
0 source/5832/5832_2.png
sum: 0If you are doing this with the directory already fully synced, it will return funny results, though:
76 .
sum: 76
