Raw multiline string in bash?
Is there a way to get an uninterpreted string in bash - it could include single and ouble quotes and Bang ! etc. ?
I want to do something like
#!/bin/bash
echo -e "Line One\nLine Two\nLine three" | python -c """
import sys
for line in sys.stdin.readlines(): print "STDIN: %s" %line
""" | awk '{print $2}'Problem is that zero STDIN: lines are printed - stdin is not being piped to the python program.
Here is a usecase: note the input size can be low GB's size:
cat "my10GBfile.dat" | python -c """ .. etc
Now using a HEREDOC in there .e.g.
#!/bin/bash
echo -e "Line One\nLine Two\nLine three" | python<<-HERE
some
multiline
python
program
HERE
| awk '{print $2}'has problem that the stdin gets coopted - and thus the input is lost.
What I really want is that uninterpreted multiline string in bash.
43 Answers
The heredoc does provide an uninterpreted multiline string (at least if you quote the delimiter); there's just no (easy) way of accessing its content.
Since STDIN is already being used for something else, you can create a new file descriptor to pass the heredoc's content to python:
exec 3<<'HERE'
import sys
print "Line Zero!\n"
for line in sys.stdin: print line
HERE
echo -e "Line One\nLine Two\nLine Three" | python /dev/fd/3No need to get fancy in any way: just single quote the multiline string:
echo -e "Line One\nLine Two\nLine three" | python -c ' import sys for line in sys.stdin.readlines(): print "STDIN: %s" %line
' | awk '{print $2}'How about (using the < <() Bash construct to emulate line-by-line input):
$ while read i; do echo -e '
#this is a multiline python program
if True: print """line: '"$i"'"""
' | python | awk '{print $0, "→", $3}'; done < <(echo -e "Line One\nLine Two\nLine Three")which gives output:
line: Line One → One
line: Line Two → Two
line: Line Three → ThreeUsing Python's internal """ syntax, the strings will be protected from everything except three quotes. Using read in this fashion will invoke the Python program once for every line. This works in the example you've given, but your actual task might be different (which is why I asked for a specific example in the comments). Perhaps the quoting "trick" might be of use anyway.
If Python is to be used at all, it seems easier to just do all the manipulation from within the Python code; it is after all a competent scripting language.
2
