Avoiding Shell Injection in Ruby, Python and PHP.
I recently found a shell injection bug in some Ruby-gem I use.
Shell injections scare me since a long time and I usually prefer to whitelist certain characters/patterns rather than to blacklist. This means that the system fails to the save side. Unfortunally it usually does fail - my whitelisting is to rigorous and data that would not cause any problems gets rejected. So I decided to take the opportunity to investigate how to prevent shell injection in my favorite scripting language (Python), the language I found the problem in and finally the language that I can not avoid (PHP).
Ruby
In Ruby you can use exec and system in a shell injection save way. If you pass a single string to them they will expand all shell characters, but if you pass the command and arguments separately, they won't be interpreted by the shell:
irb(main):011:0> system("echo -e $(seq 5)")
1 2 3 4 5
=> true
irb(main):012:0> system("echo", "-e", "$(seq 5)")
$(seq 5)
=> true
Unfortunally IO.popen and the backticks don't have such a feature. Ruby really leaves you in the rain here (and that triggers bugs like the one mentioned in the intro).
Here the escape-library jumps in:
irb(main):001:0> require 'escape'
=> true
irb(main):002:0> s=`#{Escape.shell_command(["echo", "$(seq)", "\"'`seq` && ||"])}`
=> "$(seq) \"'`seq` && ||\n"
Something like that should really become part of the ruby standard library.
Python
Since version 2.4 Python has the subprocess-module which claims to handle arguments securly:
Unlike some other popen functions, this implementation will never call /bin/sh implicitly. This means that all characters, including shell metacharacters, can safely be passed to child processes.
By default shell characters will not be expanded.
>>> import subprocess >>> subprocess.call(["ls", "*.c"]) ls: *.c: No such file or directory
However, you can tell it, to do so by setting shell=True:
>>> import subprocess
>>> subprocess.call("ls *.c",shell=True)
crack_rsa_in_qudratic_time.c
Did you notice, that the syntax slightly changed? I passed "ls *.c" instead of ["ls","*.c"].
The latter wont work without shell=True:
>>> subprocess.call(["ls", "*.c"],shell=True) a_short_proof_of_fermats_last_theorem.tex loveletter.tex crack_rsa_in_qudratic_time.c showargs.sh 0
In fact it just calls "ls". It took me quite some time to figure out, what's going on.
Digging through the subprocess source I found:
if shell:
args = ["/bin/sh", "-c"] + args
if executable is None:
executable = args[0]
….
if env is None:
os.execvp(executable, args)
else:
os.execvpe(executable, args, env)
So calling subprocess.call(["ls", "*.c"],shell=True) is (roughly) equivalent to os.execvp("/bin/sh", ["-c", "ls", "-l").
And the man page of bash explains:
-c string If the -c option is present, then commands are read from
string. If there are arguments after the string, they are
assigned to the positional parameters, starting with $0.
Reading this carfully you might guess that sh -c in fact does not "call" the program after the -c parameter but invokes the "shell-script" and sets the positional parameters $0, $1, $2, ... according to the following arguments. The following should clarify what's going on:
>>> import subprocess >>> subprocess.call(['echo $0 $1 $2 $3', 'arg0', 'arg1', 'arg2', 'arg3'], shell=True) arg0 arg1 arg2 arg3 0
This is a bit sad, since it implies that using shell=True, changes the syntax of of call() and Popen(). If you really want to use shell expansion, it is probably best to put the whole command in one string.
Read more about subprocess.
PHP
PHP offers the usual commands:
system(), exec(), passthru() and backticks. None of them protects you from shell injection by default. You have to use escapeshellarg() or escapeshellcmd().
Perl has more or less the same solutions as Ruby. Except you can use open in a safe manner too (open my($process), '-|', 'ls', @args), and the taint mode is really helpful to avoid most stupid security flaws.
The best way to avoid shell injection is to not rely on any shell functionality whatsoever. If you need to communicate with another program, that program should be running already, under the appropriate security restrictions and with its own IPC endpoint set up. If your process is quick to start but expensive to keep running, then you can go the route Apple went with launchd: build a shell that listens on a pipe or Unix socket or…, and when an inbound requests comes in, `fork` and `exec`.
@web design.
I read your argument alreay on reddit and wanted to write a post about it - at some time.
In short: I don't agree.
It's a bit like saying "to protect yourself from remote exploits, turn off your computer". It's true, but it does not really help. You call other programs, because they solve a problem for you. To turn these into longrunning 'deamons' may take quite some effort.
The spawning of a subprocess for a certain task is quite common in unix, including noteble examples like qmail by Daniel J. Bernstein. In qmail it is an explicit security feature.
While I know very little about launchd reading the "Some background—why launchd?" and "Benefits of launchctl" sections it seems to me that you misunderstood the purpose of it. Though there might reasons to run subprocess through it, e.g. if you want to replace one program by another, or you want to fine-control the amount of memory that a certain subprocess may consume without stopping your main application. If this is worth the overhead (startup, configuration, programming, …) depends on the speciffic situation.
I still plan to write more about it. Some day.
Thanks a lot for this post… your writeup of Python's subprocess gave me the right perspective to figure out a problem that's been bugging me for the better part of a day.
I found subprocess to be weird, then I figured out that the "shell = True" keyword argument really messes everything up. However, my target commands didn't function properly without it.
Security becomes a big concern when setting shell = True. Taking a closer look at the subprocess security promise: "this implementation will never call /bin/sh implicitly" (http://docs.python.org/library/subprocess.html#security). As you point out in the subprocess source, the executable is now /bin/sh. We are now explicitly calling /bin/sh … !
Your suggestion is useful for subprocess calls; "If you really want to use shell expansion, it is probably best to put the whole command in one string." This helps avoid inserting numerous positional parameters. However, this provides an opportunity for injection.
subprocess.call(r'echo arg0 arg1 arg2 arg3; echo 123; echo $HOME', shell = True)
output:
arg0 arg1 arg2 arg3
123
/home/username
A workaround is: keep things in a list (not a string) and use positional parameters. For example:
subprocess.call([r'echo $0', r'arg0 arg1 arg2 arg3; echo 123; echo $HOME'], shell = True)
Here, the output is:
arg0 arg1 arg2 arg3; echo 123; echo $HOME
That looks to be sanitized. The $0 trick with a 2 element list allows you to use subprocess with the shell (some calls will need it) while ignoring the exact number of arguments (to avoid extra work/mistakes in listing a series of positional parameters).
This is now a ridiculously long comment, but I hope it helps to avoid shell injection!
Thanks!
-Ron
@Ron
> However, this provides an opportunity for injection.
> subprocess.call(r’echo arg0 arg1 arg2 arg3; echo 123; echo $HOME’, shell = True)
>…
>/home/username
This is the desired behavior if you want shell expansion (well, almost, as shell expansion allows you to chain commands other ways than by semicolon too). Only if you know what you are doing and you really want shell expansion then you can put everything in one string.
But your tip still seems helpfull, if you want to mix expanded and unexpanded parameters:
Like in
lsopts="-l"; subprocess.call(["ls $0 *.py",lsopts], shell = True)
the parameter lsopts is protected against injection, while "*.py" will correctly expand to all py-files in the working directory.
Thank you.
@Henryk
> This is the desired behavior if you want shell expansion …
Agreed. I was faced with a couple inherited binaries which wouldn't work in subprocess without the shell, but I still needed to prevent injection. Your lsopts example is a good one.
Take care,
Ron