Avoiding Shell Injection in Ruby, Python and PHP.

I recently found a shell injection bug in some Ruby-gem I use.

Shell injections have scared me for a long time and I usually prefer to whitelist certain characters/patterns rather than to blacklist. This means that the system fails to the save side. Unfortunally it usually does fail – my whitelisting is to rigorous and data that would not cause any problems gets rejected. So I decided to take the opportunity to investigate how to prevent shell injection in my favorite scripting language (Python), the language I found the problem in and finally the language that I can not avoid (PHP).

Ruby

In Ruby you can use exec and system in a shell injection save way. If you pass a single string to them they will expand all shell characters, but if you pass the command and arguments separately, they won't be interpreted by the shell:

irb(main):011:0> system("echo -e $(seq 5)")
1 2 3 4 5
=> true
irb(main):012:0> system("echo", "-e", "$(seq 5)")
$(seq 5)
=> true

Unfortunally IO.popen and the backticks don't have such a feature. Ruby really leaves you in the rain here (and that triggers bugs like the one mentioned in the intro).

UPDATE: As 13k mentioned below there is a stdlib method now: shellescape.

Here the escape-library jumps in:

irb(main):001:0> require 'escape'
=> true
irb(main):002:0> s=`#{Escape.shell_command(["echo", "$(seq)", "\"'`seq` && ||"])}`
=> "$(seq) \"'`seq` && ||\n"

Something like that should really become part of the ruby standard library.

Python

Since version 2.4 Python has the subprocess-module which claims to handle arguments securly:

Unlike some other popen functions, this implementation will never call /bin/sh implicitly. This means that all characters, including shell metacharacters, can safely be passed to child processes.

By default shell characters will not be expanded.

>>> import subprocess
>>> subprocess.call(["ls", "*.c"])
ls: *.c: No such file or directory

However, you can tell it, to do so by setting shell=True:

>>> import subprocess
>>> subprocess.call("ls *.c",shell=True)
crack_rsa_in_qudratic_time.c

Did you notice, that the syntax slightly changed? I passed "ls *.c" instead of ["ls","*.c"].
The latter wont work without shell=True:

>>> subprocess.call(["ls", "*.c"],shell=True)
a_short_proof_of_fermats_last_theorem.tex  loveletter.tex 
crack_rsa_in_qudratic_time.c               showargs.sh
0

In fact it just calls "ls". It took me quite some time to figure out, what's going on.
Digging through the subprocess source I found:

            if shell:
                args = ["/bin/sh", "-c"] + args

            if executable is None:
                executable = args[0]
           ....
                       if env is None:
                        os.execvp(executable, args)
                    else:
                        os.execvpe(executable, args, env)

So calling subprocess.call(["ls", "*.c"],shell=True) is (roughly) equivalent to os.execvp("/bin/sh", ["-c", "ls", "-l").
And the man page of bash explains:

       -c string If  the  -c  option  is  present, then commands are read from
                 string.  If there are arguments after the  string,  they  are
                 assigned to the positional parameters, starting with $0.

Reading this carfully you might guess that sh -c in fact does not "call" the program after the -c parameter but invokes the "shell-script" and sets the positional parameters $0, $1, $2, ... according to the following arguments. The following should clarify what's going on:

>>> import subprocess
>>> subprocess.call(['echo $0 $1 $2 $3', 'arg0', 'arg1', 'arg2', 'arg3'], shell=True)
arg0 arg1 arg2 arg3
0

This is a bit sad, since it implies that using shell=True, changes the syntax of of call() and Popen(). If you really want to use shell expansion, it is probably best to put the whole command in one string.

Read more about subprocess.

PHP

PHP offers the usual commands:
system(), exec(), passthru() and backticks. None of them protects you from shell injection by default. You have to use escapeshellarg() or escapeshellcmd().

Share and Enjoy:
  • description
  • Reddit
  • Digg
  • Google
  • del.icio.us
  • MisterWong

10 Comments

  1. Jedai says:

    Perl has more or less the same solutions as Ruby. Except you can use open in a safe manner too (open my($process), '-|', 'ls', @args), and the taint mode is really helpful to avoid most stupid security flaws.

  2. web design says:

    The best way to avoid shell injection is to not rely on any shell functionality whatsoever. If you need to communicate with another program, that program should be running already, under the appropriate security restrictions and with its own IPC endpoint set up. If your process is quick to start but expensive to keep running, then you can go the route Apple went with launchd: build a shell that listens on a pipe or Unix socket or…, and when an inbound requests comes in, `fork` and `exec`.

  3. Henryk says:

    @web design.
    I read your argument alreay on reddit and wanted to write a post about it – at some time.

    In short: I don't agree.
    It's a bit like saying "to protect yourself from remote exploits, turn off your computer". It's true, but it does not really help. You call other programs, because they solve a problem for you. To turn these into longrunning 'deamons' may take quite some effort.
    The spawning of a subprocess for a certain task is quite common in unix, including noteble examples like qmail by Daniel J. Bernstein. In qmail it is an explicit security feature.

    While I know very little about launchd reading the "Some background—why launchd?" and "Benefits of launchctl" sections it seems to me that you misunderstood the purpose of it. Though there might reasons to run subprocess through it, e.g. if you want to replace one program by another, or you want to fine-control the amount of memory that a certain subprocess may consume without stopping your main application. If this is worth the overhead (startup, configuration, programming, …) depends on the speciffic situation.

    I still plan to write more about it. Some day.

  4. Ron says:

    Thanks a lot for this post… your writeup of Python's subprocess gave me the right perspective to figure out a problem that's been bugging me for the better part of a day.

    I found subprocess to be weird, then I figured out that the "shell = True" keyword argument really messes everything up. However, my target commands didn't function properly without it.

    Security becomes a big concern when setting shell = True. Taking a closer look at the subprocess security promise: "this implementation will never call /bin/sh implicitly" (http://docs.python.org/library/subprocess.html#security). As you point out in the subprocess source, the executable is now /bin/sh. We are now explicitly calling /bin/sh … !

    Your suggestion is useful for subprocess calls; "If you really want to use shell expansion, it is probably best to put the whole command in one string." This helps avoid inserting numerous positional parameters. However, this provides an opportunity for injection.

    subprocess.call(r'echo arg0 arg1 arg2 arg3; echo 123; echo $HOME', shell = True)

    output:
    arg0 arg1 arg2 arg3
    123
    /home/username

    A workaround is: keep things in a list (not a string) and use positional parameters. For example:

    subprocess.call([r'echo $0', r'arg0 arg1 arg2 arg3; echo 123; echo $HOME'], shell = True)

    Here, the output is:
    arg0 arg1 arg2 arg3; echo 123; echo $HOME

    That looks to be sanitized. The $0 trick with a 2 element list allows you to use subprocess with the shell (some calls will need it) while ignoring the exact number of arguments (to avoid extra work/mistakes in listing a series of positional parameters).

    This is now a ridiculously long comment, but I hope it helps to avoid shell injection!
    Thanks!

    -Ron

  5. Henryk says:

    @Ron
    > However, this provides an opportunity for injection.
    > subprocess.call(r’echo arg0 arg1 arg2 arg3; echo 123; echo $HOME’, shell = True)
    >…
    >/home/username
    This is the desired behavior if you want shell expansion (well, almost, as shell expansion allows you to chain commands other ways than by semicolon too). Only if you know what you are doing and you really want shell expansion then you can put everything in one string.

    But your tip still seems helpfull, if you want to mix expanded and unexpanded parameters:
    Like in
    lsopts="-l"; subprocess.call(["ls $0 *.py",lsopts], shell = True)
    the parameter lsopts is protected against injection, while "*.py" will correctly expand to all py-files in the working directory.

    Thank you.

  6. Ron says:

    @Henryk
    > This is the desired behavior if you want shell expansion …

    Agreed. I was faced with a couple inherited binaries which wouldn't work in subprocess without the shell, but I still needed to prevent injection. Your lsopts example is a good one.
    Take care,

    Ron

  7. Your Friendly Grammar Nazi says:

    "Shell injections scare me since a long time"

    The correct way to say this is:

    "Shell injections have scared me for a long time."

    When using "since", you use a definitive point in time:

    "Shell injections have scared me since 2001."

  8. Henryk says:

    @YFGN
    Thanks for the english lesson.

  9. 2013 says:

    Apparently, you now can separately pass args to Open3.capture3 as well such that no intermediate shell is used.

Leave a Reply