Sandboxing proprietary applications with Docker

About

Docker is an open-source project that automates the deployment of applications inside software containers, providing that way an additional layer of abstraction and automatization of operating system-level virtualization on Linux. Docker uses resource isolation features of the Linux kernel such as cgroups and kernel namespaces to allow independent “containers” to run within a single Linux instance, avoiding the overhead of starting virtual machines.

https://en.wikipedia.org/wiki/Docker_%28software%29

Getting Started

The Docker documentations is very good, so this is where you should start: https://docs.docker.com/.

Installing Docker

Just follow the guide for your Linux distribution: https://docs.docker.com/installation/#installation.

On Fedora it looks like this:

[chris@thinkpad ~]$ sudo yum -y install docker-io
[chris@thinkpad ~]$ sudo systemctl start docker
[chris@thinkpad ~]$ sudo systemctl enable docker
[chris@thinkpad ~]$ sudo usermod -a -G docker chris # to run docker without sudo

Note

To securely use USB devices inside Docker containers we need at least Docker 1.2.0. Since Fedora 20 is still shipping Docker 1.1.2 we have to enable the updates-testing repository while installing docker:

[chris@thinkpad ~]$ sudo yum --enablerepo=updates-testing install docker-io

User Guide

Now follow the user guide to get a bit familiar with the basics of Docker: https://docs.docker.com/userguide/.

Docker Security

Some notes about Docker security:

Docker supports a so called “privileged” mode that was previously necessary to access USB devices for example (docker run -t -i -privileged -v /dev/bus/usb:/dev/bus/usb ubuntu bash). But this mode is really unsafe and should not longer be used. The Docker 1.2 release introduced two new flags for docker run --cap-add and --cap-drop that give a more fine grain control over the capabilities of a particular container.

One of the (many!) features of Docker 0.6 is the new “privileged” mode for containers. It allows you to run some containers with (almost) all the capabilities of their host machine, regarding kernel features and device access. [...] Note, however, that there are serious security implications there: since the private Docker instances run in privileged mode, they can easily escalate to the host, and you probably don’t want this!http://blog.docker.com/2013/09/docker-can-now-run-within-docker/

Dockerizing your Application

The Interactive Approach

Basics

Start a new Docker container with Ubuntu 14.04, the first time you start it it will download the Ubuntu 14.04 base image (213 MB) automatically:

[chris@thinkpad ~]$ docker run ubuntu:14.04 /bin/echo 'Hello world'

Note

Fedora:

[chris@thinkpad ~]$ docker run -i -t fedora /bin/bash
[chris@thinkpad ~]$ docker run -i -t fedora:20 /bin/bash

Let’s see what we can do with that image (https://docs.docker.com/userguide/dockerimages/):

[chris@thinkpad ~]$ docker run -it ubuntu:14.04 /bin/bash
root@65a77c4901b3:/# echo "test"> test.txt
root@65a77c4901b3:/# cat test.txt
test
root@65a77c4901b3:/# exit
exit
[chris@thinkpad ~]$ docker run -it ubuntu:14.04 /bin/bash
root@c688e3146470:/# cat test.txt
cat: test.txt: No such file or directory
root@c688e3146470:/#

As seen above, if we change something and start the image again a new container will be created and all changes are gone, but nothing is lost as we can restart old containers (https://docs.docker.com/userguide/usingdocker/):

The command docker ps shows all our running containers:

[chris@thinkpad ~]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED
c688e3146470        ubuntu:14.04        "/bin/bash"         About a minute ago

STATUS              PORTS               NAMES
Up About a minute                       drunk_torvalds
[chris@thinkpad ~]$

Whereas docker ps -a lists all containers:

[chris@thinkpad ~]$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED
c688e3146470        ubuntu:14.04        "/bin/bash"         4 minutes ago
65a77c4901b3        ubuntu:14.04        "/bin/bash"         5 minutes ago

STATUS                     PORTS               NAMES
Up 4 minutes                                   drunk_torvalds
Exited (0) 4 minutes ago                       stupefied_mestorf
[chris@thinkpad ~]$

We can use the name of the old container (stupefied_mestorf) to start it again and reattach:

[chris@thinkpad ~]$ docker start stupefied_mestorf
stupefied_mestorf
[chris@thinkpad ~]$ docker attach stupefied_mestorf
root@65a77c4901b3:/# cat test.txt
test
root@65a77c4901b3:/#

Short form (with container ID or container name):

[chris@thinkpad ~]$ docker start -ai 65a77c4901b3
root@65a77c4901b3:/# cat test.txt
test
root@65a77c4901b3:/#

Shared Folders

Setting up a shared folder to exchange files with the host is quite easy, lets say we want to share the directory /home/chris/share/docker/, all you have to do is to specify it with the -v flag:

[chris@thinkpad ~]$ docker run -it -v /home/chris/share/docker/:/share:rw ubuntu:14.04 /bin/bash
root@ae6a18e47f64:/# ls /share/
Text File
root@ae6a18e47f64:/# echo "test" > share/test.txt
root@ae6a18e47f64:/# exit
[chris@thinkpad ~]$ ls -l share/docker/
total 8
-rw-r--r--. 1 root  root  5 Sep  7 16:19 test.txt
-rw-rw-r--. 1 chris chris 5 Sep  4 17:40 Text File
[chris@thinkpad ~]$

If you are using Fedora and have SELinux enabled it is necessary to change the security context of the directory to get rid of a permission denied error:

[chris@thinkpad ~]$ sudo chcon -Rt svirt_sandbox_file_t /home/chris/share/docker/

Else you would get:

root@01425bf2de51:/# ls /share
ls: cannot open directory share/: Permission denied
root@01425bf2de51:/#

Other possibilities:

  • Disable SELinux (su -c "setenforce 0"): Bad idea!

  • Run Docker in privileged mode (docker run --privileged=true): Also not a good idea.

    The -privileged flag gives all capabilities to the container, and it also lifts all the limitations enforced by the device cgroup controller. In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker.

https://stackoverflow.com/questions/24288616/permission-denied-on-accessing-host-directory-in-docker

It’s also a good idea to have a more in-depth look at data volumes: https://docs.docker.com/userguide/dockervolumes/

TODO: What happens with hardlinks? Is it possible to create a hardlink to a file outside of the shared folder?

USB Support

Docker 1.2 has introduced the new --device option which makes it really easy to use USB devices of the host inside docker containers without the need for the really unsafe --privileged option. For more information have a look at the release announcement: http://blog.docker.com/2014/08/announcing-docker-1-2-0/.

Example:

[chris@thinkpad ~]$ docker run -it --device=/dev/ttyUSB0:/dev/ttyUSB0 jann/ubuntu:14.04 /bin/bash
root@31e6ea1db83b:/# ls -l /dev/tty*
crw-rw-rw-. 1 root root   5, 0 Sep  7 14:52 /dev/tty
crw-rw----. 1 root root 188, 0 Sep  7 14:52 /dev/ttyUSB0
root@31e6ea1db83b:/# apt-get update
root@31e6ea1db83b:/# apt-get install screen
root@31e6ea1db83b:/# screen /dev/ttyUSB0 115200
[screen is terminating]
root@31e6ea1db83b:/#

I can access my USB serial converter so USB support seems to be fine.

Previously, it was necessary to bind mount devices with -v inside a --privileged container:

If you have some application that uses libusb to enumerate the USB devices by ID, then you have to add /dev/bus/usb/<bus>/<device> too. First get the bus and device number by using lsusb:

[chris@thinkpad ~]$ lsusb
Bus 002 Device 015: ID 0403:6010 Future Technology Devices International, Ltd FT2232C Dual USB-UART/FIFO IC
[chris@thinkpad ~]$ ls -l /dev/bus/usb/002/015
crw-rw-r--. 1 root plugdev 189, 142 Sep 24 12:29 /dev/bus/usb/002/015
[chris@thinkpad ~]$

Then you can add it to the container by using --device=/dev/bus/usb/002/015 and if you are using the dockapp script from below:

[chris@thinkpad ~]$ dockapp run -u -r 1280x1000x24 -d /dev/ttyUSB0 -d /dev/ttyUSB1 -d /dev/bus/usb/002/015

Running GUI Applications

There are many approaches to run run GUI applications inside containers:

Most are either tunneling X11 over SSH which weakens security (as far as I understand):

Any application that has access to the X server can do a lot of things. It can snoop on other applications that display windows on the same server. It can log key presses. It can rebind keys. It can inject key presses into other applications. It has access to the clipboard. –https://security.stackexchange.com/questions/6209/running-proprietary-software-on-linux-safely

or using VNC. I’ve decided to use VNC for better security.

Inside the docker container:

root@31e6ea1db83b:/# apt-get install -y xvfb x11vnc openbox firefox vim
root@31e6ea1db83b:/# mkdir /.vnc
root@31e6ea1db83b:/# x11vnc -storepasswd mypassword ~/.vnc/passwd
root@31e6ea1db83b:/# Xvfb :1 -extension GLX -screen 0 1024x780x24 &
root@31e6ea1db83b:/# DISPLAY=:1 /usr/bin/openbox-session &
root@31e6ea1db83b:/# x11vnc -usepw -forever -display :1 &

It’s probably a good idea to put these command into a little bash script for later use:

root@96b71e543910:/# vim dockapp-start
root@96b71e543910:/# cat dockapp-start
#!/bin/bash

# Initialize variables
resolution="1024x780x24"
additional_cmd="ls -l"
x11vnc_args=""
unixsockonly=0

while getopts "h?r:a:x:fu" opt; do
    printf " -%s '%s'\n" $OPTION $OPTARG
    case "$opt" in
    h|\?)
        help
        exit 0
        ;;
    r)  resolution=$OPTARG
        ;;
    a)  additional_cmd=$OPTARG
        ;;
    f)  x11vnc_args+="-forever "
        ;;
    u)  unixsockonly=1
        ;;
    esac
done

shift $((OPTIND-1))

help() {
  echo "./dockapp-start [arg...]"
}

echo "resolution='$resolution'"
echo "cmd='$cmd'"
echo "x11vnc_args='$x11vnc_args'"
echo "leftovers: $@"

# start framebuffer display server
Xvfb :1 -extension GLX -screen 0 $resolution &

# export DISPLAY or you have to use "DISPLAY=:1 /usr/bin/cmd"
export DISPLAY=:1

/usr/bin/openbox-session &

# Wait a bit until the X server is ready
sleep 1

# Launch an additional command in the background
$additional_cmd &

# Launch an xterm
xterm &

# Without using the -forever flag the container will stop
# as soon as vncviewer disconnects
#x11vnc -usepw -display :1 -forever

if [ $unixsockonly -eq 0 ]; then
    x11vnc -usepw -display :1 $x11vnc_args
else
    # switch to user app so that the socket gets created
    # with the correct user:group on the host
    su app -c"/x11vnc-0.9.13/x11vnc/x11vnc -unixsockonly /share/x11vnc-socket -display :1 $x11vnc_args"
fi

# To make sure we can restart this container properly
# Fatal server error: Server is already active for display 1
# If this server is no longer running, remove /tmp/.X1-lock
rm /tmp/.X1-lock
root@96b71e543910:/# chmod +x dockapp-start
root@96b71e543910:/#

Instead of using dockapp-start, you may want to have a look at supervisord:

On the host:

[chris@thinkpad ~]$ sudo yum install x11vnc vnc # Ubuntu: sudo apt-get install x11vnc xvnc4viewer
[chris@thinkpad ~]$ mkdir -p ~/.docker/vnc
[chris@thinkpad ~]$ x11vnc -storepasswd mypassword ~/.docker/vnc/passwd
stored passwd in file: /home/chris/.docker/vnc/passwd
[chris@thinkpad ~]$

Run docker ps to get the name of the container where x11vnc is running:

[chris@thinkpad ~]$ docker ps
CONTAINER ID        IMAGE                  COMMAND             CREATED
31e6ea1db83b        jann/ubuntu:14.04     "/bin/bash"         2 hours ago

STATUS              PORTS               NAMES
Up 2 hours                              stupefied_perlman
[chris@thinkpad ~]$

Then run docker inspect container_name | grep IPAddress to get the IP of the container:

[chris@thinkpad ~]$ docker inspect stupefied_perlman | grep IPAddress
      "IPAddress": "172.17.0.15",
[chris@thinkpad ~]$

Now you can connect the vncviewer:

[chris@thinkpad ~]$ vncviewer -PasswordFile=$HOME/.docker/vnc/passwd 172.17.0.15:5900
Press F8 inside the VNC window to get a configuration menu for vncviewer.

Press F8 inside the VNC window to get a configuration menu for vncviewer.

Right click on the desktop in Openbox for a menu.

Right click on the desktop in Openbox for a menu.

Firefox running inside a Docker container under Openbox.

Firefox running inside a Docker container under Openbox.

Openbox: middle mouse button to view minimized applications.

Openbox: middle mouse button to view minimized applications.

Remarks:

  • Openbox is quite cool.

  • Is it a good idea to run Openbox as root inside Docker?

  • It is also possible to view just a single window with x11vnc:

    Basic steps:

    Run xwininfo from a console. It will change your cursor. Click on the window you want to share. xwininfo will print out the window id. Run x11vnc -id “id from xwininfo” replacing “id from xwininfo” with the appropriate id.

    [chris@thinkpad ~]$ xwininfo -display :0 -name "Firefox" | grep "Window id" | cut -d' ' -f4
    0x3c00001
    [chris@thinkpad ~]$ x11vnc -id 0x3c00001
    [chris@thinkpad ~]$ vncviewer localhost:5900
    

    You only need to create a script to start just a singe application inside a Docker container and then use the vncviewer to display its window contents.

Commiting Changes

After making a lot of changes to our container its a good idea to commit our changes and create a new image.

For later use create a new unprivileged user app and then exit the container:

root@96b71e543910:/# useradd app
root@96b71e543910:/# ln -s /share/home/app /home/app
root@96b71e543910:/# passwd app
Enter new UNIX password: app
Retype new UNIX password: app
passwd: password updated successfully
root@96b71e543910:/# id
uid=0(root) gid=0(root) groups=0(root)
root@96b71e543910:/# id app
uid=1000(app) gid=1000(app) groups=1000(app)
root@96b71e543910:/# exit
exit
[chris@thinkpad ~]$ mkdir -p $HOME/share/docker/home/app/
[chris@thinkpad ~]$ id
uid=1000(chris) gid=1000(chris) groups=1000(chris)
[chris@thinkpad ~]$

Then we can commit a copy of this container (with the ID 96b71e543910) to an image using the docker commit command:

[chris@thinkpad ~]$ docker commit -m="VNC, Openbox, Firefox" -a="chris" 96b71e543910 jann/ubuntu:14.04v1
f06178ddf53b32302b08de13f0693b8279ddaba589538724ba0fd06c2c0edf12
[chris@thinkpad ~]$

I used the commit message VNC, Openbox, Firefox, specified the author with -a, created a new repository jann/ubuntu and set the tag of the image to 14.04v1.

Later we will use this image as a basis to install our proprietary software.

With the command docker images we can get a list of all our images:

[chris@thinkpad ~]$ docker images
REPOSITORY      TAG       IMAGE ID       CREATED         VIRTUAL SIZE
jann/ubuntu    14.04v1   f06178ddf53b   7 minutes ago   657.5 MB
jann/ubuntu    14.04     775566c59cb9   3 days ago      213 MB
ubuntu          14.04     c4ff7513909d   3 weeks ago     213 MB
fedora          20        88b42ffd1f7c   7 weeks ago     373.7 MB
fedora          latest    88b42ffd1f7c   7 weeks ago     373.7 MB
[chris@thinkpad ~]$

Remarks:

Restrict Network Access

Now since we have installed all basic software we can look down the container a bit more.

To run the container completely without network access:

[chris@thinkpad ~]$ sudo docker run -i -t --net=none jann/ubuntu:14.04v1 /bin/bash

But this does not work since we need to access the VNC server to view our application.

So we have to find a way to block all network traffic except VNC (only allow incoming connections from localhost to port 5900).

Docker networking:

This solution is by far not perfect (because it will affect all containers and the host too, no Network in VirtualBox), but setting ip_forward to 0 seems to work:

[chris@thinkpad ~]$ sudo sysctl -w net.ipv4.ip_forward=0
[chris@thinkpad ~]$ #sudo su -c "echo 0 > /proc/sys/net/ipv4/ip_forward"

It is also possible to do it for a specific interface:

[chris@thinkpad ~]$ sudo su -c "echo 0 > /proc/sys/net/ipv4/conf/veth80d5/forwarding"

But there is no easy way to get the interface name or to do it before the container starts, except creating the network interface on your own with previously selecting --net=none.

It’s far easier to do it for all docker containers by disabling ip_forward for the docker bridge:

[chris@thinkpad ~]$ sudo su -c "echo 0 > /proc/sys/net/ipv4/conf/docker0/forwarding"

In the logs you may notice the messages docker0: port 1(veth97c3) entered forwarding state but this affects only one container and IP forwarding should still be disabled for the Docker bridge docker0. A ping from inside the container is not possible.

[chris@thinkpad ~]$ cat /var/log/messages |grep forwarding |tail -n1
Sep  9 21:27:57 thinkpad kernel: docker0: port 1(veth97c3) entered forwarding state
[chris@thinkpad ~]$

Note

It seems that VNC needs considerable longer to connect if ip_forward is disabled but connects nevertheless, I don’t know why...

I’m looking for a better solution, maybe something with iptables.

Using Unix Sockets for VNC

It seems the best solution is to disable networking (--net=none) for a container that runs untrusted software alltogether and to use a unix socket for VNC.

Have a look at the excerpt from man x11vnc:

-unixsock str
~~~~~~~~~~~~~

In addition to the regular TCP port, listen on the unix socket (AF_UNIX) 'str' for incoming connections.  This mode is for either local connections or a tunnel endpoint where one wants the file
permission  of the unix socket file to determine what can connect to it.  Example: mkdir ~/s; chmod 700 ~/s; x11vnc -unixsock ~/s/mysock -rfbport 0 ...  same as: x11vnc -unixsockonly ~/s/mysock
...  (see -unixsockonly below.)

This mode currently requires the modified libvncserver bundled in the the x11vnc 0.9.13 tarball and later.

Note that the SSVNC unix vncviewer can connect to unix sockets, for example: ssvnc -viewer unix=./s/mysock

As a special mechanism, if 'str' for either -unixsock or -unixsockonly is of the form "fd=n" where n is a non-negative decimal integer, then  instead  of  creating  a  unix  socket,  that  file
descriptor  (assumed  already opened and O_RDWR) will be attached as a VNC client.  Perhaps the program that execs x11vnc has created a socketpair(2) to communicate over.  Use this mechanism if
-inetd (which is basically fd=0) is not flexible enough for you.

-unixsockonly str
~~~~~~~~~~~~~~~~~

Listen on unix socket 'str' only, no TCP ports. First note that one can disable all tcp listening ports by specifying '-rfbport 0'. The option '-unixsockonly str' is functionally equivalent  to
'-unixsock str -rfbport 0'

So wee need ssvnc (the normal vncviewer does not support unix sockets) and because there exists no Fedora package (old RPM spec) I have to build it from source.

Building ssvnc from source:

[chris@thinkpad Downloads]$ sudo yum install xorg-x11-server-devel libXaw-devel libXmu-devel openssl-devel libjpeg-devel zlib-devel imake
[chris@thinkpad Downloads]$ curl -L http://downloads.sourceforge.net/ssvnc/ssvnc-1.0.29.src.tar.gz -o ssvnc-1.0.29.src.tar.gz
[chris@thinkpad Downloads]$ tar -xf ssvnc-1.0.29.src.tar.gz
[chris@thinkpad Downloads]$ cd ssvnc-1.0.29/
[chris@thinkpad ssvnc-1.0.29]$ make config
[chris@thinkpad ssvnc-1.0.29]$ make all
[chris@thinkpad ssvnc-1.0.29]$ ./vnc_unixsrc/vncviewer/vncviewer --help |gerp unix
bash: gerp: command not found
[chris@thinkpad ssvnc-1.0.29]$ ./vnc_unixsrc/vncviewer/vncviewer --help |grep unix
       vncviewer [<OPTIONS>] /path/to/unix/socket
       vncviewer [<OPTIONS>] unix=/path/to/unix/socket
         it is interpreted as a unix-domain socket (AF_LOCAL/AF_UNIX
         instead of AF_INET)  Prefix with unix= to force interpretation
         as a unix-domain socket.
        -unixpw str Useful for logging into x11vnc in -unixpw mode. "str" is a
                    used for the -unixpw login.  Other VNC servers could do
                    you are using a unix program (e.g. our ultravnc_dsm_helper)
[chris@thinkpad ssvnc-1.0.29]$ #sudo make install
[chris@thinkpad ssvnc-1.0.29]$

Instead you could also build the ssvnc-viewer RPM:

[chris@thinkpad ~]$ sudo yum install @development-tools
[chris@thinkpad ~]$ sudo yum install fedora-packager
[chris@thinkpad ~]$ mkdir -p ~/rpmbuild/SOURCES
[chris@thinkpad ~]$ mkdir -p ~/rpmbuild/SPECS
[chris@thinkpad ~]$ cd ~/rpmbuild
[chris@thinkpad rpmbuild]$ curl https://endur.fedorapeople.org/downloads/specfiles/ssvnc-viewer.spec -o SPECS/ssvnc-viewer.spec
[chris@thinkpad rpmbuild]$ curl -L http://downloads.sourceforge.net/ssvnc/ssvnc-1.0.29.src.tar.gz -o SOURCES/ssvnc-1.0.29.src.tar.gz
[chris@thinkpad rpmbuild]$ #sudo yum-builddep SPECS/ssvnc-viewer.spec
[chris@thinkpad rpmbuild]$ rpmbuild -ba SPECS/ssvnc-viewer.spec
error: Failed build dependencies:
        xorg-x11-server-devel is needed by ssvnc-viewer-1.0.29-3.fc20.x86_64
        libXaw-devel is needed by ssvnc-viewer-1.0.29-3.fc20.x86_64
        libXmu-devel is needed by ssvnc-viewer-1.0.29-3.fc20.x86_64
        libjpeg-devel is needed by ssvnc-viewer-1.0.29-3.fc20.x86_64
        imake is needed by ssvnc-viewer-1.0.29-3.fc20.x86_64
[chris@thinkpad rpmbuild]$ sudo yum install xorg-x11-server-devel libXaw-devel libXmu-devel openssl-devel libjpeg-devel zlib-devel imake
[chris@thinkpad rpmbuild]$ rpmbuild -ba SPECS/ssvnc-viewer.spec
[chris@thinkpad rpmbuild]$ sudo yum install RPMS/x86_64/ssvnc-viewer-1.0.29-3.fc20.x86_64.rpm
[chris@thinkpad rpmbuild]$ rpm -ql ssvnc-viewer
/usr/bin/ssvnc-viewer
/usr/share/applications/%{vendor}-ssvnc-viewer.desktop
/usr/share/doc/ssvnc-viewer
/usr/share/doc/ssvnc-viewer/COPYING
/usr/share/doc/ssvnc-viewer/README
/usr/share/man/man1/ssvnc.1.gz
/usr/share/man/man1/ssvncviewer.1.gz
/usr/share/man/man1/stunnel.1.gz
[chris@thinkpad rpmbuild]$

So ssvnc-viewer is installed:

[chris@thinkpad ~]$ ssvnc-viewer --help |grep unix/socket
       ssvnc-viewer [<OPTIONS>] /path/to/unix/socket
       ssvnc-viewer [<OPTIONS>] unix=/path/to/unix/socket
[chris@thinkpad ~]$

Let’s try it:

[chris@thinkpad ~]$ docker run -it --name network_test -v /home/$USER/share/docker/:/share:rw jann/ubuntu:14.04v1 /bin/bash
root@ea4726494bb8:/# Xvfb :1 -extension GLX -screen 0 1024x780x24 &
root@ea4726494bb8:/# DISPLAY=:1 /usr/bin/openbox-session &
root@ea4726494bb8:/# x11vnc -usepw -unixsockonly /share/x11vnc-socket -display :1
...
10/09/2014 12:55:30 Not listening on IPv6 interface.
10/09/2014 12:55:30 no support for rfbListenOnUnixSocket()
root@ea4726494bb8:/#

Why no support for rfbListenOnUnixSocket(), it’s enough that there exists a Debian bug with working patches for vncviewer that adds support for unix sockets for at least 3 years but what is wrong with x11vnc?

#ifndef LIBVNCSERVER_ALLOW_NON_AF_INET_SOCKETS
  rfbLog("no support for rfbListenOnUnixSocket()\n");
  clean_up_exit(1);
#else
  int fd = rfbListenOnUnixSocket(unix_sock, screen);
  if (fd < 0) {
  rfbLog("rfbListenOnUnixSocket() failed to create unix socket: %s\n",
  unix_sock);
  clean_up_exit(1);
  }
#endif

Using -unixsock instead of -unixsockonly got me a bit further but failed later nevertheless:

root@ea4726494bb8:/# x11vnc -unixsock /share/x11vnc-socket -rfbport 0 -display :1
10/09/2014 13:33:35 passing arg to libvncserver: -rfbport
10/09/2014 13:33:35 passing arg to libvncserver: 0
...
10/09/2014 13:33:36 Not listening on IPv6 interface.
10/09/2014 13:33:36 listen_unix: AF_UNIX socket /share/x11vnc-socket
10/09/2014 13:33:36 listen_unix: warning: libvnserver does not support AF_UNIX sockets.
10/09/2014 13:33:36 linux_unix: listening on unix socket: /share/x11vnc-socket fd=9
10/09/2014 13:33:36 fb read rate: 819 MB/sec
10/09/2014 13:33:36 fast read: reset -wait  ms to: 10
...
10/09/2014 13:35:32 accept_unix: warning: libvnserver does not support AF_UNIX sockets.
10/09/2014 13:35:32   other clients:
10/09/2014 13:35:32 setsockopt failed: Operation not supported
10/09/2014 13:37:49 accept_unix: warning: libvnserver does not support AF_UNIX sockets.
10/09/2014 13:37:49   other clients:
10/09/2014 13:37:49 setsockopt failed: Operation not supported
^Ccaught signal: 2
10/09/2014 13:48:54 deleted 32 tile_row polling images.
10/09/2014 13:48:54 deleting unix sock: /share/x11vnc-socket
root@ea4726494bb8:/#

Sometimes I get really angry about these Debian maintainers, either they are patching their packages to death (away from upstream or what other distros do without upstreaming their changes which sooner or later breaks something), or they are building hundreds of small packages from a single tarball that gets shipped as a single package elsewhere and now they are disabling useful features too :(

So just build it from source:

root@ea4726494bb8:/# x11vnc --version
x11vnc: 0.9.13 lastmod: 2011-08-10
root@ea4726494bb8:/# curl -L http://sourceforge.net/projects/libvncserver/files/x11vnc/0.9.13/x11vnc-0.9.13.tar.gz/download -o x11vnc-0.9.13.tar.gz
root@ea4726494bb8:/# tar -xf x11vnc-0.9.13.tar.gz
root@ea4726494bb8:/# cd x11vnc-0.9.13
root@ea4726494bb8:/x11vnc-0.9.13# apt-get build-dep x11vnc
root@ea4726494bb8:/x11vnc-0.9.13# ./configure
root@ea4726494bb8:/x11vnc-0.9.13# make
root@ea4726494bb8:/x11vnc-0.9.13# #make install

Note

You may need to enable IP forwarding again to be able to install packages:

[chris@thinkpad ~]$ sudo su -c "echo 1 > /proc/sys/net/ipv4/conf/docker0/forwarding"

And it works!

root@ea4726494bb8:/x11vnc-0.9.13# su app
app@ea4726494bb8:/x11vnc-0.9.13$ ./x11vnc/x11vnc -unixsockonly /share/x11vnc-socket -display :1 &
[chris@thinkpad ~]$ ssvnc-viewer ~/share/docker/x11vnc-socket

Proto: RFB 003.008

Connected to RFB server, using protocol version 3.8

SelectSecurityType:
  sec-type[0]    1  (rfbSecTypeNone)
Security-Type:   1  (rfbSecTypeNone)  Latency: 0.21 ms
No VNC authentication needed
VNC authentication succeeded (0) for rfbSecTypeNone (RFB 3.8)

Desktop name "ea4726494bb8:1"

VNC server default format:
  32 bits per pixel.  Least significant byte first in each pixel.
  True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
Warning: Cannot convert string "-*-helvetica-medium-r-*-*-12-*-*-*-*-*-*-*" to type FontStruct
Using default colormap which is TrueColor.  Pixel format:
  32 bits per pixel.  Least significant byte first in each pixel.
  True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
geometry: 1024x780+446+138 ycrop: 0
create_image()
try_create_image: shm image create fail: image == NULL
try_create_image: created *non-shm* image: 1024x780
try_create_image: image->bytes_per_line: 4096

guessed: -compresslevel 1
guessed: -qualitylevel  9
enabling 'delay_sync' mode for faster local drawing,
disable via env SSVNC_DELAY_SYNC=0 if there are painting errors.
Docker, x11vnc -unixsockonly, SSVNC

Docker, x11vnc -unixsockonly, SSVNC

Now exit and commit the changes:

root@ea4726494bb8:/# rm /tmp/.X1-lock
root@ea4726494bb8:/# exit
exit
[chris@thinkpad ~]$ docker commit -m="x11vnc -unixsockonly" -a="chris" ea4726494bb8 jann/ubuntu:14.04v2
567bff8880d6420a0ec44c4e82db1ba2746b2a3329f7eb5fa686effbbdfb4bc7
[chris@thinkpad ~]$

Remove old Containers and Images

Deleting not longer needed images is quite easy, just do a docker images to list all images and then run docker rmi <IMAGE ID>.

Whereas deleting not longer needed containers is a bit more work. Every time we start an image Docker will create a new container, after some days I have already a quite long list of old containers:

[chris@thinkpad ~]$ docker ps -a
CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS                         PORTS               NAMES
9b20c8f49ea0        jann/ubuntu:14.04v3   "/dockapp-start"         15 minutes ago      Exited (-1) 11 minutes ago                         dockapp
96b71e543910        jann/ubuntu:14.04v2   "/bin/bash"         5 hours ago         Exited (0) About an hour ago                       desperate_carson
31e6ea1db83b        jann/ubuntu:14.04     "/bin/bash"         6 hours ago         Up 6 hours                                         stupefied_perlman
ae6a18e47f64        jann/ubuntu:14.04     "/bin/bash"         7 hours ago         Exited (0) 7 hours ago                             prickly_bohr
c7908c149d89        jann/ubuntu:14.04     "/bin/bash"         7 hours ago         Exited (0) 7 hours ago                             nostalgic_archimedes
c688e3146470        ubuntu:14.04           "/bin/bash"         8 hours ago         Exited (1) 7 hours ago                             drunk_torvalds
65a77c4901b3        ubuntu:14.04           "/bin/bash"         8 hours ago         Exited (0) 7 hours ago                             stupefied_mestorf
[chris@thinkpad ~]$

You can delete single containers by specifying its ID:

[chris@thinkpad ~]$ docker rm cd17b9c6c511

As seen here delete all containers created weeks ago (not yet tested):

[chris@thinkpad ~]$ docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs docker rm

Delete all containers:

[chris@thinkpad ~]$ docker rm $(docker ps -aq)
Error response from daemon: You cannot remove a running container. Stop the container before attempting removal or use -f
980003f922be
90850bd1ccf2
50b1c16b7057
3f1968acb2b7
...
3dde4dd374b7
d98181ecfbb3
2014/09/07 14:57:49 Error: failed to remove one or more containers
[chris@thinkpad ~]$ docker ps -a
CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS              PORTS               NAMES
5505bf8349a0        jann/ubuntu:14.04v1   "/bin/bash"         43 minutes ago      Up 43 minutes                           distracted_euclid
[chris@thinkpad ~]$

All containers and images are stored somewhere under /var/lib/docker/: http://blog.thoward37.me/articles/where-are-docker-images-stored/

Install Proprietary Software

Since we have locked down our container as mush as possible we can finally install the proprietary crap.

We will use our newly created image jann/ubuntu:14.01v2 as a base for all our future work:

[chris@thinkpad ~]$ #sudo su -c "echo 0 > /proc/sys/net/ipv4/conf/docker0/forwarding"
[chris@thinkpad ~]$ docker run -it --net=none -v /home/chris/share/docker/:/share:rw jann/ubuntu:14.04v2 /bin/bash

See here for the full post: Installing Xilinx ISE inside a Docker Container.

Launching Applications more easily

We can use our newly crated Docker image jann/ubuntu:14.04v2 as a basis to start a new container with all our changes from above:

[chris@thinkpad ~]$ docker run -it jann/ubuntu:14.04v2 /bin/bash
root@e44b476c2911:/# ls
Desktop  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  share  srv  dockapp-start  sys  test.txt  tmp  usr  var
root@e44b476c2911:/# ./dockapp-start
...
root@e44b476c2911:/# exit

Or just run our container with the dockapp-start script in the background and connect the vncviewer:

[chris@thinkpad ~]$ CID=$(docker run -d jann/ubuntu:14.04v2 /dockapp-start)
[chris@thinkpad ~]$ vncviewer -PasswordFile=$HOME/.docker/vnc/passwd $(docker inspect --format '{{ .NetworkSettings.IPAddress }}' ${CID}):5900

To run our software we can either start every time a new container from an image that contains all the needed software and delete the container afterwards:

  • CID=$(docker run..., docker stop ${CID}, docker rm ${CID}
  • Application data needs to be saved to a shared folder

Or stop the container if we no longer need it and start it later again

  • CID=$(docker run..., docker stop ${CID}, docker start ${CID}
  • It’s a good idea to save the application data to a shared folder nevertheless
  • Need to mange the containers somehow and keep track of the CIDs, using the --name option is a good idea

The following script can be used to start/stop your application:

[chris@thinkpad ~]$ vim bin/dockapp
[chris@thinkpad ~]$ chmod +x bin/dockapp
[chris@thinkpad ~]$ cat bin/dockapp
#!/bin/bash

# Initialize variables
name="dockapp"
image="jann/ubuntu:14.04v2"
vnc_passwd_file="$HOME/.docker/vnc/passwd"
share="$HOME/share/docker"
network=1
socket="$HOME/share/docker/x11vnc-socket"
hostname="docker"

start_args=""
docker_args=""

run() {
  echo "dockapp: creating the container and starting it"
  shift $((OPTIND - 1))
  while getopts "r:a:x:fu" OPTION "$@"; do
    printf " -%s '%s'\n" $OPTION $OPTARG
    case "$OPTION" in
    r)  start_args+="-$OPTION $OPTARG "
        ;;
    a)  start_args+="-$OPTION $OPTARG "
        ;;
    f)  start_args+="-f "
        ;;
    u)  start_args+="-u "
        docker_args+="--net=none "
        ;;
    esac

  done
  shift $((OPTIND - 1))
  #printf "Leftover args: '%s'" "$@"

  # delete old sockets
  if [ -S "$socket" ]; then
    rm "$socket"
  fi

  docker run -d --name $name -h $hostname -v $share:/share:rw $docker_args $image /dockapp-start $start_args
}

view() {
  echo "dockapp: running vncviewer"
  if [ -S "$socket" ] #see man test
  then
        ssvnc-viewer "$socket"
  else
        vncviewer -PasswordFile=$vnc_passwd_file $(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $name):5900
  fi

}

start() {
  echo "dockapp: starting a stopped container"
  docker start $name
}

stop() {
  echo "dockapp: stoping a container"
  docker stop $name
}

pause() {
  echo "dockapp: pause a container"
  #docker pause $name
}

unpause() {
  echo "dockapp: unpause a paused container"
  #docker unpause $name
}

ps() {
  echo "dockapp: active dockapp containers:"
  docker ps -a |grep $name
}

default() {
  if ! docker inspect $name &> /dev/null; then
    echo "dockapp: container $name not running, starting it"
    dockapp run -u -r 1280x1000x24
  else
    start
  fi

  sleep 3
  view
}

help() {

echo "
SYNOPSIS
   dockapp [COMMAND] [arg...]
COMMANDS
    run
        -r,
        -a,

    view
    start
    stop
    pause
    unpause
    help
"

echo "dockapp [run|view|start|stop|pause|unpause|help] [arg...]"
echo "dockapp run -r 1280x1024x24 -a firefox -x top"
echo "dockapp run -u -r 1280x1000x24"
echo "dockapp run -u -r 1280x1000x24 -d /dev/ttyUSB0 -d /dev/ttyUSB1"
echo "dockapp run -u -r 1280x1000x24 -d /dev/ttyUSB0 -d /dev/ttyUSB1 -d /dev/bus/usb/002/015"
echo "dockapp"

}


if [ -z "$1" ]
then
  echo "No arguments given, using defaults."
  default
elif [[ $1 =~ ^(run|view|start|stop|pause|unpause|ps|help)$ ]]; then
  "$@"
else
  echo "Invalid argument $1" >&2
  help
  exit 1
fi

Of course dockapp can be further extended and maybe I will find a better solution but at the moment it works quite well:

[chris@thinkpad ~]$ dockapp run
dockapp: creating the container and starting it
26707bcf8d587f65009ad7bb38594eb9696f6ee6c443b2e47e5601c78d8a0cf6
[chris@thinkpad ~]$ dockapp view
dockapp: running vncviewer

TigerVNC Viewer 64-bit v1.3.0 (20140319)
Built on Mar 19 2014 at 17:09:18
Copyright (C) 1999-2011 TigerVNC Team and many others (see README.txt)
See http://www.tigervnc.org for information on TigerVNC.

Thu Sep 11 14:29:52 2014
 CConn:       connected to host 172.17.0.10 port 5900
 CConnection: Server supports RFB protocol version 3.8
 CConnection: Using RFB protocol version 3.8
 PlatformPixelBuffer: Using default colormap and visual, TrueColor, depth 24.
 CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
 CConn:       Using Tight encoding

Thu Sep 11 14:30:52 2014
 Viewport:    Unexpected release of FLTK key code 65481 (0xffc9)

Thu Sep 11 14:30:59 2014
 Viewport:    Unexpected release of FLTK key code 65481 (0xffc9)

Thu Sep 11 14:31:07 2014
 Viewport:    Unexpected release of FLTK key code 65377 (0xff61)
[chris@thinkpad ~]$
Docker, TigerVNC, with internet access

Docker, TigerVNC, with internet access

Once the container is created its not possible to change its runtime arguments, so if we want to start dockapp with other arguments dockapp run -u -r 1280x1000x24 which gets also executed if no arguments are supplied at all, then we have to remove the old dockapp container first:

[chris@thinkpad ~]$ docker rm dockapp
dockapp
[chris@thinkpad ~]$ dockapp
No arguments given, using defaults.
dockapp: container dockapp not running, starting it
dockapp: creating the container and starting it
 -u ''
 -r '1280x1000x24'
b289043b4de6d7bd8a2cfbbfba5ec2182d5e23c2ebda5ed1032bf0e0106e27c9
dockapp: running vncviewer

Proto: RFB 003.008

Connected to RFB server, using protocol version 3.8

SelectSecurityType:
  sec-type[0]    1  (rfbSecTypeNone)
Security-Type:   1  (rfbSecTypeNone)  Latency: 0.26 ms
No VNC authentication needed
VNC authentication succeeded (0) for rfbSecTypeNone (RFB 3.8)

Desktop name "docker:1"

VNC server default format:
  32 bits per pixel.  Least significant byte first in each pixel.
  True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
Warning: Cannot convert string "-*-helvetica-medium-r-*-*-12-*-*-*-*-*-*-*" to type FontStruct
Using default colormap which is TrueColor.  Pixel format:
  32 bits per pixel.  Least significant byte first in each pixel.
  True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
geometry: 1280x1000+318+28 ycrop: 0
create_image()
try_create_image: shm image create fail: image == NULL
try_create_image: created *non-shm* image: 1280x1000
try_create_image: image->bytes_per_line: 5120

guessed: -compresslevel 1
guessed: -qualitylevel  9
enabling 'delay_sync' mode for faster local drawing,
disable via env SSVNC_DELAY_SYNC=0 if there are painting errors.

2014/09/11 14:41:46 VNC Viewer exiting.

[chris@thinkpad ~]$
Docker, SSVNC, no network connection

Docker, SSVNC, no network connection

If something isn’t working as expected then have a look at the output of docker logs dockapp.

Using a Dockerfile

All the steps above that were necessary to create our base image can be automated using a Dockerfile, then you just have to run docker build -t chris/dockapp.

But doing it manually first is really a good idea to understand how everything works so that you can fix possible issues more easily.

To be continued...

You can find the complete code at https://github.com/christianjann/dockapp.

Conclusion

It works, it’s running at native speed, no virtual machine is needed, starting a container needs just a second, maybe it’s not as secure as using a virtual machine but mush more handy and less resource hungry and if the software really tries to break out of the container then it should get noticed and I don’t know how a company would explain such behavior to its customers.

It might be handy as well if I just need a Debian system to compile and test some applications. Dockers image management is really easy and fast.

I can’t explain it exactly but somehow I don’t trust proprietary software. But now that all the proprietary software is running in a sandbox my system feels clean and secure again ;)