Korn Shell Launcher for Windows Subsystem for Linux

According to Microsoft, the Windows Subsystem for Linux (WSL) is a new Windows 10 feature that is essentially a compatibility layer which enables you to run native Linux command-line tools and utilities directly on Windows 10, alongside your traditional Windows desktop applications. Currently a beta feature, it is scheduled to be officially released in the Windows 10 Anniversary Update which is due CY2016Q3. There is no support for Linux GUI-based or TUI-based tools, utilities or applications not is any planned.

When WSL is enabled, a Canonical-created Ubuntu user-mode image (currently named trusty-server-cloudimg-amd64-root.tar.gz and based on Ubuntu 14.04 Long-Term Support) is downloaded, unpacked and installed into %LOCALAPPDATA%\lxss\rootfs which is a hidden system folder. Aliases for the Linux home directories at /home and /root are created in the non-hidden parent folder at %LOCALAPPDATA%\lxss to facilitate easy file sharing between Linux and Windows.

A crucial difference between a standard GNU/Linux multiuser platform and WSL is that WSL is installed on a per-user basis; it is not a shared user subsystem. See further down this post for screenshots of all the relevant folders for my WSL installation.

The Windows bash.exe application, when started, then loads the user-mode image and invokes the bash shell supplied with the image. The bash shell runs within a Windows console which emulates the behavior of an xterm console.

Here is an diagram of the WSL components:

Read this MSDN blog post for more information about the WSL architecture.

By the way, picoprocesses and a Linux subsystem are not new Microsoft Windows features. Support for picoprocesses was first added in Microsoft Windows 8 and fully fleshed out in Microsoft Windows 8.1. The recently shuttered Project Astoria (Windows subsystem for Android) also included a Linux subsystem because Android uses a Linux kernel so a lot of that work should have been useful to the WSL development team.

Currently only a bash shell launcher is provided by Microsoft. Personally I prefer ksh93 (Korn Shell 1993) as my default shell, and in this post I will show you how to create a ksh93 launcher as shown below:

Naively, I initially believed that if I used apt-get to install the ksh package and modified my entry in /etc/passwd, my login shell would be changed to ksh93 the next I invoked WSL (after all, this is what happens in the GNU/Linux world). Not so in WSL! Installing the ksh using apt-get install worked as expected once I resolved the well known WSL bug with /etc/resolv.conf. Editing /etc/passwd as root using vipw also worked as expected. However,upon invoking the WSL, my shell was still the bash shell.

After thinking about it for a few minutes and looking around the subsystem (see above), the answer came to me. There is no login process when you invoke WSL and therefore the user environment which is normally set up by the login process and /etc/passwd is being set up by some other mechanism. That mechanism turned out to be the Microsoft-developed %SYSTEMROOT%\System32\bash.exe.

The low-hanging and easy solution would be to simply exec ksh93 from .bashrc by doing something like the following:

# switch to Korn shell
if [ -t 1 ]
then
    exec /usr/bin/ksh
fi

but I decided that this was no a good long term solution.

Using IDA Pro, I quickly determined the ASCII and Unicode strings in bash.exe, which is a 64-bit Visual C++ v10 or v11 application, as shown below:


Note the \\\\.\\pipe\\%s string. That is a clue that a Windows pipe is being used for communications between Windows and the subsystem. Note also the 4 hardcoded Linux environmental variables. We need to change the SHELL variable to /bin/ksh. Other interesting strings are the two lxrun.exe strings and the registry key Software\\Microsoft\\Windows\\CurrentVersion\\Lxss.

Looking at the Unicode (more precisely UTF-16) strings, you will see two instances of /bin/bash. Both of these strings need to be changed to /bin/ksh.

By the way, symbol information, i.e. bash.pdb, is available from the usual Microsoft Symbol Server source.

I have not researched the issue in any detail but LXSS appears to have logging facilities build it. Note the LxssVerboseLog and other LXSS-related strings.

Next make a copy of bash.exe named ksh.exe and edit it using your favorite binary editor. I happen to like WinHex. Replace the two UTF-16 bash strings with ksh and change the SHELL variable from to /bin/ksh as shown below:



I did not bother changing the bash strings in the StringFileInfo block at the end of the executable but I suppose one should do so for the sake of completeness.

Once you have made the three necessary modifications I described above, save the executable as ksh.exe and place it in the C:\Windows\System32 folder with the same permissions as bash.exe.

I decided to mimic the look and feel of the bash launcher, including the user prompt (PS1) and the xterm console title. This is very easy to do within the bash shell because of extensive builtin support for custom prompts, but quite difficult to do in the ksh93. Here is the ~/.kshrc script I developed:

# .kshrc for WSL

# set editor options
export VISUAL=vi
export EDITOR=${VISUAL}
set -o ${VISUAL}

# set history options
HISTSIZE=500
HISTEDIT=${EDITOR}
HISTFILE=.ksh_history
export HISTSIZE HISTFILE

# specify search path for autoloadable functions
FPATH=/usr/share/ksh/functions:~/.func

# avoid certain file types in completion
FIGNORE='@(*.o|~*)'

# set ksh prompt and xterm title to match what bash on WSL displays
_PSX='$( p="${PWD/~(El)${HOME}/\~}"
         printf "%s@%s:%s" "${LOGNAME}" "$(hostname -s)" "${p}"
)'

function PS1.get
{
   .sh.value="^[]0;${_PSX}^G${_PSX}$ "
}

# some useful aliases
alias ll='ls -l'
alias la='ls -al'
# alias h='fc -l | less'

cd ${HOME}

This script explicitly cds to your home directory whereas the WSL-supplied .bashrc does not do this. Instead, the Windows shortcut launches bash.exe with an argument of ~ to specify that the Bash shell should cd to your home directory. Note the use of a ksh93 discipline function to set the PS1 prompt and the xterm console title. This is the only way to get both the xterm console title and the PS1 prompt to update simultaneously and consistently.

To complete things, I made an Windows icon comprising of a gold shell on a red background for the Windows shortcut.

Here is what the folders of my WSL subsystem look like after the creation and installation of the Korn shell launcher:



Note the location of ksh.ico, the Korn Shell launcher shortcut icon and the presence of .kshrc in my home directory.

So far, two problems have surfaced when I use the Korn shell launcher.

The first problem is that pipes, e.g. ls -al | less, do not work. The work around is to invoke the Korn shell launcher with elevated privileges (Run as Administrator.) This is a due to a known sockets problem within WSL.

The second problem is more serious and there is no workaround for it at present. The “-c” command line option simply does not work.

This issue has plagued Cygwin for years with no resolution. With the layoff of David Korn and Glenn Fowler by AT&T in 2013 and their subsequent move to Google, the future of ksh93 is in serious doubt. What has surprised me is that nobody has actually stepped up to the plate to continue maintaining and enhancing ksh93.

The Korn shell launcher still launches you into a Ubuntu subsystem (namespace). It should be relatively easy to create alternatively ecosystems. One that I have come across recently is the Fedora 23 subsystem created by Seth Jennings. See his blog Obfuscation’s End for more information.

WSL is an exciting development for Microsoft Windows. It builds on work done years ago for the original POSIX subsystem for Windows NT, by long gone companies such as Interix, research work done in the early 2000s on picoprocesses and Embassies by Jon Howell et al at Microsoft Research and the recently shuttered Android subsystem.

Hopefully Microsoft will follow through with this work and not kill it off as they have done with numerous projects in this space.

P.S. You can download all the files for this Korn shell launcher from here.

[Updated 2016-06-16]