[Top] [Contents] [Index] [ ? ]

The GNU Hurd

1. History  Short history of the Hurd.
2. GNU/Linux is like Unix  
3. GNU is not Unix  
4. Breaking it up  
5. And putting it together again  
6. Accessing the servers  
7. Inviting the user  
8. Solving some problems in Unix  
9. Solving the problems in the Hurd  


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

1. History

1983
Richard Stallman founds the GNU project.
1988
Decision is made to use Mach 3.0 as the kernel.
1991
Mach 3.0 is released under compatible license.
1991
Thomas Bushnell, BSG, founds the Hurd project.
1994
The Hurd boots the first time.
1997
Version 0.2 of the Hurd is released.
1998
Debian hurd-i386 archive is created.
2002
Debian GNU/Hurd snapshot fills four CD images.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

2. GNU/Linux is like Unix

The GNU/Linux system consists of the GNU userland running on top of the Linux kernel. The GNU programs are written in a portable way. They are based on the POSIX.1 programming interface, which is implemented by the GNU C library. Some areas of the API are implemented by the C library exclusively, or only with little help from the kernel. In some other areas, the C library functions are just small wrappers around the kernel system calls. A system call is a trap into the kernel. The program continues execution in the kernel while doing some privileged operation. After that operation is finished, the program continues to run at the location right after the system call.

Linux contains around 200 system calls, which implement several huge and important parts of the POSIX API almost directly:

Linux also contains some other, non-standard interfaces, like clone (used to implement thread libraries) or the modules interface to load and unload kernel modules at run time.

All these features are implemented by one large program, the kernel. This program runs in a privileged mode of the processor, and has full control over the hardware. A bug in the network stack can potentially result in random changes to the filesystem. Any random bug can crash the whole system or caus other serious malfunction. More code in the kernel results also in more bugs.

Still, a lot of code is included in he kernel, and this is often done for maximum efficiency. One prominent example is the webserver in Linux, motivated by a proprietary solution beating the GNU/Linux + Apache combo performance wise. Another example is the network stack itself, which in the early days of Unix was in userland, until some people optimized it by moving it into the Unix kernel.

However, putting a lot of functionality into the kernel has several disadvantages:


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3. GNU is not Unix

The GNU/Hurd system replaces the monolithic kernel by a more flexible design that solves all these problems and brings further advantages.

At the core of the GNU system is a microkernel. A microkernel runs in the same privileged mode of the process as the Unix like kernels, but it does not implement most of its features. The idea is that the microkernel provides only primitive operations that are sufficient to implement the operating system on top of it. The Hurd currently runs on GNU Mach, which provides the following concepts:

Mach does not contain any of the following:

Mach is actually rather big for a microkernel, and quite dated. It has only a few system calls, but it uses its own IPC mechanism, and the full kernel API contains over 150 functions. In the future, the Hurd might be ported to other, more modern microkernels. A good candidate seems to be L4, a tiny microkernel with a size of about 50kb and only 7 system calls. Those 7 system calls provide enough functionality to create a whole POSIX compatible operating system on top of it.

The microkernel is providing primitive operations that make it possible to share the hardware resources (processor time, memory, ...) and make different tasks cooperate (IPC). However, it doesn't define the actual policy by which these tihngs should happen. This has to be done by user space programs running on top of the microkernel.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

4. Breaking it up

The Hurd servers, which run on top of the Mach microkernel, implement the functionality that is usually found in a Unix-like kernel. But instead having one large monolithical server, it was decided to split up the system into many individual server processes, each server implementing a separate and distinct function.

All the Hurd servers communicate with each other and cooperate in implementing the features of the system. For example, if a user tries to open a file, the file system server has to rely on information provided by the authentication server to check if the user can be allowed access to the file. Another example: If a process wants to send a signal to another process, it has to look up the message port (to which the signal is then sent) of this process by consulting the proc server.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5. And putting it together again

The interfaces by which the servers communicate are at the core of the Hurd system. Although you can write programs specifically to talk directly with the Hurd servers, the usual way to add programs to the Hurd system is by basing them on the POSIX.1 interfaces. Except for bugs, the Hurd is a POSIX compatible operating system. However, as in GNU/Linux, the actual POSIX layer is provided by the GNU C library. Sometimes, POSIX functions map directly to corresponding Hurd server interfaces. For example:

 
read (fd, ...)  <=> io_read (ioport, ...)
write (fd, ...) <=> io_write (ioport, ...)

getpid ()       <=> proc_getpids (procport, ...)

However, a lot of the POSIX functionality can actually be implemented by the user itself, rather than punting it into the system code. The C library does this for the user, too. For example, fork() is not implemented by any Hurd server, but in the C library directly, which uses Mach and Hurd server primitives to create a mirror image of the current process. The user is free to use his own implementation of fork(), if he desires so. For the standard case if POSIX.1 behaviour is desired, the default implementations in the GNU C library provides exactly that.

Although the GNU C library provides a POSIX personality on top of the Hurd, and this is the main personality of the Hurd today, there is no reasons why other libraries shouldn't use the Hurd server's functionality to provide a compatibility layer to some other API. If this is done carefully enough, then programs from both worlds can take advantage of the common Hurd base, for example by sending signals to each other.

By putting the lot of the system's functionality into userland, the Hurd has several advantages:


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

6. Accessing the servers

Accessing the various Hurd servers is not done via a nameserver as traditionally has been done in Mach. The simple reason is that POSIX.1 compliant systems already have a name service we can use, and this is the file system. Every mounted filesystem is provided by a different server. Open files and directories are actually represented by ports to the servers providing these files. So it makes sense to just use the filesystem servers as name services to access other Hurd servers.

For example, the Hurd password server hands out authentication handles in exchange for a password. It is used by programs like login, so those programs don't need to be run suid root. The password server has the canonical place /servers/password in the filesystem. If a user wants to communicate with the password server, he looks up the /servers/password node, and sends RPCs to the port he gets returned. The filesystem server has intimate knowledge about the translators attached to its nodes, so it can redirect the user to the translator's service on each such file name lookup.

There is no real difference between filesystem servers like ext2fs and special purpose servers like password. They implement different protocols, but they are all part of an hierarchical tree of Hurd servers, each server connected to the node in the parent filesystem it is attached to. Even a single file node like /servers/password is implementing a small filesystem.

The Hurd provides a way to store a translator command into a node of the filesystem. When an access on the node is made, and there is no active translator yet, the passive translator setting will be read out and the filesystem server will make an attempt to start up the filesystem using the command line from the passive translator setting. Because the information is written into the filesystem, it remains there across reboot, so there is no need to set up the translators after every boot. They will be started dynamically on demand and transparently to the user.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

7. Inviting the user

One problem in Unix-like systems mentioned earlier is that a lot of useful functionality is reserved to the superuser, because it frequently involves privileged operations. For example, only the superuser can mount filesystems by default, although the mounting of filesystem image files is useful, for example when preparing a CD image. And even if users are allowed to mount filesystems, they can only mount filesystems of the supported types for which modules exist in the kernel. In general, there is a barrier around the system code that restricts the users freedom in using the system.

The Hurd eliminates the system code barrier as far as possible. This is achieved by designing the Hurd servers interfaces in a way that allows for communication between untrusted servers and users, without compromising the security of the system. This makes it safe to allow users to start their own Hurd servers and attach them to the filesystems at nodes that are owned by this user.

A central role is taken by the auth server, which is a central authority giving out authentication handles to users. Only root can request arbitrary authentication handles, other users are restricted to merge and subset operations on existing handles. Beside support routines for managing auth handles, the auth server provides a handshake protocol to establish a trusted channel of communication between an untrusted server and an untrusted client. As the result of the protocol, the server can associate the messages from the client with the user and group ids in the clients authentication handle, and thus implement appropriate access control.

For this to work, the server must trust the auth server, and the client must use the auth server trusted by the server. This means that if you want to communicate with a server, you will have to authenticate yourself with the auth server it trusts. All system servers will use the system auth server as the trusted one by default. Nothing forces the user to register itself with the system wide auth server, but nothing else will allow the user to communicate with the other system servers, so it is usually unavoidable for a user to use the system auth server.

However, it is possible and useful for a user to start his own authentication server in addition to the system server and use that as the trusted authserver for the user's own servers.

For other servers, the system often provides a standard way to use a user provided server rather than the system default (where it doesn't do that yet it probably should). For example, the crash server takes control over crashing tasks. It can suspend them for later debugging, dump a core file, or just kill the task without leaving a trace of the crash. However, if a user implements his own crash server that sends a message to a mobile phone when the tasks crashes, the user just needs to set the environment variable CRASHSERVER to point to his own server when starting the program to take advantage of that feature.

Similarly, the exec server is reponsible to parse an executable file and take the appropriate action to start the program, for example by starting the interpreter. A user could provide his own exec server that understands a new object format, and set the EXECSERVER variable before starting it. This feature is disabled currently, though, because of security concerns. We are not sure yet that the current code covers all possible corner cases in a safe manner.

Another way for a user to extend the system is to add new filesystems and attach them to nodes in his home directory. With the ftpfs server, everyone can add a remote ftp repository to their filesystem hierarchy. Other filesystems might provide transparent access to other hierarchically organized data, as LDAP servers, or even tar files. Of course, beside the full blown filesystem servers, small servers only providing one file can also do a useful job. One program I wrote, the run translator, provides a file that on each open() spawns the program given on its argument line and provides its output as the content of the translated file. This can very easily be used to randomize ~/.signature for example, without worrying about mail user agent configuration at all.

If replacing or adding individual servers is not sufficient, a user can boot a second Hurd system from a filesystem image with the program boot. This will start another Hurd system that runs in parallel to the first one. Because this Hurd system contains its own auth server, the user will have root permissions in his Hurd system, but despite this appearance there is no actual permission leak, as the system defaults server in the original Hurd system will continue to use the trusted system auth server. And that won't be fooled by any process related to the second Hurd system to have any different permission as the user running the second Hurd system has.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

8. Solving some problems in Unix

A common problem in using computer systems is interoperability between individual programs. The output of a program can serve as the input for another, for example by using a pipe. In general, one program will use the services of some programs, and provides services itself to other programs in the system.

In Unix, pipes and sockets are powerful features to connect programs, but they are limited to the communication between user space processes. If you want to affect the communication between a user space process and the system code (eg the kernel), Unix does not provide you with any standard facility to do that. Usually a program is linked to the C library, which directly invokes the relevant Unix kernel system calls, for example to rename a file.

But, there is a real need for users and developers to intercept or redirect communication with the kernel. Here are some examples where it is attempted to do that in Unix, and how it is achieved:

So despite the fact that Unix makes it difficult, tedious and sometimes downright impossible to implement these features in their cleanest form, the need for them has forced people to implement them anyway and cope with the limitations. How can the situation be improved?


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

9. Solving the problems in the Hurd

The Hurd avoids this problem in two ways. First, it moves the system code that implements the filesystem and other functionality provided by the Unix kernel into the user space. But more importantly, it breaks up the monolithic system core into many individual servers, and uses a challenge-response protocol to establish a communication between an untrusted server and an untrusted client. This makes it possible to allow users to plug their own servers into the system.

The GNU C library implements the POSIX environment that applications expect on top of this. It ensures that all server communication happens transparently to the user. Filesystem services provided by the user's server are transparently started and accessed. At startup, a program inherits some basic server ports from its parent. Among these are the root filesystem port, the current directory filesystem port, a port to the authentication server establishing its identity to the system and others.

This is all very abstract, so let's take a look at how we can solve some real world problems this way:


[Top] [Contents] [Index] [ ? ]

Table of Contents

1. History
2. GNU/Linux is like Unix
3. GNU is not Unix
4. Breaking it up
5. And putting it together again
6. Accessing the servers
7. Inviting the user
8. Solving some problems in Unix
9. Solving the problems in the Hurd

[Top] [Contents] [Index] [ ? ]

Short Table of Contents

1. History
2. GNU/Linux is like Unix
3. GNU is not Unix
4. Breaking it up
5. And putting it together again
6. Accessing the servers
7. Inviting the user
8. Solving some problems in Unix
9. Solving the problems in the Hurd

[Top] [Contents] [Index] [ ? ]

About this document

This document was generated using texi2html

The buttons in the navigation panels have the following meaning:

Button Name Go to From 1.2.3 go to
[ < ] Back previous section in reading order 1.2.2
[ > ] Forward next section in reading order 1.2.4
[ << ] FastBack previous or up-and-previous section 1.1
[ Up ] Up up section 1.2
[ >> ] FastForward next or up-and-next section 1.3
[Top] Top cover (top) of document  
[Contents] Contents table of contents  
[Index] Index concept index  
[ ? ] About this page  

where the Example assumes that the current position is at Subsubsection One-Two-Three of a document of the following structure:

This document was generated by Marcus Brinkmann on June, 1 2002 using texi2html