DCE: Alive and well in a data-centre near you

Why Linux Developers should know about DCE


Table of Contents
History
Current Status
The Players
What's it used for? Who uses it?
Why am I telling you all this?
The Future
More Information
Why Linux Developers should know about DCEChris Benson

The OSF's Distributed Computing Environment was developed in the early 1990s to meet an urgent need: to simplify the development of secure, scaleable, distributed applications.

DCE is middleware available on all commercial computer systems from PCs to mainframes and supercomputers. Until recently it has not been available on the free Unix platforms, which has limited the use of Linux in environments where it would otherwise be very useful. A few years ago the licensing for DCE was changed and the source code made available (but not open-sourced). A non-commercial project to port DCE to Linux fizzled out, partly due to Linux's rapidly changing code-base. Now a commercial port of DCE for Linux is available from one of the existing DCE vendors.

DCE is used by some application developers directly, but is more often embedded in other packages and development environments. DCE is used all over the world, usually in very large systems, by business, government departments and academia.

This article gives a brief history of DCE, describes the services it provides and reports the current availability of DCE on Linux. It looks at the companies using DCE and indicates the sort of applications for which it is used.

I will explain why Linux developers should know about DCE -- and speculate on the opportunities for Linux and DCE in the 21st century.

First EditionCopyright © 2001 by Treepax Ltd.June 2001

All trademarks are property of their respective owners.

History

Business Aims

The Open Software Foundation was created in 1988 by Digital, IBM, HP, Bull and other companies as a balance to Sun and AT&T's formation of the Unix International consortium and their promotion of Unix SVR4. The aim of the organisation was to promote interoperability between computer systems and to standardise (on their version of) Unix.

The OSF developed three major products:

  • OSF/1. The OSF's version of "the One True Unix".

  • Motif. The OSF's participant in the Unix Desktop Wars.

  • DCE. The Distributed Computing Environment. A software environment for the development of distributed applications. Another piece of "middleware".

Technical Aims

Use proven, available technology. The timescale for a finished product did not allow for research, trials and rework. The OSF selected technologies from those submitted by its members and integrated them to create DCE.

Be platform (vendor) neutral and guarantee interoperability. The OSF could not favour any of its members over another. It wanted to ensure interoperability between all implementations of a technology.

Enable scaleable, resilient solutions. The completed system should not set arbitrary limits on the size of the environment. The environment and applications running in that environment should not have single points of failure.

Implement platform not policy. Make the environment universally applicable by not making limiting decisions. This enables DCE to be used for N-tier and peer-to-peer applications, as well as for simple client/server applications.

The Technologies

Interprocess Communication

A distributed environment requires a communication method. The OSF selected Apollo's Network Computing System, contributed by Hewlett Packard with some input from Digital.

NCS is a Remote Procedure Call (RPC) mechanism. Developers specify the interfaces to applications as function calls, using the Interface Definition Language:
[
    uuid(406c45a0-3b7e-11d0-80a3-0000c08810a7),  version(1.0)
]
interface drink
{
   typedef [context_handle] void *drink_handle;
   drink_handle	open_tab();
   long		get_drink( [in]	drink_handle my_tab);
   void		get_tab( [in,out]	drink_handle *my_tab);
}
An idl compiler reads the IDL file and creates stub code for client and/or server programs. The stub code hides the data marshalling, network transfers and, if necessary, data format conversion.

The features of the DCE RPC mechanism are:

  • Each interface is identified by a universally unique number, the UUID, and a version number. The version number enables multiple client versions to find compatible servers.

  • Data is transferred across the network using Network Data Representation (NDR), the native representation of the sender with information about byte-order etc. The receiver is responsible for converting to its native format. This minimises the amount of format conversions and distributes the load between clients and servers.

  • The original IDL defined C-like data types: integer and floating point numbers, arrays and pointers and generated stubs in C. In DCE v1.2 the interface definition language was extended to support C++ objects and their methods. An IDL compiler is available for Java; other languages are usually supported by calling C wrappers around the RPC.

  • As well as simple data types, the IDL supports dynamically sized arrays and passing only a portion of an array.

  • The RPC mechanism is independent of the underlying network protocols: while TCP/IP is most commonly used, it has run on DECnet, Novell's IPX/SPX and IBM's SNA.

  • DCE RPC also provides several different call behaviours: the default is "at most once", the RPC will be executed once or not at all. This reliable behaviour is independent of the network transport protocol: it works over both UDP and TCP. If it does not matter that an RPC is executed more than once, it can be marked "idempotent" which saves some bookkeeping overhead. Normally RPCs are sent to a specific server. The "broadcast" attribute sends the RPC to all connected servers and uses the first reply. This requires a transport protocol that supports broadcasts. Finally, if the client does not need a response from a server the RPC can be marked with the "maybe" attribute.

Security

The technology chosen to provide authentication across an insecure network is Kerberos 5 from MIT. But DCE was designed as an open platform: the authentication method to be used is passed as a parameter when authenticating. Since version 1.1 DCE has also supported the GSS-API, enabling DCE applications to authenticate with other services.

Kerberos provides authentication of "principals", people, machines or programs using a shared secret, without transferring any passwords across the network in clear text.

DCE has extended Kerberos (in a compatible way!) to store and manage information about groups of principals. In DCE this is called the privilege service.

One or more replicated Kerberos servers provide an authentication service to a group of computers in a realm. In DCE the Kerberos realm is called a cell. All the machines in a cell share security, directory and time services and all access to those services is controlled by the privileges granted to the requesting principal.

A secure application requires both client and server to be authenticated (and each computer automatically authenticates itself before the programs attempt to authenticate). A client wishing to contact a server gets a session key from the security service that can be used only by the target server. If a server receives a valid session key, it can be sure that the client is authenticated. If the server can decrypt the session key, the client making the request can be sure that the server is authenticated.

After authentication and obtaining credentials from the privilege service, access is controlled by POSIX Access Control Lists (ACLs) supplied by HP. ACLs are used by all parts of DCE. In the security part of the namespace, ACLs control maintenance of and access to principals and groups. In the directory namespace, ACLs control creation of new directories and access to server binding information. Applications can also use the same APIs and tools as the DCE services to control access to services.

Directory Service

In a large, dynamic, distributed environment just finding the required service can be difficult. The DCE Directory Service provides a secure, scaleable, hierarchical namespace for storing information, usually information about servers. The service is delivered through several other services: CDS, DNS and X.500.

Within a cell server programs register their address information, their "binding information", with a Cell Directory Service server. Client programs request the binding information of servers with compatible interfaces. The request is mediated by a "CDS Clerk" running on each machine in the cell. The clerk caches results of queries for future use.

Requests outside the local cell are passed to the Global Directory Agent which determines the type of address and passes the request to the relevent global directory service: either X.500 or DNS. The global directory service should return the binding information of a CDS server in the remote cell and the lookup continues.

The information stored by CDS servers can be replicated on multiple servers for reliability and performance. The CDS namespace can also be partitioned for scaleability and performance, and to facilitate delegation.

All access to the CDS namespace is controlled by the rights granted to authenticated principals by ACLs. Server applications need permission to store their binding information; clients must be given permission to read that information.

CDS came from Digital and the Global Directory Service (an X.500 implementation) from SNI.

Time Service

In a distributed environment each computer clock may run at a different rate, giving a different system time on each machine. This can cause problems for some applications, for example backups or make builds. DCE's security depends on the timestamped tickets used to authenticate principals; for this to work all machines in the cell must have a common view of time. The Distributed Time Service keeps the system clock of all the machines in a cell synchronised by running a DTS client or server on each machine. One of the DTS servers can be connected to an external time provider to keep the cell time accurate.

The DTS is unusual in that DCE does not require it to be running. DCE only needs machines to share a common time, so other time protocols can be used. DTS was based on the Digital Time Service.

Threads

What have threads to do with distributed computing? They provide portability. All the DCE servers are multithreaded for performance, and applications written using DCE can also be multithreaded. Each client connection is allocated a thread within the server application to deal with that request. DCE includes a thread library so that there is a standard interface to threads on all supported platforms.

The OSF supplies a user-space library for those operating systems that do not support threads. Most vendors now implement threads in the operating system, so only have to provide an interface that is compatible with the DCE implementation.

DCE threads are based on a draft copy of the POSIX 1003.4 standard which is itself based on Digital's pthreads.

Integration

The strong point of DCE is not so much the technologies themselves but the integration of the parts. All the DCE services make use of the other services. Security clients use CDS to find security servers. CDS servers are authenticated by security servers so that clients can be sure they are dealing with a real CDS server. DTS clients use CDS to find DTS servers, and both clients and servers authenticate with the security service to get the privileges they need. All the servers are multithreaded and all communication between programs is by RPC.

Timeline

1987. OSF formed.

1988. The OSF issued a "Request for Technology" (RFT) to members.

1989. Selection of the technologies to be used.

1990-1. Integration.

1991. DCE v1.0 released by the OSF.

1994. DCE v1.1 released. This version made important improvements to security, administration and performance.

1996. DCE v1.2 released. This release improved performance and robustness, and extended the facilities for developers. The OSF merged with X/Open to form The Open Group.