A Scalable Internet Resource Service, Part 2

M.R. van Steen, A.S. Tanenbaum

September 29, 1999

1 Background

The initial SIRS project (referred here to as SIRS-1) concentrates on developing client software for supporting Globe distributed shared objects. As of this writing, a prototype version of the client software is running, allowing end users to download Globe Web Documents from existing browsers. Client software consists of:
  1. A Globe name server based on BIND8, which resolves a Globe URN into a location-independent object handle (OH)
  2. A Globe location server, written entirely in Java, that resolves an OH into one or more contact addresses for the identified Globe object
  3. A proxy that can bind to an object given a contact address, and which interfaces to existing Web browsers, by translating Globe URNs into equivalent URLS. The proxy is also written in Java.

SIRS-1 has originally been proposed as consisting of two phases. Development in phase 1 concentrates on client software, including the name server, location server, and the proxy. Phase 1 is scheduled to finish by 01.12.1999. For phase 1, the following activities remain to be done:

  1. Extending the location server with persistence, so that contact addresses can be stored on disk.
  2. Testing, the client software by configuring a number of local Web pages as Globe Web documents, and which are accessible through our current client software (installed at the VU).

All client software will be made available by 01.12.1999 to a number of sites. The sites are yet to be selected.

Phase 2, from now on referred to as SIRS-2, concentrates on the development of server software, and the development of a serious application demonstrating the feasibility of the Globe approach.

SIRS-2 is divided into two subprojects. Subproject SIRS/Server deals with developing server software. SIRS/GDN (Globe Distribution Network) concentrates on development of a network for the distribution of files, in particular freeware and shareware.

2 Subproject SIRS/Server

We have already started the process of designing a generic Globe server, capable of supporting Globe Web documents and Globe distributed shared objects in general. A collection of Globe servers will form a Globe virtual network for physically distributing Globe objects. The development of the Globe server is in accordance to the original SIRS-1 project plan. The main difference is that we plan to deliver prototype versions earlier. Planned dates and activities are as follows:

Date of delivery Activity
01.12.1999 Initial design of the Globe server, describing its functionality and internal workings.
01.01.2000 Detailed technical design of the Globe server, expressed in terms of interfaces offered by its various components. This design acts as the basis for the implementation efforts.
01.06.2000 First prototype implementation Globe server, to be tested on the VU local network (see also subproject SIRS/GDN).
01.09.2000 Second prototype implementation Globe server, to be tested on a wide-area network with only a few (2-5) sites (see also subproject SIRS/GDN).
01.12.2000 Prototype implementation Globe server, now working to support the worldwide Globe Distribution Network (see also subproject SIRS/GDN).

Subproject SIRS/Server requires an estimated effort of I FTE. Details concerning this subproject can be found in the original project proposal for SIRS-1.

3 Subproject SIRS/GDN: The Globe Distribution Network

In addition to the development of the Globe server, we propose to develop software that will allow Globe to be used for the (worldwide) distribution of files, in particular freeware and shareware.

3.1 Overview

As an application of a Globe virtual network, we propose to develop a network consisting of Globe servers configured to support a worldwide infrastructure for distributing software (and other resources). This Globe Distribution Network (GDN) will have the following properties:

The development of the Globe Distribution Network (GDN) should be done in parallel to the development of the Globe server in SIRS/Server. In particular, as soon as the design of the Globe server has finished, we will have insight in what specific tools we need to set up and maintain GDN. For example, we will need to tools to monitor and set-up servers, make it easy to add, remove, and update resources, etc.

3.2 Details

Within SIRS/Server we design and develop a generic Globe server capable of handling distributed shared objects. To demonstrate the feasibility of our approach, SIRS/GDN concentrates on using the Globe software for implementing an overlay network to support distributions of freeware and shareware. To that end, we need to develop a number of tools that support the following functions:

  1. Listing files stored in the GDN
  2. Searching for specific files
  3. Downloading a selected file
  4. Adding, a file to the GDN
  5. Removing a file

Operations are permitted by authorized users only. In particular, adding a file to the GDN requires special security measures. In addition, the GDN should be capable of handling "traditional" malicious attacks. To simplify security issues, we initially assume the GDN servers operate behind a firewall.

The GDN will logically consist of two different parts. The first part consists of the actual files that are stored in GDN. Each group of files (i.e., that make up a distribution package) is implemented as a simple Globe distributed shared object. The simplicity comes from the object's interface, which allows a user only to fetch the contents of a file to store it locally. However, each distribution package can have its own replication strategy associated with it.

The second part of the GDN handles naming and locating distribution packages. It consists of a name space implementation which maps user-readable names into object handles. This name space implementation is the one developed in SIRS-1. In addition, a location service returns the address of each replica of the referred file. Again, this is the same implementation as partly developed in SIRS-1.

The GDN will initially be implemented throuoh a limited number of sites (5-15), where each site runs a GDN server. To simplify matters, each site also runs a complete version of the name space implementation. This means that the name space is replicated on each GDN site. A simple update protocol will be used to add names. The location server is distributed across the GDN sites by means of a distributed search tree.

To build GDN, the following components and tools are needed:

When a user has selected a distribution package, it can be downloaded in its entirety, or per file. Details on the actual downloading need to be filled in. A possible scenario is the following:

  1. A client uses a standard Web browser, possibly cooperating with the SIRS client software to handle Globe URNS, to access the Globe Distribution Network.
  2. A client binds to a Globe directory object, allowing, it to list and select other directory objects, or distribution packages. Selection of another object implies that the client binds to that object.
  3. When a client has selected, and is bound to, a distribution package, it is offered a (possibly dynamically generated) HTML page to download the files. Each file appears as a hyperlink, implemented as an FTP URL pointing to the file on the specific GDN server to which the client is connected.
  4. The GDN server is colocated with an FTP server on the same machine. Downloading, files is completely handled through the FTP server.

This approach has the benefit that GDN is used only to handle per-package distribution requirements, simplifying its initial design and implementation. Existing protocols and software can be used to handle the actual transfer of packages to the client.

3.3 Plan of Work

SIRS/GDN can start on 01.12.1999. The following activities have been planned, and to be completed at the mentioned dates:

Date of delivery Description of activity
01.02.2000 Initial design of the support environment needed to set-up and maintain a full-fledged GDN. The design will also consist a list of specific tools that need to be developed to manage the GDN. A selection will be made of the tools to be developed as part of SIRS/GDN.
01.06.2000 A first collection of tools will have been developed to instantiate and manage a prototype version of the GDN. The GDN will be installed on the VU local network, to allow testing and feedback from local users.
01.07.2000 A plan will be drawn up to systematically test the prototype version. This plan will also contain a list of sites external to the VU that have indicated to assist in running the GDN in a wide-area network.
01.09.2000 An improved version of the first prototype will be developed and installed at a few (2-5) sites outside the VU. This prototype is used to test the wide-area version of the GDN.
01.12.2000 A final, again improved version of the GDN prototype will be developed and installed at the complete set of selected sites. This version acts as the first worldwide prototype of the GDN.

Subproject SIRS/GDN requires the estimated effort of 1 FTE.

4 Summary

The two subprojects are closely related to each other. SIRS/Server concentrates on the core development efforts of a Globe server capable of supporting distributed shared objects. SIRS/GDN concentrates on the development of tools and actual set-up of a distribution network, based on the Globe server. There are a number of joint deliverable dates:

01.06.2000 First prototype Globe server and local prototype of GDN
01.09.2000 Improved server and initial wide-area version of GDN
01.12.2000 Final prototype version Globe server and worldwide version of GDN

The software developed and running on 01.12.2000 will be made publicly available as source code to stimulate further developments by other parties, unless the VU and NLnet foundation decide otherwise.

Back to SIRS project page

Back to Stichting NLnet projects page

RCSID: $Id: sir2-proposal.html,v 1.1 2001/03/08 09:41:46 wytze Exp $