Goto Source [Stable: Version 1.2/ Snapshots]; Documents

Programs of the StarBED Project

Table of Contents
sheepdog snapshot xant snapshot

Terminology

node
The unit of experiment resource. Most of them, the term means PC.
RM, ERM -- (experiment) resource manager
Resource manager serves information of resources and stores ownership for exclusive using. The program dispatches resources according to the condition of requests.
ENCD -- experiment node configuration driver
The program drives nodes of experiments.
FNCP -- facility node configuration pilot
The program navigates nodes to their driver(ENCD) for configuration.
DMAN -- directory manipulator
Dman makes and removes symlinks on the remote filesystem. It is useful to switch the node behavior on TFTP and FTP servers' directories
NA -- node agent
NA is the program on the node.
NI -- node initiator
An implemenation of NA. The program does a data transfer between local disk and remote file server. So, it is upload/download of diskimage.
wolagent, wol_agent -- Wake on LAN (WoL) agent
The program issues Magick Packet to boot node, but it is not relay.

Experiment Overview

An experiment consists of a driver host and runnner hosts. The driver sends to the runners a sequence of experiment steps to be performed on the runner. Each runner executes the steps in the sequence. The driver and the runner exchange messages. These messages are used as parameters and commands on the experiment.

To save cost, network experiment facilities are generally designed to support running many experiments at the same time. However, when many experiments run on the same facility, it requires an arbitration of resources. In addition, data storage is required. To this purpose we employ dedicated facility side management servers to control the experiment hosts (driver and runner).

Programs included in SpringOS and related to it are divided into 2 sides and 3 categories.

Experiment SideFacility SideDescription
CategoryDriverRunnerManagement
SpringOS master, sbpsh, sns, ks.pl slave, ifscan, snmpmine erm, fncp, dman, wolagent Used during normal operation
pickup, wipeout ni   Used for disk backup/restore
   xant, sheepdog Used for health checking
Related tools    dhcpd,tftpd,ftpd Used to ...

Facility-Side Servers

In order to manage multiple simultaneous experiments running on a facility with a wide range of equipments, we designed the resource manager, erm. According to the database of resource manager, a node monitoring tool, sheepdog, probes the state of resources by using protocols such as ICMP, TCP and IPMI.

Initially nodes are not aware of their role in an experiment, and have no configuration. To avoid setting them up manually, we employ PXE, DHCP and TFTP related technologies in boot phase, and employ FNCP (Facility Node Configuration Pilot) and ENCD (Experiment Node Configuration Driver) negotiation in the experiment phase.

Here we only describe FNCP and ENCD negotiation. Detailed information about PXE, DHCP and TFTP can be found in REFERENCES. The agent program on each node aims to retrieve node configuration from the coresponding storage. However, the agent doesn't know in the beginning which host stores this data. We decided to use a single contact point for node agents. On the contact host, the program named FNCP redirects the agent to ENCD. Through the redirect of FNCP, the agent knows which is the configuration folder, ENCD.

ENCD has to register with FNCP in order to benefit of its service. The following figure depicts the flow of the node configuration retrieval by node agent, NI, via FNCP.

the negociation for node configuration

SpringOS expects WoL-capable PCs for node booting, since WoL is the de-facto standard for PC booting. The program wolagent is the WoL packet sender. On request from other programs, the program sends WoL packets in the data link network. Therefore one should build the program on each data link.

As an optional feature, we implemented a directory manipulator, dman. Using dman, one can access the directory of TFTP daemon. Thus one can change the boot loader, the disk images, etc. In StarBED we employ this program to switch the operating systems of experiment nodes.

User-Side Programs

Several programs can be used on the user side, and they will be briefly described in what follows.

The node initiator, NI, is a node agent software. The program controls disk partition and network configuration. According to configuration specified for each node, the program downloads the partition contents from FTP server programs. As it was mentioned above, the program retrieves the configuration from ENCD (usually via FNCP). This program runs before the experiment, in the node setup phase. Most users do not need to modify its behaviour.

StarBED already supports multi-OS bootable nodes. The boot loader is a key compornent for this purpose. The original boot loader was supplied by NEC. However, it is not open source. Therefore we developed a custom-made boot loader, coil. This program runs before the experiment, in the node setup phase. Most users do not need to modify its behaviour.

Kuroyuri

Kuroyuri is package of ENCDs and related programs.

Several programs are used in connection with Kuroyuri. ifsetup is a package managing network interfaces on nodes. ifscan scans all network interfaces on the node. slave (an evaluator of Kuroyuri) expects these programs to run in order to setup interfaces. ifscan should be installed on target nodes. In many cases, the user installs that on the disk image of the reference machine. For automatic reboot and shutdown SNMP is used in StarBED. snmpmine is an implementation used for this purpose.

SheepDog

SheepDog is a program that probes host status on the facility, and allows users to display it. The program is an http server (httpd), therefore users can check status of hosts using WWW clients. Futhermore, this program supports host power control. Users can boot any hosts using the provided interface.

By clicking on the image, a larger version can be displayed.

xant

Xant is a program that shows the direction and volume of traffic in StarBED. The program scans interfaces on specified switches, and retrieves statistical values. These values are graphically displayed over a specified topology figure.

By clicking on the image, a larger version can be displayed.

In large-scale testbed, it is difficult for operators and users to have a global knowledge about the topology and the traffic. Xant was designed to help them with these issues. The table below indicates the configurable elements of Xant.

ElementConfigured by
Switch name & community nameUser
Switch interfacesAutomatic (via SNMP's GETNEXT)
Topology figureUser
Position of flowsUser

Key Points of SpringOS Installation

For SpringOS installation, 4 categories of programs are required:

Other recommended programs may be installed as well:

References

[IPMI]
Intel, Hewlett-Packard, NEC, Dell: Intelligent Platform Management Interface Specification Second Generation, Feb, 2004.
See IPMI v2.0 specification Document Revision 1.0 in http://developer.intel.com/design/servers/ipmi/
[PXE]
Intel Corporation: Preboot Execution Environment (PXE) Specifiction Version 2.1, Sep, 1990.
[DHCP] RFC2131
R. Droms: Dynamic Host Configuration Protocol, RFC2131, March, 1997.
[TFTPD] RFC1350
K. Sollins: The TFTP Protocol (Revision 2), RFC1350, July, 1992.
[FTPD] RFC959
J. Postel and J.K. Reynolds: File Transfer Protocol, RFC959, October, 1985.

Appendix

Table of Logical Modules
namedescriptionimplements
RM, ERM(experiment) resource manager erm
FNCP facility node configuration pilot fncp
ENCD experiment node configuration driver master, pickup, wipeout
DMAN directory manipulator dman
NA node agent ni
WOLA WoL agent wolagent

Table of Protocols
#namedescription
RFC? HTTP HyperText Transfer Protocol  
? ICMP -  
? ARP -  
? DHCP -  
? TFTP -  
? FTP File Transfer Protocol  
? SNMP Simple Network Management Protocol  
? IPMI -  
? PXE -  
private ERRP Experiment Resource Reservation Protocol  
private ESQP Experiment Structure Query Protocol  
private DMANP Dman protocol  
private WOLAP WoL agent protocol relay packet for power on

Table of Packages and the Programs They Contain
side package program
name description name description
facility erm resource manager erm database and dispatcher of resources
sheepdog resource monitor sheepdog resource monitor
fncp facility node configuration pilot fncppilot
dman directory manipulator (symlink operator) dmanmanipulator
wolagent node boot agent wolagent boot nodes with WoL packet
user (hidden) ni node agent ni node initiator (disk image installer in the node)
coil boot loader (NEC compatible and more) coil boot loader which supports hierachical partition (e.g., /dev/hda11)
user kuroyuri ENCD (Experiment Node Configuration Drivers) Sometimes work-in-progress packages are named differently (ex. mtk- and dck- ). mastermaster of evaluators; parse experiment description language, distrubutes scenario for each nodes, and evaluate global scenario.
slavenode scenario evaluator
pickupdisk image uploader
wipeoutdisk image wiper (bulk downloader)
sbpshshell for StarBED
skipnsetupbulk reporting program link 'dummy-ni'
ks.plGUI demo program
ifsetup network interface set up ifscan NIC scanner
mine shutdown and reboot agent (NEC compatible) snmpmine shutdown and reboot agent via SNMP with NEC-MIB


Copyright StarBED Project © 2006. All rights reserved.