SiteCast

From GridSiteWiki

SiteCast is a file-location technology developed for the GridSite project (and added in version 1.1.12) SiteCast clients use HTCP messages transmitted over UDP multicast to query GridSite fileservers for the presence of specific files within a shared SiteCast URL space. The UDP responder is implemented as part of the GridSite Apache module, and if the file is present, the responder replies with an HTTP redirection to the transfer URL of the file on that server. If multiple servers reply, then the client must choose from which to fetch the file.

Table of contents

Using HTCP for file location

The HTCP Protocol was developed as a lightweight way for a group of web caches to query each other for the presence of a given URL. The main operation, TST, presents an HTTP request to the cache. The cache replies by saying whether it has the corresponding file/page, and may return the cached HTTP response headers. The HTCP messages are stateless and sufficiently short that they are naturally transmitted via UDP rather than TCP.

Within SiteCast, HTCP queries are interpreted as the question "how would you respond to this HTTP request?" This interpretation allows HTCP to be used to query file servers rather than web caches.

Since HTCP is naturally transmitted over UDP, then it is straightforward to take it a step further and multicast the queries, rather than direct them to individual servers.

Groups, domains and URL-spaces

IP multicast allows hosts to subscribe to multicast groups, which take the form of reserved IP numbers in the range 224.0.0.0 - 239.255.255.255. Since groups are mapped to ethernet addresses, most unwanted multicast groups can be ignored in the ethernet hardware or device driver layer, without reaching the level of the IP stack or the userspace code.

SiteCast clients can query multiple multicast groups, perhaps in order with a time delay between each one. This feature allows fileservers to be grouped according to network proximity, CPU/memory or even who owns the machine, and then higher priority given to particular groups. For example, a group could be assigned to each rack of hosts which share the same switch; clients can then query their local group to try to find a "close" source for the file, and if that fails, then query the group containing all hosts in the storage farm.

A transfer URL is needed to really fetch the file, but this will include the specific hostname which has the file. When querying all hosts, the file is expressed as a SiteCast URL, containing a site-wide domain name, scheme, port and the local path on the fileservers:

https://sitecast.hep.man.ac.uk:488/dir/file.txt

All of the hosts which will respond to SiteCast queries containing the generic domain name "sitecast.hep.man.ac.uk", are said to be part of the same SiteCast Domain.

To test for the presence of the file, the host maps the path part of the URL into a path in the local filesystem. If the file is found, the host replies to the client, specifying an HTTP Location: header which gives the absolute URL of its copy of the file:

https://host001.hep.man.ac.uk:488/dir/file.txt

Consequently, the SiteCast domain forms a shared, site-wide URL-space, in which all hosts in the storage farm participate.

Operational advantages for storage farms

As well as the grouping of hosts by proximity, this architecture provides a measure of load balancing (unloaded hosts are likely to respond first), failover when hosts fail (if a host goes down, then it stops responding to multicast queries), and natural support for replication of files (multiple hosts can reply if the file is replicated.)

Support in htfind and htcp commands

This command will copy a file located via SiteCast to the local disk:

htcp --domain sitecast.hep.man.ac.uk --groups 224.0.1.111 \
 https://sitecast.hep.man.ac.uk:488/dir/file.txt /tmp/file.txt

See also the htcp man page (http://www.gridsite.org/doc/HEAD/htcp.1.html)

Support in SlashGrid

SlashGrid includes support for reading files located via SiteCast, with virtual paths in the local filesystem like /grid/https/sitecast.hep.man.ac.uk:488/dir/file.txt

These files can be read just like any other other remote file accessible to SlashGrid via HTTP(S). The SiteCast domain(s) and groups to use must be specified on the command line of the SlashGrid daemon, and this easily can be done via the OPTIONS variable in the /etc/sysconfig/slashgrid file.

mod_gridsite directives

mod_gridsite man page (http://www.gridsite.org/doc/HEAD/mod_gridsite.8.html)