Data Transport Network
Introduction
The Data Transport Network coordinates the collection of scientific data, instrument telemetry, post-processing and the delivery of the real-time results over the Internet. It is designed for the delivery of data from scientific instruments located at remote field sites with limited or unreliable network connections. The core idea behind the network is that data files are transfered as attachments in Usenet messages.
A common problem faced by researchers with instrumentation at remote field sites is reliably accessing their data files over a network connections that can be unreliable and limited in bandwidth. The Data Transport Network was originally developed to in response to these needs for the Incoherent Scatter Radar Facility in Sondrestrom, Greenland. The Data Transport Network collects data files locally written by instruments and buffers them up for delivery over the Internet. During disruptions of the satellite connection to the site, these files are queued up until the network is available again. The instruments themselves are insulated from these network interruptions and do not need to take an special action.
The connection out of many remote sites is bandwidth limited as well, which limits the amount of data that can be transferred. To help reduce the volume of data, the system allows for the integration of post-processing programs. These programs are automatically run as new data files are detected and can be used to transform large raw data files into more compact results. Only those compact results are then transfered across the link. The system has been successfully delivering data from multiple radars, lidars and imagers at Sondrestrom since 1999.
Current work on the Data Transport Network is being funded through a grant from the NSF Information Technology Research program. The project has been extended to support data delivery and system monitoring at a number of different sites. These include meteor radars at the South Pole and an autonomous instrument platform located in northern Alaska. Some of the more novel applications include a space weather alert system and real-time instrument displays.