Building your own Open SkyNode

Intro

This guide is designed to simply outline requirements for a full compliance open skynode in the open skyquery project, or just osq. If you're building your own skynode, it should give you an idea of what it will need to do, but will not go into any implementation specific details; exactly how you accomplish these items is up to you.

Overview

Open SkyQuery is consists of:

NVO Registry

All nodes must be registered in this registry. To view all nodes through the registry, go to advanced search and look for "serviceType like 'skynode'". Required fields for skynodes are 'ShortName', 'SkyNode Compliance' and 'ServiceUrl'.

Open SkyNodes

The meat of OSQ. Nodes are where the queries are actually run; this is where the real work happens. They are not particullarly trivial to implement, but they are not that difficult either. See this guide for more info.

Open SkyPortal

The starting point for a query is the portal. Here it is divied up organized into sorted plan and sent off to the first node. This particular portal provides asychronous execution and decent query status tracking, but these are bonus features. The only thing a portal really has to do is split up a query and ship it.

Execution Overview

After the portal has constructed an object containing node-specific queries it then sends it to the first node. However, this node does not execute its section yet, instead it passes it off to the next node and this continues until it reaches the last node. Then this last node will run its section, return its results to the previous node, and continue until it has reached the portal again. This is less confusing than it sounds (no, really), and is a nice, easy way to implement a distributed query without having to track state.

The ExecPlan

This is the actual object or document that the portal passes to the nodes. A sample is provided below.

Nodes need only care about two sections in the plan; the 'Format' and the 'PlanElements'. Format is the specified transport for this particular plan, and you should expect to receive and send results in that format. It is almost always VOTABLE, but may occasionally be DataSet. VOTABLE is the only required supported format for nodes.

The PlanElements object is an array of PlanElement objects. Each PlanElement is the relevant section for a particular node; each node need only execute its own PlanElement. A PlanElement consists of 'Statement', 'Hosts' and 'Target'. Target is the shortName (from the registry) of the intended node, and Hosts is a list of mirrors for that node (serviceURL in the registry). The PlanElements array is sorted from lowest index to highest; so the the node at PlanElements[0] would be the first to receive the plan.

Implementations

There's extensive information on building a skynode with SQLServer at SkyServer.org.
To turn this into an open skynode you just need :

Download the latest node template
extract it into the same directory.

There is also a template for the Basic SkyNode in Java. This was most recently used at the NVO Summer school and is in the summer school download. See the last part of the Server Side tutorial