Serialization

[kddnewton]

We want a binary serialization of the syntax tree such that it can be deserialized and read in any other language. This issue is to begin looking at how that would work and what the structure would be. I'm imagining something to the effect of the below design.

First, the header:

# bytes	field
4	YARP
1	major version
1	minor version
1	patch version
8	comments offset into file

Next, the tree. For each node, it'll walk the tree starting from the top. It will dump:

# bytes	field
4	type
8	offset of next node in tree*
8	start offset in source
8	end offset in source

Then for each child node from this node it will recurse. Each child node type will have to have its own kind of serialization. I'll discuss those next.

*I think this is a good idea to include so that folks can skip past this node if it's trivial for the implementation. For example, if it's a Self node and you don't care about offset in the source then you can just skip directly to the next node in the tree.

For a child node that is itself a node, it will use the schema defined above. For a child node that is a string, it will use:

# bytes	field
8	start offset
8	end offset

For a child that is a node list, it will use:

# bytes	field
4	number of elements in the list

and then for each node it will recurse.

Finally, at the end it will have a section for comments, which will look like:

# bytes	field
4	number of comments in the file

then for each comment:

# bytes	field
8	start offset
8	end offset

I believe this is enough information to get started. Certainly we're going to iterate on this.