The STASH Project has been superceded by the work of the Free Network Project.
The Free Network Project has a more elegant approach to the same problem. The source code developed here in pursuit of
the same goals is being rolled into other projects, available here at AndTheHorseYouRodeInOn.com.
the Secured Treasury And Shared Hoard (STASH)
Background
One lesson of History states that control over information is tantamount to control over people. The free flow of information and ideas is antithetical to authoritarianism.
This lesson is well understood not only by those who would share information with the world, but also to those who would suppress it. Organizations, Corporations, and Governments often attemp to control information in order to strengthen their own positions at the expense of others. These entities frequently resort to strong-arm tactics to suppress information; often these actions are contrary to the interests of the community. Any student of history could cite examples of cover-ups, censorship, and ideological suppression that have caused unnecessary suffering and loss.
The Secured Treasury And Shared Hoard is an attempt at a technological solution to a social problem. It endeavors to make information, however unpopular, available to people without regard to ideology, politics, or dogma.
It is acknowledged that this is a dangerous undertaking. We all have our sacred cows, and we all have our secrets. But one person's sedition is another person's free speech; one person's faith is another's blasphemy. By maintaining a neutral stance, the STASH will serve as a repository for information that is threatened.
Goals
- Create an Open Source application that implements the STASH.
- Preserve access to information, regardless of the climate surrounding that
information.
- Protect the individuals who harbor the information.
- Protect the integrity of the information.
These goals will be achieved by splitting up documents into small, unoffensive pieces which are then distributed across a cloud of independent nodes. Each node may know about it's neighboring nodes, but may or may not know about distant nodes. Document pieces are replicated, and spread as widely as possible.
Rules
- The STASH is a collection of Nodes. Nodes are processes running on network connected servers.
- Information will be organized into Documents. Documents may be any kind of
digital file, in any format.
- Documents must be replicated across multiple Nodes.
- Documents must be broken into multiple Pieces. A Piece is defined as a "stripe" of the document e.g., every nth byte of the Document).
-
No one Node may contain all the Pieces to assemble any given Document. Typical Nodes will not contain all of the Pieces to assemble a given Document.
- Documents may not be deleted from the STASH.
- Newer versions of Documents may be added to the STASH (without replacing the older versions).
Assumptions
- Nodes may join the STASH at any time.
- Nodes are not reliable, and may leave the STASH at any time.
- Attackers may attempt use hostile Nodes to flood the STASH with bogus data.
- Node Operators may or may not know the contents of their own Node.
- New Nodes know of at least one other Node.
- People submitting Documents know of at least one Node.
- People retrieving Documents know of at least one Node.
- Retrieval may not be instantaneous.
How it Works
Node Startup
- When a Node starts up, it queries the known node for a list of (all/some/a variable number of) other known Nodes.
- There also needs to be a mechanism for sniffing out other Nodes in case the known Node is unavailable. This gets convoluted!
- Node decides on its Modulus. It should do this based upon the least common Modulus in the set of known Nodes. The Modulus is the "stripe" offset, and should range from 0 to 4. Fo
r example, a Node with a Modulus of 3 contains the byte 3, byte 8, byte 12, byte 17, etc of a f
ile. If a Node has a Modulus of -1, that Node is in Brave Mode, and contains the entire Documen
t.
- The new Node then queries all listed Nodes for a list of Documents that they are aware of. Documents are identified by a unique identfier along with a text name and text synposis.
- The new Node then populates its internal tables:
- For each Document in the list, the new Node queries each known Node for the list of Pieces it is storing.
- The new Node then requests Pieces from the Nodes according to certain Priorities.
- Node checks Piece's signature for validity (TBD: how do nodes get public keys? Link to known Keyservers?) Discards Piece if signature is invalid, and downgrades internal trust rating for Node that served questionable Piece.
- Node periodically polls known Node list for new Documents. When new Documents are added to the list, they are added to the Node database as described in previous step.
Submitting a Document
- Author submits a full Document to a Publisher (a trusted Node).
- Publisher queries Author for private key.
- Publisher signs document using Author's private key.
- Publisher breaks document into Pieces.
- Publisher signs individual Pieces with Author's private key.
- Publisher maintains all Pieces until they have been requested by at least two other Nodes.TBD: Is there a good way to do this where the entire Document never resides on the Publisher at once?
- Publisher removes Pieces that do not match its Modulus.
- There may be a better way for publishing Documents to Nodes that are in Brave mode
Requesting a Document
- User requests Document from Node using Client Program.
- Node returns all Pieces of the Document that it has in its database, and a list of known Nodes.
B
- The Client then queries each known Node for the list of Pieces it is storing.
- Client requests Pieces from the Nodes according to Node trustworthiness. Determined by?
- Client checks Piece's signature for validity (TBD: how does Client get public keys?
Link to known Keyservers?) Discards Piece if signature is invalid, and downgrades internal
trust rating for Node that served questionable Piece.
- If not all pieces are found, Client sequentially queries each Node for its list of known Nodes, and repeats.
- Client assembles Pieces into full Document.
Other Considerations
- A Node may become Full. Full Nodes no longer poll other Nodes for new Documents. They continue to serve all other Node functions.
- TBD: Determining Node trustworthiness.
- Nodes must intelligently manage their lists of known Nodes. If a Node becomes unavailable, it must be removed from the list. If a Node is contacted by a hitherto unknown Node, it must be added to the list.
- How do we prevent hostile Nodes from overwhelming the storage space?
- This system is not guaranteed to have all pieces of all documents, especially if number of Nodes is low, or number of Full Nodes is high. How do we fix this?
- Node attempts to collect the rarest Piece (least number of copies on other Nodes).
- Node collects Pieces with IDs that fits their Modulus.
- Node collects Pieces according to other Node's trustworthiness.
STASH Server Test Implementation
I'm woring on a test implementation in Java. I'm about 10% into it. As soon as I have something to post, I'll put the javadoc here and the source here.
STASH Server Functionality: Commands
My latest thinking is the STASH server should talk HTTP, so as to made portable through more filters and firewalls. The commands below will need to be updated and adapted.
|
get ip [known Node ip]
|
Gets a list of all Node ip addresses known by the known Node.
|
|
get doclist [known Node ip]
|
Gets a list of all Documents (by name and ID) for which
known Node has Pieces.
|
|
query docpieces [known Node ip] [Document ID]
|
Asks known Node if it has any pieces of Document ID. (returns a
list of Piece IDs.
|
|
get piece [known Node ip] [Document ID] [Piece ID]
|
Gets a Document Piece from
known Node.
|
References
- PGP Signing/Validation Perl library http://www.perl.com/CPAN-local/modules/by-module/PGP/
- GNUPg. Free Privacy code. http://www.gnupg.org/
-
Initial STASH document by Samuel Goldstein (samuelg@andthehorseyourodeinon.com)
Contributors:
Latest Update: 04 February 2000
This site designed for IE or Netscape versions 3.0 or bigger...
Copyright © 1999-2000, Patsy Butts, Nick Verstehen, and Contributors
(contact us)