Tuesday, January 18, 2011

A Private Network of Data and Events (used to share family histories, and more)

I'm very excited about the future of sharing information.  Now, there are many projects that help distribute and share your stuff, but there aren't too many that allow you to keep it all to yourself but also share it selectively if you want.  I feel that nuance of private ownership is important: it may not seem like a big deal to let Facebook and Yahoo and Flickr host your data, until they shut down services or delete your account or send your info to the FBI.  I've been motivated by this interview of Eben Moglen published by the P2P Foundation... it really opened my eyes to the possibilities if we were to take charge of our own data on our own machines.

So I'm trying to make my own private network, where my friends share my data and can get updates if they like.

The first use case is for family histories:  I've started collecting histories of my ancestors, and I'm part of an organization with a large family database.  Now, I've put up a search where someone can find histories for people in their own ancestry.  The next step is to enable a text search through those histories, and also allow people to add other histories to the collection... but I've realized that, rather than build a central internet service where people go, I'd like to make it easy for people to have their own copies and use it all in their own private network.  I plan to share history (or even personal) files such that friends will get updates easily, then allow for tagging the history files with genealogical identifiers so that people can pull the ones of interest... and maybe even just pull pieces of data, such as small bits of data that might be part of my own ancestral database.

Does this kind of stuff already exist?  If so, please send me pointers.  If not, this may deserve a name... maybe P2P with events..  P2P3.  :-)

Here are some some other projects and people I'm watching closely:

(Wow.  I started writing this to share some concrete ideas, and it just devolved into a general overview with no details; I even published that stuff above, forgetting my original intent.)

Here are my requirements:
  • Share files with selected people
  • Share groups of files with groups of people
  • Share between individuals without the need of any central server
  • Notify me about changes to files, and show me just those changes
  • Notify me about new files someone has added, and let me choose whether to add it to mine
  • Ideally: do the same for bits of information, such as ancestor data found in a history
  • Install easily on Windows, Mac, Linux
I don't envision this as a platform for all types of information; it will most likely be less convenient than Facebook and Google... but I hope to help this type of technology mature such that people find it easy and important to store some of their info this way.

So here's my somewhat-more-concrete plan:
  • Find some P2P program to leverage for sharing files (AllianceP2P looks interesting)
  • Build plugins or something that defines which files to share and with whom
  • Manage additions, eg. when a history file is added then my network will be notified of the addition
  • Manage updates, eg. when a blog file is updated then my network will be notified of the update and be able to review the change
  • Manage changes to linked data (and this part could be really fun), eg. have an ancestor database that notifies me when there's a change to the dataset (RDF?) in the histories in my ancestry while ignoring changes to branches that aren't in my line
Next step:
  1. I can use AllianceP2P to define my own network of people and files to share.  Use that as the transport to sync files, and write a local app to detect changes to files and allow me to see the updates.
I'd love to hear your impressions.