Tools and Intro to P2P Collaboration

OK, after researching, experimenting, and building, I can report on collaboration via P2P.

First, to restate the goal: I want to share family histories and other files, some with just my family and some with the whole world, and do it in a way that we all share the benefits as we each work on our own copies.  Yes, I said "copies": everyone should be able to play with all their data, on their own machine, with whatever tools they choose.  For more sample uses, see familyhistories.info.

Now, a single paragraph for this year in review: after failing to create a workable plugin for one P2P program, I found that all file sharing is not equal, but it's close enough... so it seemed a good idea to make something to cooperate with most file-sharing tools/models.  Thus began P2P-Docs, a program to keep track of your file libraries as well as the other copies of similar content that other people might be working on.  You point it to the files that others are sharing with you ("incoming") and the files that you are sharing with others ("outgoing"), and it will assist you in reviewing and reconciling changes as they get passed around.  It's wrapped with a simple search program, mostly to make it attractive bait for my relatives to be able to search for stories in our shared family histories.  So it feels good to have a deliverable as well as more instructions for actual use.

I've started inviting family and friends to share files.  I hope to have some back-and-forth collaboration before RootsTech 2012.  If you'd like to test out some sharing, contact me.

So... you can see there's a bunch of items that are "not done yet" in my project README.  I want to make these tools and instructions more useful for transferring files.  I'm most excited to give my peeps good tools to search through their information; I love the idea of organizing content by assigning semantics.  It's meaningful work (ha!).  But seriously... someday I assume our computers will parse through all our content and understand it completely enough that it can find everything for us... but even then, we determine what is interesting and important, and we'll apply these values as we search for answers and/or meaning in the electronic records being gathered.  Anyway, if you're interested in any aspect of this, well, interest is what drives what I do, so let's talk.  :-)

A Private Network of Data and Events (used to share family histories, and more)

I'm very excited about the future of sharing information.  Now, there are many projects that help distribute and share your stuff, but there aren't too many that allow you to keep it all to yourself but also share it selectively if you want.  I feel that nuance of private ownership is important: it may not seem like a big deal to let Facebook and Yahoo and Flickr host your data, until they shut down services or delete your account or send your info to the FBI.  I've been motivated by this interview of Eben Moglen published by the P2P Foundation... it really opened my eyes to the possibilities if we were to take charge of our own data on our own machines.

So I'm trying to make my own private network, where my friends share my data and can get updates if they like.

The first use case is for family histories:  I've started collecting histories of my ancestors, and I'm part of an organization with a large family database.  Now, I've put up a search where someone can find histories for people in their own ancestry.  The next step is to enable a text search through those histories, and also allow people to add other histories to the collection... but I've realized that, rather than build a central internet service where people go, I'd like to make it easy for people to have their own copies and use it all in their own private network.  I plan to share history (or even personal) files such that friends will get updates easily, then allow for tagging the history files with genealogical identifiers so that people can pull the ones of interest... and maybe even just pull pieces of data, such as small bits of data that might be part of my own ancestral database.

Does this kind of stuff already exist?  If so, please send me pointers.  If not, this may deserve a name... maybe P2P with events..  P2P3.  :-)

Here are some some other projects and people I'm watching closely:

• anything about the "personal data store/locker"
• Pull by David Siegel
• VRM, information cards
• Google's Data Liberation Front
• Phil Windley, Doc Searls, Dave Winer
• Diaspora (a distributed social network)
• something Sam Curren is doing (dunno what yet)

...

(Wow.  I started writing this to share some concrete ideas, and it just devolved into a general overview with no details; I even published that stuff above, forgetting my original intent.)

Here are my requirements:

• Share files with selected people
• Share groups of files with groups of people
• Share between individuals without the need of any central server
• Notify me about changes to files, and show me just those changes
• Notify me about new files someone has added, and let me choose whether to add it to mine
• Ideally: do the same for bits of information, such as ancestor data found in a history
• Install easily on Windows, Mac, Linux

I don't envision this as a platform for all types of information; it will most likely be less convenient than Facebook and Google... but I hope to help this type of technology mature such that people find it easy and important to store some of their info this way.

So here's my somewhat-more-concrete plan:

• Find some P2P program to leverage for sharing files (AllianceP2P looks interesting)
• Build plugins or something that defines which files to share and with whom
• Manage additions, eg. when a history file is added then my network will be notified of the addition
• Manage updates, eg. when a blog file is updated then my network will be notified of the update and be able to review the change
• Manage changes to linked data (and this part could be really fun), eg. have an ancestor database that notifies me when there's a change to the dataset (RDF?) in the histories in my ancestry while ignoring changes to branches that aren't in my line

Next step:

1. I can use AllianceP2P to define my own network of people and files to share.  Use that as the transport to sync files, and write a local app to detect changes to files and allow me to see the updates.

I'd love to hear your impressions.