P2P Data Streams

Current P2P networks are designed around providing a number of discrete files – this ISO, that audio file, the other piece of video. However, what would happen if a P2P system could distribute both data and updates to that data, in a way which let you obtain the updates in a timely fashion, and be certain the updates were provided by the provider of the original file?

  • Spam blacklist providers have terrible trouble with DOS attacks from spammers. You could distribute the blacklists over this system, removing the central vulnerable point and providing anonymity of network location for the anti-spammers.
  • Browser users would appreciate access to databases of phishing URLs, but don’t want to send every URL they visit to a third party or regularly download large lists. The lists could be distributed and updated by P2P, maintaining anonymity of access.

P2P systems have proved adept at distributing content that people don’t want distributed. The obvious example is the illegal copyright infringement which takes place when popular music and movies are shared. So why not use the same system to distribute content that the bad guys don’t want distributed?

Although I have only a limited understanding of current P2P technology, I suspect some fairly deep underlying principles would need to change. For example, you’d need to be able to identify content independently of its hash. You’d also need to be able to sign content, and link pieces of content together in a defined order, based on both being signed by the same key. There are a lot of technical issues here. But it would still be very cool…

11 thoughts on “P2P Data Streams

  1. Gerv,

    I have to say that you constantly impress me (and I’m sure lots of your other readers) with the way that you are always coming up with good ideas for how to improve all sorts of things, not just web browsers. :-D

    This is a fantastic idea, if this were implemented then in theory we could use such a system to serve firefox and extension updates…


  2. There’s a problem though with the concept (of which I agree).

    Simply those who block ports, and those behind NAT firewalls. They still need the data, but often won’t return data. How do you leverage this dead weight on a P2P network?

    Many broadband ISP’s have decent download (3Mbps+), but most limit upload to 32k, 64k or 128k. This prevents spam from having such an impact (as it did in the days before download caps), and slows P2P networks.

    Everyone is capable of taking more than they are giving.

    You could use a bittorrent approach to such an effort. But the problem is still the dead weight. Are we going to say those behind NAT firewalls don’t get blacklists? Are we going to say they can’t get security updates (if distributed through p2p)?

    IMHO a distributed network of caches similar to coral cache is best. Fast downloads, and doesn’t prohibit those who have network admin’s that prohibit or limit uploading.

    Comcast is the #1 broadband ISP in the US. All their modems uplinks are capped at 128k. All their users can download at 3.5Mbps (or greater). Consider that deficit. P2P requires mutual respect for it to work well.

    IMHO a cache system could achieve greater than 128k, thus being a better system. It would bring more downloads to the users.

    What we need to do is encourage a http header for a proxy to cache a rich media object (video, audio, download), and encourage ISP’s to have such servers setup. So when you download firefox.exe, your really downloading from your ISP. In network at much faster rates.

    Problem is most ISP’s at most cache some html pages, and only the very popular ones. Most of them are pretty fast on their own. Thus making proxies rather silly.

    If they cached downloads, it would drastically reduce bandwidth the ISP needs with their uplinks, and peers.

    IMHO that’s the better approach.

  3. Also, you need to verify the data that is being submitted. If someone submits mozilla.org as an evil site, you’d obviously not want this entered into the list

  4. @Robert Accettura: Ranking systems are good. But you must be aware that especially when dealing with spammers the adversary can leverage zombie nets with ten thousands of hijacked computers to “rerank” his entries.

  5. I have heard this exact same idea mentioned by someone during the question period of Lie�s Opera lecture about websites for handheld devices at XTech 2005. Was that you, or do you just happen to have the same idea…? :)

    In any case, yeah, it sounds good. I already heard about this somewhere else, I believe, something with the PSP it was I believe. Basically the idea was to have an RSS feed from a trusted site which lists the stuff you want (e.g. in a certain category) accompanied with hashes. Which are then used to automatically retrieve those files to your harddisk, ready for you to take a look at them.


  6. I am not sure if this ideas has been mentioned before, but if this is possible then why not make websites as p2p files, granted https sites could not be done like this but a dynamic site could be done in such a manner if such an update works. Benefits of this are obvious i.e no more /. effect or a person can host their own large website without buying large amounts of bandwidth.

  7. Avalon: More Power, Less Pain

    Gervase Markham is attending XTech, an XML focused conference in Amsterdam.  He attended my…

  8. Robert: a lot of the problems you outline are general to P2P networks, not specific to this idea. Other smart minds are working on solving them; I’m happy to leave them to it. :-)

    Andr�: similarly, that’s a problem for the compilers of spam blacklists and phishing site lists. :-)

    Grauw: yes, that was me.

    poningru: websites-as-P2P has the problem that setting up a P2P connection is expensive, and websites change a lot. Also, which bits of the website would you send? Let’s use the right tool for the right job – HTTP and the web work well together.

  9. Gerv: I dont understand your first comment how is setting up a p2p connection expensive? second I said IF you had a method of updating p2p data automatically (like you suggest: However, what would happen if a P2P system could distribute both data and updates to that data, in a way which let you obtain the updates in a timely fashion, and be certain the updates were provided by the provider of the original file?) then this would be possible not with current tech. About which bits of the website to send, whatever file is large (images, sound, etc.)