FeedMesh: The Secret Behind RSS Search Engines

In a conversation with Greg Gershman of blogdigger, he mentioned Feedmesh. I had never heard of it. Apparently, it is:

Temporary working group to discuss Feed Mesh technology for weblog ping aggregation and peering.

Bob Wyman has more on his weblog:

Hopefully, FeedMesh will soon be seen as a regular and established component of a strengthened and more open blogging infrastructure. FeedMesh will allow a wide variety of new services to rapidly get into the stream of updates. New and established services will be able to focus more on building better and more distinctive services rather than the dirty and duplicative work of tracking down updates. Given the ethics of openness in the blogging and syndication movement, it seems right that we should all compete not based on which or how many updates we know about but rather on the quality of the services we provide. Industry efforts like FeedMesh make it possible for us to achieve that goal and should result in better services being provided to our users.

Very interesting stuff. In the bit that we've looked at aggregating event data, the challenge has been: how do we know when something is added to a feed or when an entry is updated. This looks like it is solving that problem with blog posts?  Can anyone that knows a bit more tell me whether this solves the problem for other microcontent?

Comments

[Bob Wyman]

You wrote: "Can anyone ... tell me whether this solves the problem for other microcontent?"

FeedMesh would "solve" the problem for any kind of microcontent contained in the feeds or sources that are currently being monitored. At present, that is just RSS and Atom feeds. However, I expect that once more people see the utility of the FeedMesh -- of having service providers share their knowledge of what is being published -- that we'll see efforts to expand the scope of FeedMesh to other areas as well.

I expect that many different types of microcontent will, in fact, be syndicated or distributed via RSS and Atom feeds. Take a look at the "structured blogging" proposal we've made at http://structuredblogging.org and you'll see that we describe a format that allows structured and typed microcontent to be embedded in HTML, XHTML, RSS, Atom, etc. Given Structured Blogging, if you publish microcontent, as soon as any of the FeedMesh members discover your entry (either because of a ping or because they crawled your feed) all of the other FeedMesh members would be immediately notified of it. This is significant...

In the future, I expect that FeedMesh will be used for a broader set of sources. Also, it is clear that there is demand among FeedMesh members to exchange the full content of discovered posts -- not just notifications of their locations. Once we start passing full content, it will mean more rapid propogation of data in the network as well as reduced bandwidth load on the publishing sites.

bob wyman