RSS and Atom feeds are not appropriate schemas for a new distributed Twitter.

RSS and Atom feeds will not a distributed Twitter make. Not because they aren’t “real-time”, but because the notion of “titles” simply isn’t conversational.

The schema of the feeds dictates to a large degree the end-user experience. Twitter’s 140 character limit has surprised, impressed, and enlightened the lot of us techies. How could such a severe limitation actually works to the end user’s benefit? Easy: it makes a tweet stream readable. If you want to publish into the conversation, you have to conform to the standards and keep it short; people are skimming this information.

Compare a Twitter.com client to an Atom feed client. Atom clients have trouble displaying information from disparate sources uniformly because not every source agrees on the semantics of “title” vs “subtitle”, nor on what to put in the “summary”, nor on whether the content embedded in the feed should be the same or different from that in the link. Re-combined Atom feed items can make for some awkward reading when you start pulling in things like photo streams, audio feeds, and Twitter statuses. Just because some information can be shoved into Atom and shown in the same feed reader as everything else doesn’t mean it should be; however, the idea that all types of Atom feeds need to be supported in a feed reader is prevalent, and has usually worked to the detriment of user experience. Even RSS’s more simple “title” and “description” are two pieces of metadata too many for today’s information glut and the new distributed world-wide conversation.

One way to get clients built for a new distributed, real-time, world-wide conversation system might be to make a new feed format that helps dictate the user experience and expectations. In doing so, we would take a lesson from Twitter: Titles are outmoded. Make your case in 140 chars and attach a link. It works.

How about a straw-man proposal then? We’ll need some some short text, the ability to attach links, and some very basic metadata to make the system work. Call these new feed items “Tweets” (and make sure they can be JSON as well as XML!).

NEW TWEET SCHEMA STRAW-MAN for distributed world-wide conversation:

  • Body. 140 chars!
  • Links. If you are tweeting a link or links to data (e.g. an article, photo, mp3, etc.) those links go here. The items are NOT embedded in the Tweets. (Another lesson learned from Twitter and another interesting discussion.)
  • Author. A person’s internet identity. I-Names?
  • Digital Signature. The whole Tweet should be signed against the author’s public key so we can verify authenticity without having to get the tweet directly from the source.
  • Datestamp.
  • Universally Unique ID. When you make a tweet, you generate a UUID for it (in ultimate accordance with a registry.) Tweets don’t live in any central location, and aren’t created through a central place, so it’s not a URL.
  • Context. If this is in reply to another tweet, or tweets, put their unique IDs here to so clients can load conversation context.

This would be the format of the Tweet itself, independent of any message routing systems. Separate metadata in routing envelopes would help get the right tweets to the right people at the right time, but those would be routing-system specific, and a separate discussion. (“Re-tweeting”, for example, should really just be re-routing, not creating a new Tweet.)

What else would you add? What’s the best counter-argument that this should be done with Atom instead?

6 Responses to “RSS and Atom feeds are not appropriate schemas for a new distributed Twitter.”

  1. Rogers Cadenhead Says:

    An RSS item does not require both a title and a description. It must contain either one or the other:

    http://www.rssboard.org/rss-profile#element-channel-item

    So it would be easy to represent Twitter updates in RSS.

  2. Mason Lee Says:

    Thanks for the correction! Yes, only in Atom is “title” required. And for that matter, “tweets” could just get crammed into the “title” elements of Atom, and Twitter could just be an Atom stream of “titles”.

    It’s true that you can pretty much force any list of items into RSS (e.g. airplane departures, library books checked out, new employees added, etc.), however, the best reason for using RSS/Atom is when you want to view these things in a news reader, combined with other article-type items.

    Would we really to want to read Tweets mixed with New York Times articles? How about our instant message history? Because we could make into that RSS, too.

    You can’t get Twitter with RSS or Atom, because you lose the 140 character constraint, and you won’t have any way to tell which feeds are meant for a news reader apps and which are meant for Twitter style apps. What you’ll get is a big crappy user experience like today’s RSS readers full of disparate streams, desperate to reuse a format that wasn’t designed for them.

    Pull Plaxo Pulse into your news reader if you want to see what I mean– it just doesn’t fit right. Twitter is a different concept as well. Not to be mixed.

  3. Rogers Cadenhead Says:

    RSS readers didn’t exist when the format was invented. The original use was for one site to run headlines from another site. I don’t think it matters what the “best reason” for the format is. It’s content neutral — any kind of updates can be delivered over it.

  4. masonlee Says:

    And since any kind of updates go over it, no one agrees on the right length for titles or descriptions, or the presence of one or the other of these, and you get a big ugly mashup when you try to combine feeds. You can get RSS feeds of twitter streams today and they don’t fit well along side NYTimes’ feed.

    How could we mark some RSS feeds as being “Twitter-like” vs others being “blog/newspaper like”?

  5. Greg Billock Says:

    I don’t understand the point about twitter feeds not looking right alongside news headlines. Isn’t that precisely what the very-successful @cnnbrk looks like? Most content there just headlines and links — essentially what you would put in your ‘body’ and ‘links’ section, and what in an rss/atom feed would go in ‘title’ and ‘link’ (and perhaps ‘description’ as either the full text or the first paragraph of the link content.

    As to a rule for how to differentiate in the UI, wouldn’t feed elements with no description (that is, only a title and an optional link) fit naturally into this? Or is the vice versa success of @cnnbrk not indicative?

    • Mason Lee Says:

      Hi Greg.

      I didn’t express that particularly well. When I said NYTimes feed, I had in mind to exemplify what I consider “proper” or “traditional” RSS, where the title element contains a formal news article title in title case, the description element contains a one or two sentence summary, and the link takes you to the article. A different sort of presentation is required for that than for today’s all-in-140 character “tweet” from Twitter.com. My preferred Twitter client, Tweetie for Mac, excels at displaying streams from Twitter, but would have to be changed significantly to also show traditional RSS. My preferred news reader, NetNewsWire excels at traditional RSS, but looks pretty bad when you pull in Twitter RSS or any other non-traditional RSS.

      It would be interesting to make a more intelligent RSS reader that was better at detecting, displaying, and combining different style feeds. Your proposed rule “is there a description?” would be a good start. It would also be nice also to 1.) not show descriptions that are the same as the title, 2.) automatically check the linked content to see if the description element contains a real description or just the linked content itself, 3.) intuit photo streams and podcasts. Each requires a different display technique, and not all combine into one list well. I like having my podcasts, photo feeds, tweets, email, and traditional RSS items in separate “apps” so that my various “lists” exhibit some consistency.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: