Chris Webb told me to try it out and so I did. I took his advice, created a Twitter account and gave IM tracking a shot. I decided to track two different phrases to start: Leopard and Twitter itself. I figure Apple's Leopard release is probably the biggest thing happening in the tech world right now so there ought to be a lot of chatter. Tracking Twitter itself simply enabled me to do some comparisons on buzz levels.
If I were to Twitter my initial reaction it would be the following: "Signal-to-noise ratio is awful...calibration options required!" (Fortunately I managed to keep the message below the 140-character ceiling.) Here, for your reading pleasure, are the first 5 IMs that came through for my "Leopard" and "Twitter" tracks:
TwitterIM (3:41:36 PM): (missusP): Freakin' grouchy today. Going tpo order a new Mac w/ Leopard - maybe that will cheer me up.
TwitterIM (3:41:52 PM): (xxdesmus): ...here goes nothing, formatting, and installing Leopard (attempt #2)...
TwitterIM (3:42:06 PM): (cyeary): Pre-Leopard backing up. The excitement builds.
TwitterIM (3:42:13 PM): (sixfoot6): Greetings from leopard! Powerbook G4 handled upgrade well. The big winner is Quicklook, but insane photomosaic screensaver a close second.
TwitterIM (3:42:25 PM): (thericketandoo): sitting on the couch with Michele. my twitter is so awesome.
See what I mean about the poor signal-to-noise ratio? I'd say maybe two out of five were (sort of) worthwhile bits of information; xxdesmus is apparently having trouble installing Leopard while sixfoot6 made it through with flying colors. But do I really have to skip over three other messages just to find two that have some value? After scanning through the last 10 minutes worth of recent track messages I'd say the 3-for-2 rate is pretty accurate, at least for these two phrases.
Despite all that, I see promise for a service like this. They need to start by adding more filtering and other calibration features. For example, I could easily live without seeing any further messages from thericketandoo, so is there any way for me to filter those message from my future feeds? To be fair, filtering individual users would require more work than I'm likely to invest in this, so how about another option?...
Is there any way to rate other users? What if I determine I really like what sixfoot6 has to say and I'd like to let others know they're likely to find value from his/her messages? Why not offer a feature where when I say "track xyz" I can also say I only want to see messages from users who average a 4-star rating out of 5? If that doesn't result in enough tweets to suit my needs, let me ratchet that setting down to 3-star ratings, but please, oh please, don't make me sift through all these other random, content-free messages!
Do customization features like this already exist in Twitter? I couldn't find them, but if I missed them, please let me know because I'd like to test it further.