11 July 2010

Consuming news content in a paywalled world

In today's Observer David Mitchell published an Op-Ed piece about Murdoch's decision to charge for access to the Time website. A number of the comments discussed the idea of paying for professional content on the web. I was moved to post the below:

I think the 'paywall' on a single provider is a wrong turn. About 20 years ago sci-fi author Harry Harrison and robotics/AI researcher Marvin Minsky suggested something I think is much more likely in their book "The Turing Option". The book is an Industrial Espionage/action/thriller story but quite early in the book they talk about a way we can consume news.

What they describe is that rather than buying a newspaper or magazine you subscribe to a service that scours the various media out there on the net (bear in mind that this was written well before the web really came about, the 'Internet' was basically USENET, email and FTP). You would tell the service the sort of things you are interested, this would be geographical areas, your work, films you like, books you like, sports you're interested, teams you follow &c. It would then identify the sorts of articles you'd be interested in and present them to you. It would also track which articles you actually read, which you re-read, which you saved for future reference &c then use this information to fine tune it's selections in future. It may also factor in things like reputation of the uthor and who else reads them. Over time, probably quite quickly, the articles presented to you would get closer and closer to what you want to read. You would pay the service a regular fixed subscription and they in turn would pay the content providers based on items used.

Given developments of the web such as Twitter, Facebook, blogs, Digg &c, I suspect that the service would also offer you the option to blog/tweet/post/comment about the article you just read then use that to refine future choices. It may even analyse your comments to judge how strong your interest is, did you just post the link with maybe a short "interesting article." comment or did you write an essay analysing the article, referencing other articles and including quotes? Some analysis of who you read may also be done, if you always read David Mitchell's articles on some subjects you're interested in then maybe you might be interested in his articles on a subject you're not usually that interested in.

We're already seeing some of this in Google Alerts and Amazon Recommendations. Google Alerts falls short due to the fact that it relies on you to create the search string, there's no analysis going on, it's just giving you the top few results if you ran this search now that are new since since the last time it sent the results to you. The results improve over time but due to the general improvement of the Google search algorithm rather than your individual choices. Amazon falls down on two things, one minor and one major. The minor one is that simply their analysis isn't up to it yet because their data volume is too small. They're just looking at what books/DVDs/CDs you've bought, which may include ones you bought as presents so they're what your great aunt Gladys is interested in not you, and most people don't buy enough to generate a meaningful data set, although you can improve the recommendations immensely by when you look at the recommendations page actually clicking the "I've already got this" or "Not interested" buttons. The major cause of failure is more fundamental and cannot be solved with technology, it's run by the marketing department and they want to sell you stuff. Because they want to sell you stuff and the cost of a bad recommendation is the same as no recommendation (you don't buy the product), they will use the most tenuous of links to make a recommendation. With a subscription service you are making multiple choices each day as to what to read and what not, and then add in things like twitter and digg to re-enforce the choices (maybe even have a "Not Interested" button), so a sizable data set will quickly build up. As it is a subscription model there is no incentive to make bad recommendations, if you read an article you're not interested in they don't make any extra money and it may cost them more as they now have to pay the content provider. If they make a lot of bad recommendations then you may leave them and go to a different service. Their business model, their profits, rely on you choosing to read a small number of articles you are very interested in every day so you'll keep paying your subscription but they won't have to pay the content providers too much.

This would also help the small content providers, who cannot afford to set up a micropayments system, as they can just license their content to the subscription services and get the aggregated payments.

I do think that something like that, where you don't pay the content provider directly but pay the aggregator who the licenses the material from the content provider, is a distinct possibility. Something like the iPad/Tablet-PC is probably the ideal platform and set top devices coupled to a 'reusable ePaper' printer would be good, but smart phones and more traditional PC/Laptop platforms would also work well.