The Aggregation Kool-Aid
Jeff Jarvis has 2 posts, first on future of classifieds and the 2nd more generally on powers of aggregation. I discovered jeff’s blog from Fred Wilson’s blog. Jeff has this vision of the future, where people just post their items for sale or jobs or personals profile on the ‘internet’ (perhaps a blog) and these aggregators will drive traffic to the listing, thus eliminating the need for a marketplace.
I think this buzz with aggregation is a trend. Aggregation has been around. Ebay is an aggregator except that they aggregate via feeds and provide tools for folks with no web presence to create virtual stores or individual listings. Monster is an aggregator, they aggregate by calling folks and selling job posting software.
The new generation (web 2.0) systems (crawlers, crawl friendly websites, openness in allowing crawlers) are reducing “friction” in the aggregation process and thus making it cheaper. As aggregation becomes cheaper and easier, smaller players compete on the basis of cost. If Monster requires you to pay 200$/job to list and you have to go through 10 steps to list your job, a start up can charge way less than that to list and automatically crawl your website.
Eventually, the cost of aggregation will be low, and there are several aggregators doing this. What will distinguish one aggregator over the other? What would make a newbie aggregator a google or ebay for their space? Why can’t google or ebay be the next google or ebay in these spaces?
All that aggregation solves is comprehensiveness problem. But the real factor that would distinguish one aggregator over the other is relevance. The concept of relevance can be extended really to Matching in case of some verticals. The way i explain matching is that it’s two way relevance. For a job, it’s not enough if the job is relevant to my search criteria, the candidates applying for a job have to relevant to the employer’s criteria.
I argue, that just simply aggregating solves a minor component of the problem. The true winner will solve relevance/matching problems in their space in a very effective/protected manner. To be able to beat ebay or google, these aggregators should have enough traffic of buyers (job seekers, partner seekers) and enough sellers (ok, we crawl the web and identify sellers) and have proper matching (two-way relevence).
Matching is really the killer problem to solve. The perfect jobs site will always send me atleast 1 candidate that would hire for the job and always produce the right job i had in mind when i was looking for one.
Aggregation is old news
April 13th, 2005 at 12:08 pm
Looking at aggregation more generally, there is another aggregation component that is not quite a solved problem: effective aggregation of user profile/preference information.
So on one side you aggregate content, and on the other side you aggregate information about the user, and then relevance is matching between the two. For some applications (you mention Personals), the content is the user information. For others - web search - this information tends to be more overlooked because understanding and drawing unambiguous conclusions from it can be hard.
Another way to think about it is that the the text portion of a search engine query is only one of potentially many inputs to finding the best matches - perhaps even the least important! If we know everything about a user, shouldn’t we be able to anticipate what they’re searching for? Maybe there is no need for the query page, only a results page
Aggregating content, as you point out, is becoming easy.
With the now-ubiquitous privacy policies most reputable sites have adopted, aggregating user information is hard if you don’t already have a lot of users coming to your site and doing things/telling you things about themselves (passively or actively). So there’s a bootstrap problem, and in my opinion this is why the most successful aggregators today might continue to be some of the most successful aggregators tomorrow: they already know alot about what their users are doing, they just need to relate it to the mass of content they aggregate (as well as to other users’ information/preferences). Amazon is the perennial example of an aggregator doing this reasonably effectively. Their next move is to bring communities into the picture (sound familiar?). “Amazon book clubs” anyone?
May 6th, 2005 at 8:25 am
With their Web Accelerator, it appears Google has elected to solve the personal content aggregation bootstrap problem in a decidedly 1999-esque way (which in turn is a 1984-esque way).
What does a big proxy cache buy the user mean in an increasingly personalized and continuously refreshed web? Not too much.
What does a big proxy cache buy Google in an increasingly personalized and continuously refreshed web? A lot better ad targeting.