Search Wikia Gets Grub

By Greg McNevin

July 31, 2007: Wikia’s search project has taken another leap forward this week, with the organisation picking up LookSmart’s distributed computing software Grub and open sourcing it.

Similar to the Seti@home and Folding@home projects, grub uses distributed computing to enhance search engine indexing capabilities. When a user installs the service their machine and internet connection becomes part of a worldwide network of computing power, sorting and indexing URLs in the background or during a screensaver with the processed information being sent back to central servers.

Born out of frustration with the transparency and accuracy of big search engines, Search Wikia is Wikipedia founder Jimmy Wale’s attempt to straighten out search by making it openly accountable, making search results classifiable by users to refine results and generally use the power of the online community to build a strong contender to commercial engines.

Speaking in the US at Portland’s Open Source Convention, OSCON, Wales used his keynote address to announce the acquisition and claim that when it comes to search "we have a handful of proprietary places” that “control the vast majority of traffic on the internet”.

“Those search results are an editorial statement even if they are driven by an algorithm. We really have no idea how they are doing it. That process is kept deliberately secret,” said Wales.

Upon completion of the deal Wikia will release the both the crawler and the full Grub source code to the open source community. Grub was originally picked up by LookSmart in 2003 for US$1.4 (AU$) million, details of this new deal were not made available, however, it appears to be at least art of a larger advertising deal announced by LookSmart and Wikia last week.

Comment on this story