Benchmark Study of Desktop Search Tools

pages

English

Documents

Lire

Le téléchargement nécessite un accès à la bibliothèque YouScribe Tout savoir sur nos offres

pages

English

Documents

Lire

Le téléchargement nécessite un accès à la bibliothèque YouScribe Tout savoir sur nos offres

Publié par

Nozur

Nombre de lectures

145

Langue

English

Poids de l'ouvrage

2 Mo

April 20, 2005
Benchmark Study of
Desktop Search Tools
There’s More to Search than Google & Yahoo!
An Evaluation of 12 Leading Desktop Search Tools
Tom Noda
Shawn Helwig
www.uwebc.org/decisiontoolsDecision Tools | Desktop Search 2
Ex Ecu Tiv E Summary
A new generation of desktop search tools is emerging that allows users to quickly find relevant documents in computers
across the enterprise the same way search engines help locate information on the Internet. Companies expect that this
technology will boost employee productivity and creativity and allow them to compete successfully in today’s knowledge-
driven economy.
Desktop search technology itself is nothing new. In fact, it has been around for years. However, some well know names
(i.e. Google and Yahoo!) have recently entered the space giving this technology a well-deserved boost in visibility. In an
effort to help understand the differences between the latest desktop search tools on the market, the UW E-Business Con-
sortium recently conducted a benchmark study of 12 popular desktop search tools. The benchmark criteria that were used
for the evaluation included usability, versatility, accuracy, efficiency, security, and enterprise readiness.
When all the results were reviewed, it was determined that most of the desktop search tools were still too immature for
significant business use due primarily to a lack of mature security and overall manageability. However, considering the
evolution of Instant Messaging ...

Voir

Publié par

Nozur

Langue

English

Poids de l'ouvrage

2 Mo

April 20, 2005Benchmark Study ofDesktop Search ToolsTheres More to Search than Google & Yahoo!An Evaluation of 12 Leading Desktop Search ToolsTom NodaShawn Helwigwww.uwebc.org/decisiontools

Decision Tools | Desktop Search2ExEcuTivE SummaryA new generation of desktop search tools is emerging that allows users to quickly ind relevant documents in computers across the enterprise the same way search engines help locate information on the Internet. Companies expect that this technology will boost employee productivity and creativity and allow them to compete successfully in today’s knowledge-driven economy.Desktop search technology itself is nothing new. In fact, it has been around for years. However, some well know names (i.e. Google and Yahoo!) have recently entered the space giving this technology a well-deserved boost in visibility. In an effort to help understand the differences between the latest desktop search tools on the market, the UW E-Business Con-sortium recently conducted a benchmark study of 12 popular desktop search tools. The benchmark criteria that were used for the evaluation included usability, versatility, accuracy, eficiency, security, and enterprise readiness.When all the results were reviewed, it was determined that most of the desktop search tools were still too immature for signiicant business use due primarily to a lack of mature security and overall manageability. However, considering the evolution of Instant Messaging from a pure consumer tool to a valuable enterprise application, desktop search may have similar potential.KEy FinDingSTOP 3 DESKTOP SEarchBased on our evaluation, the best overall desktop search tool is Copernic 1.5 UsabilityBeta with Coveo.REentaedripnreissesVersatilityYahoo! Desktop Search 1.1 Beta was rated the second best tool in our evalu-ation. See other notes.Likasoft Archivarius 3000 came in a surprisingly close third in our evaluation. SecurityAccuracyThis software is available commercially from Likasoft. Archivarius’ index efi-ciency is outstanding and was the clear winner in our tests. The user inter-Eficiencyface and navigation scheme is well designed and easy to use.1. copernic 1.5 Beta with coveo2. yahoo! Desktop Search 1.1 Beta3. likasoft archivarius 3000Source: UW E-Business ConsortiumTaBlE OF cOnTEnTS2Executive Summary3Overall ratings3Benchmark criteria4criteria ratings4Product reviews11appendix a12appendix Bwww.uwebc.orgBEnchmarK nOTESThe benchmark evaluation testing was performed in March, 2005. This research was not funded or supported by any speciic companies or institutions. The benchmark evaluations were conducted solely by the UW E-Business Consortium.© 2005. UW E-Business Consortium

Decision Tools | Desktop Search3OvErall raTingSThese are the overall benchmark evaluation ratings. Some tools are very good in speciic areas such as usability, versatil-ity or search accuracy (explained later), but to be the best desktop search tool, a balance of all criteria is critical. Desktop Search ToolVersion Score (Min = 1.00, Max = 5.00)Bettercopernic Desktop Search1.5 Beta4.11yahoo! Desktop Search1.1 Beta3.66likasoft archivarius 30003.143.62mSn Toolbar Suite2.0 Beta3.45google Desktop1.03.26ask Jeeves1.0 Beta3.16Enish Professional6.13.10iSyS Desktop6.03.05dtSearch Desktop6.53.02diskmETa Pro1.0.1Blinkx3.0hotBot DesktopBeta36.236.243.2Source: UW E-Business ConsortiumBEnchmarK criTEriaOur benchmark evaluation was performed across six main criteria. Each criterion was quantiied and was given a rating, ranging from 1 (worst) to 5 (best). The rating is based on sub criteria, which align with the main criterion’s objective. For sub criteria and their rating details, please refer to Appendix A - Comparison Table.1. usability2. versatilityGood desktop search tools must be easy to use, have Versatility describes how wide and deep the tool al-a lower learning curve, have professional aesthetics, lows you to search. This includes factors such as sup-and require fewer steps to reach desired output.ported document types, web/e-mail integration, and multi-language support.w3. accuracy“Can you ind what you are looking for?” This criterion addresses accuracy of search results as well as otherfactors that help users ind the desired information.5. SecuritySecurity and privacy are big concerns, especially in an enterprise environment. This criterion considers how well vendors have incorporated security mechanisms.wwu.ewcbo.gr4. EficiencyThis criterion assesses the tool’s technical eficiency including memory usage, indexing time or indexed ile sizes. The best tool should not jeopardize overall PC performance.6. Enterprise readinessWhile most tools are designed for the consumer/home PC environment, some are ready to be used in an enterprise. This criterion may be especially helpful for IT managers. ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search4criTEria raTingSThe following charts summarize the best tools’ ratings for each criterion. Blinkx and ISYS are versatile tools but struggle to deliver their powerful features in a user-friendly fashion. On the other hand, Ask Jeeves excels in usability, eficiency and security, but lacks versatility. Copernic is excellent in almost all criteria. 1. Usability 2. VersatilityCopernic4.80CopernicArchivarius4.75Yahoo!Google4.40BlinkxMSN4.40ISYSAsk Jeeves4.25 3. AccuracyCopernicNSMdtSearch 5. SecurityYahoo!Ask JeevesGoogle05.392.31.3431.305.402.4 4. EficiencyArchivariusCopernicAsk Jeeves41.488.357.357.304.402.408.3 6. Enterprise ReadinessCopernic4.00ISYS4.00Yahoo!4.00* Copernic with Coveo, and Yahoo! with X1Source: UW E-Business ConsortiumPrODucT rEviEwSThis section examines the details for each desktop search tool individually. The benchmark performance for each tool is expressed with a Spider Chart (see description), in order to convey the performance in each one of six criteria as well as the overall balance.UsabilitySpider charts05.1Spider Charts have been used to show how each criterion was scored, as well REentaedripnreissesVe5r.s0at0ilityas the overall balance. For instance, the chart at left indicates that this tool is 3.00extremely good at Versatility but needs some improvement in Usability and Ef-iciency.SecurityAccuracy3.503.00Achieving the maximum scores in all criteria and maintaining a good hexagon shape are ideal, but that is not required by all users. For instance, if Enterprise EficiencyReadiness is not critical for a speciic user, an unbalanced shape that lacks 2.00Enterprise Readiness features may still be a solid it.ww.wwube.crog ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search5copernic Desktop Search 1.5 Betawww.copernic.comUsabilityCopernic is the most well-balanced desktop search tool among those evaluated. 08.4The tool is intuitive and easy to use. The new beta version supports FireFox for EnterpriseVersatilityReadiness4.14Web history search, and Thunderbird and Eudora for e-mail (as well as IE, Out-00.4look and Outlook Express) “Search as you type” and “dynamic indexing,” which 4.11detects new and modiied iles/e-mails on the ly, are useful features. Filtering, SecurityAccuracysorting and grouping search results are well reined. The application has a small 3.004.50technology footprint and pro-Eficiencyvides detailed index controls. 4.20One potential improvement, Source: UW E-Business Consortiumhowever, is that it should let users choose a default web search engine. It only supports the all-theweb.com” web search within the application, which is not as popular as Google or Yahoo!For business use, Coveo, a spin-off company from Copernic, provides enterprise desktop search products, which enhance security, manage-ability and network capability. The client applications are identical. However, the enterprise version works with additional server products Copernic Desktop Search 1.5 Betasuch as Microsoft SharePoint.yahoo! Desktop Search 1.1 Betadesktop.yahoo.comUsabilityYahoo! Desktop Search is based on X1 Desktop Search, so usability will be 00.4familiar to existing X1 users. Yahoo! integrates X1’s technology into its own portal EnterpriseVersatilityReadiness3.88services such as Yahoo! E-mail and Instant Messaging. It can index Yahoo! IM 00.4logs as well as Yahoo! Address Book. Versatility is excellent, as Yahoo! claims 3.66their tool supports more than 200 types of documents. It indexes Adobe Pho-SecurityAccuracytoshop and Illustrator 3.293.20iles in addition to many Eficiencymedia iles. Contents of 06.3Source: UW E-Business Consortiumzip iles are examined and displayed in a tree structure. The tool’s preview feature is well reined, but its search results are somewhat clumsy because too many columns are displayed in a vertical view. As opposed to Copernic, there is no dynamic indexing or web history search.X1 offers an enterprise version of the desktop search tool as a server-based product. IT managers may want to check it out.Yahoo! Desktop Search 1.1 Betawww.uwebc.org© 2005. UW E-Business Consortium

Decision Tools | Desktop Search6likasoft archivarius 3000 3.14www.likasoft.comUsabilityIn contrast to Copernic or Yahoo!, this is a commercial product, which costs 57.4EnterpriseVersatilityfrom $25 to $45, depending on your status and purpose of use. Usability and Readiness3.38eficiency are astonishing. It is a very simple GUI design, yet organizes a lot of 00.3features compactly. It demonstrated the fastest initial indexing time among all 3.62products, and memory usage was relatively low in idle time. It does not support SecurityAccuracyany media ile index-3.003.20ing (image, audio or Eficiencyvideo) or web/web 4.40history searches. On Source: UW E-Business Consortiumthe other hand, e-mail support is wide, ranging from Outlook and Outlook Express to Eudora, Thunderbird and Lotus Notes/Domino. One unique feature is that it offers remote search functionality. The ap-plication acts as a small Web server, allowing remote users to search the computer through a web browser. Of course, it has user/group account management capability built-in.Likasoft Archivarius 3000 3.14mSn Toolbar Suite 2.0 Betatoolbar.msn.comUsabilityMSN Toolbar Suite is similar to Google Desktop, and has almost the identical 04.4EnterpriseVersatilityfunctionality and navigation scheme. Moreover, it includes a Popup Blocker and Readiness3.63“Auto Form Fill” features which are already available with the Google Toolbar. 00.2MSN performed well in terms of search accuracy. Word accuracy was very good 3.45in our test. “Shortcut keyword” is a unique feature, which lets users associate SecurityAccuracya keyword with speciic 2.864.20iles. With this association, Eficiencyusers can type a keyword 3.60in Windows Explorer’s Source: UW E-Business Consortiumaddress bar to fetch a ile, instead of crawling multiple folders. One challenge is that it does not support PDF iles by default. To index PDF contents, users must download and install an add-in tool called “IFilter.”ww.wwube.crogMSN Toolbar Suite 2.0 Beta ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search7google Desktop 1.0desktop.google.comUsabilityGoogle seamlessly integrates desktop search into its popular web search en-04.4EnterpriseVersatilitygine. The browser-based desktop search tool is easy to use and will be familiar Readiness3.50to anyone who has used Google. A loating bar is a unique feature and allows 00.2users to type keywords from anywhere on the screen. Google Desktop API is 3.26another promising feature, allowing software developers to develop add-ins to SecurityAccuracyenhance the tool’s func-3.133.20tionality. OpenOfice and EficiencyICQ index add-ins are 3.33already available. Unfortu-Source: UW E-Business Consortiumnately, iltering and sorting functions are quite limited. It appears as if Google is so focused on its relevance algorithm that other sorting functions seem to be ignored.Google Desktop 1.0ask Jeeves Desktop Search 1.7 Betasp.ask.com/docs/desktopUsabilityAsk Jeeves’ usability is remarkably simple and well reined. It searches all types 52.4EnterpriseVersatilityof documents simultaneously, and users can look through each type of results Readiness2.57via tabbed panes. It has a nice preview pane, which even plays Windows Media 00.2Player. The application is very small and eficient. Users are given index control 3.16when they can choose either fast or gradual indexing. However, Ask Jeeves’ big-SecurityAccuracygest challenge is to 3.143.20improve versatility. EficiencyCurrently, the sup-3.80ported document Source: UW E-Business Consortiumtypes are very lim-ited. Also, the preview pane does not render Microsoft Excel or PowerPoint. Web history search is not supported, either. If it would support more ile types, Ask Jeeves has the potential to become one of the top desktop search tools.ww.wwube.crogAsk Jeeves Desktop Search 1.7 Beta ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search8Enish Professional 6.1www.enish.comUsabilityEnish is a commercial software product. We tested the Professional version, 04.3EnterpriseVersatilitywhich costs $199.95. Enish is a lot different from the other 11 tools in terms of Readiness3.38usability. It gives users full customization of views. It provides calendar, contacts 00.3and weather views in addition to search/preview views. It almost simulates a 3.10Microsoft Outlook SecurityAccuracyenvironment. Users 3.002.40can create multiple Eficiencyindex iles and associ-04.3Source: UW E-Business Consortiumate them with differ-ent views. However, this customization creates a steep learning curve. It takes some time to get used to its operations. Enish has “Relevant Search” features, but its purpose appeared unclear. When we searched “Open Ofice,” for example, the relevant search high-lighted “support@amazon.com,” which made us wonder why.Enish Professional 6.1iSyS Desktop 6.0www.isys-search.comUsabilityISYS is a versatile tool. It supports multi-language indexing, FTP indexing, SQL 57.1EnterpriseVersatilityindexing (requiring XML output), and supports many e-mail clients including Readiness3.75Compuserve, Eudora and VIM. It is also good at enhancing search accuracy. 00.4It includes a spell checker, synonym rings, fuzzy logic search (which correct 3.05misspells in indexed documents) and intelligent date/number format (e.g. ind SecurityAccuracy“1/1/05” from “Jan. 1, 2005”). In contrast to those valuable features, however, us-3.002.80ability is signiicantly poor. The application creates multiple Windows menus and Eficiencyconfuses users. Custom query 3.00syntaxes create another steep Source: UW E-Business Consortiumlearning curve for users who do not want to memorize them. Toolbar icons don’t have text descrip-tions, yet their symbols are somewhat vague. When we searched MP3 iles, we were stuck because it did not provide any links or enable us to play them inside the application. ISYS must improve its usability to be able to capitalize on its powerful versatility and accuracy features.ww.wwube.crogISYS Desktop 6.0 ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search9dtSearch Desktop 6.5www.dtsearch.comUsabilitydtSearch Desktop is outstanding in terms of word accuracy features. It provides 05.2EnterpriseVersatilityphonic and fuzzy search, boolean and wildcard keywords, multi language and Readiness3.25encoding support, noise word list, which excludes common words such as “is” or 00.3“a,” and case/accent sensitive indexing. The most regrettable aspect, however, 3.02is that it does not deliver SecurityAccuracythose features very well 2.883.50to the end user because Eficiencyof poor usability. There is 00.3Source: UW E-Business Consortiumno search keyword ield on the main window, and users have to go through the menu. Setting up the above features one by one is also a tedious task. The indexer treats most of the binary iles as text and messes up the index ile with none char-acters. If it enhanced the GUI and reined the usability, it could become a very interesting desktop search tool.dtSearch Desktop 6.5diskmETa Pro 1.0.1www.diskmeta.comUsabilitydiskMETA is also a commercial product. We tested the most advanced version, 05.2EnterpriseVersatility“Professional,” which costs $97.50. The application is one of the simplest, but Readiness2.43comes with very limited functionality. diskMETA does not support any web history 00.2or e-mail search. Filtering and sorting are also limited, and there is no preview 2.63pane. However, it does have remarkable word accuracy features. It includes a SecurityAccuracydictionary feature that can identify a word, like “criterion” from a keyword “crite-2.862.60ria.” Surprisingly, most Eficiencydesktop search tools 3.40cannot do this. Iterating Source: UW E-Business Consortiumsearch results is another nice feature. Most desktop search tools we tested show all search results regardless of its amount (e.g. 2,000 matches), which can often overwhelm users. diskMETA’s page iteration is intuitive and easy to use.ww.wwube.crogdiskMETA Pro 1.0.1 ©0250 .WUE B-sunise soCsnroitmu

Decision Tools | Desktop Search10Blinkx 3.0www.blinkx.comUsabilityBlinkx 3.0 has a Metal theme, which makes it look somewhat similar to the Ma-00.3EnterpriseVersatilitycintosh user interface. The tool’s versatility is somewhat limited, but it does have Readiness3.75some unique features. “Blinkx Visualizer” produces a tree view of search results, 00.2and users can see the 2.63tree growing. Smart-SecurityAccuracyfolder” crawls the web 2.632.60to ind relevant informa-Eficiencytion to the documents 08.1Source: UW E-Business Consortiumin the folder. Unfortu-nately, the application has major problems with eficiency. Indexing is painfully slow. In fact, in our test, the application couldn’t complete the process. It runs four instances and consumes signiicant memory. In con-sequence, search outputs are slow, and a window often lickers. Hopefully, this will be improved in the future release.Blinkx 3.0hotBot Desktop Betawww.hotbot.com/tools/desktopUsabilityHotBot is a toolbar-based desktop search tool and displays output in the brows-00.2EnterpriseVersatilityer’s left pane, where Favorite and History links are often displayed. The tool Readiness3.00is very compact and has some unique features. It supports RSS indexing and 00.2allows users to associate keywords to custom web sites (e.g. “eb <keyword>” 2.34for eBay search). Unfortunately, we couldn’t complete indexing on this tool for SecurityAccuracyunknown reasons. The soft-2.862.20ware seems to have been Eficiencyrushed for the beta release. 2.00HotBot offers a deskbar Source: UW E-Business Consortiumversion, but usability is very inconsistent from the toolbar. Coniguration is text ile based, and this is simply not user-friendly. It provides a lot of custom search syntax-es but has a steep learning curve. Search results are automatically saved as HTML iles, and this may cause some security concerns. We expect signiicant improvements in its inal release.ww.wwube.crogHotBot Desktop Beta ©0250 .WUE B-sunise soCsnroitmu

Voir