WebCrawler

(redirected from Web crawler)
Also found in: Dictionary, Wikipedia.

WebCrawler

(World-Wide Web)
A free World-Wide Web search engine developed by Brian Pinkerton at the University of Washington and now moved to America Online, Inc. WebCrawler collects URLs by searching the Internet and allows users to perform keyword searches through a World-Wide Web browser.

http://webcrawler.com/.
This article is provided by FOLDOC - Free Online Dictionary of Computing (foldoc.org)

WebCrawler

(1) A search portal that searches other search engines for results, provides yellow and white page lookup and offers three levels of searching: general Web pages, news and photos. It also provides featured listings and site recommendations by category. The site is run by Infospace, Inc., which also owns the MetaCrawler.com and Dogpile.com brands. For more information, visit www.webcrawler.com. See Web search engines.

(2) A Web crawler. See spider.
Copyright © 1981-2019 by The Computer Language Company Inc. All Rights reserved. THIS DEFINITION IS FOR PERSONAL USE ONLY. All other reproduction is strictly prohibited without permission from the publisher.
References in periodicals archive ?
These services are not actually incorporated into the open Internet [1], and it is difficult to retrieve their content with traditional methods (e.g., web crawlers).
For the rest of the search engines, greater proportion of the respondents indicated that they never used them as shown by their respective percentage response; Ask (48.1%), Bing (44.8%), Yahoo 44.2%), Blekko (46.8%) and Web Crawler (41.6%).
In view of the advantages of distributed system and the properties of web crawlers, distributed web crawler is feasible.
Another search engine named WEB CRAWLER is also launched din 1994 which index the entire pages.
Therefore, in order to crawl web pages effectively, researchers proposed web crawlers. Web crawlers are programs that collect information from the Internet.
The deep web content is not directly accessible via a standard web crawler that follows links.
A search for Web pages relevant to "The Murder Case of Nanping" under the domains shown in Table 1 was performed with the use of the independently developed BUT Web crawler tool.
Clapper confirmed the outlines of a New York Times report that the former NSA contractor had used a web crawler, a commonly available piece of software, to sweep up a huge trove of documents.
Web crawler can be configured to use valid user credentials (e.g.