Search this Directory or just a Section (Category) of the Directory
Introduction To Search EnginesSkip Intro
I. Crawler-based search engines have three major elements.
A. First is the robot, also called a spider or crawler. Actually, the "robot" "crawls" the web following links and indexing web pages.
The robot visits a web page, reads it, indexes it and then follows links to other pages within the site. This is what it means when someone refers to a site being "spidered" or "crawled." The robot returns to the site on a regular basis, such as every month or two, to look for changes.
B. Everything the robot finds goes into the second part of the search engine, the index. The index, (an index to a database or set) sometimes called the catalog, is like a giant book containing a copy of every web page that the robot finds. If a web page changes, then this book is updated new information. Sometimes it can take a while for new pages or changes that the robot finds to be added to the index. Thus, a web page may have been "spidered" but not yet "indexed." Until it is indexed -- added to the index -- it is not available to those searching with the search engine.
C. Search engine software is the third part of a search engine. When you type in your keyword and press Search (or whatever); this is the program that sift's through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant.
All crawler-based search engines have the basic parts described above, but there are differences in how these parts function (the algorithms used by the robot, the indexing algorithms, the keyword relevant algorithm). That is why the same search on different search engines often produces radically different results.
A search engine is a database along with the tools to generate that database and search it. A catalog (directory) is an organizational method with a related database and the tools to generate it. There are several sites that try to be a complete search service; providing news, libraries, dictionaries and other resources, and some of these can be very useful.
The search box, at the top and bottom of the white portion of this and every page on this site, is all three parts in one. That is, once a week, the FreeFind robot indexes this entire directory and maintains it in a seperate database.
It performs a simple search using Title and Description Meta Tags and text. It also accesses it's own database (FreeFind) and displays any relevant results along with the results of searching this directory. It ain't Google, but it's FREE. -:)
AltaVista is consistently one of the largest search engines on the web, in terms of pages indexed. Its comprehensive coverage and wide range of power searching commands makes it a particular favorite among researchers. In addition to crawler-based web page matches, it also offers news search, shopping search, multimedia search and human-powered directory results from LookSmart (see below). AltaVista opened in December 1995. It was owned by Digital, then run by Compaq (which purchased Digital in 1998), then spun off into a separate company which is now controlled by CMGI.
Direct Hit measures what people click on in the search results presented at its own site and at its partner sites, such as HotBot. Sites that get clicked on more than others rise higher in Direct Hit's rankings. Thus, the service dubs itself a "popularity engine." Aside from running its own web site, Direct Hit provides the main results which appear at HotBot (see below) and is available as an option to searchers at MSN Search. Direct Hit is owned by Ask Jeeves (above). Some Direct Hit information appears at Ask Jeeves. See the Using Direct Hit Results page to learn more about Direct Hit.
Excite offers a medium-sized crawler-based web page index, as well as access to human-powered directory results from LookSmart. Excite was launched in late 1995. It grew quickly in prominence and consumed two of its competitors, Magellan in July 1996, and WebCrawler in November 1996. Magellan was discontinued in April 2001. WebCrawler continues to operate as a separate service, but it provides the same results as the Excite.com site itself.
FAST Search consistently has one of the largest indexes of the web. FAST also offers large multimedia and mobile/wireless web indexes, available from its site. The site, also known as AllTheWeb.com, is a showcase for FAST's search technologies. FAST's results are provided to numerous portals, including those run by Terra Lycos. FAST Search launched in May 1999.
Google is a search engine that makes heavy use of link popularity as a primary way to rank web sites. This can be especially helpful in finding good sites in response to general searches such as "cars" and "travel," because users across the web have in essence voted for good sites by linking to them. The system works so well that Google has gained wide-spread praise for its high relevance. Google also has a huge index of the web and provides some results to Yahoo and Netscape Search.
Oveture [New Window]
Was Goto but recently changed its name to Overture. The most popular Pay for ranking search service.
HotBot [New Window]
HotBot is a favorite among researchers due to its many power searching features. In most cases, HotBot's first page of results comes from the Direct Hit service (see above), and then secondary results come from the Inktomi search engine, which is also used by other services. It gets its directory information from the Open Directory project (see below). HotBot launched in May 1996 as Wired Digital's entry into the search engine market. Lycos purchased Wired Digital in October 1998 and continues to run HotBot as a separate search service.
Inktomi [New Window]
Originally, there was an Inktomi search engine at UC Berkeley. The creators then formed their own company with the same name and created a new Inktomi index, which was first used to power HotBot. Now the Inktomi index also powers several other services. All of them tap into the same index, though results may be slightly different. This is because Inktomi provides ways for its partners to use a common index yet distinguish themselves. There is no way to query the Inktomi index directly. It is only made available through Inktomi's partners with whatever filters and ranking tweaks they may apply.
Northern Light [New Window]
Has ceased being a public search service, but you can still the document collection.
Northern Light was a favorite search engine among researchers. It featured a large index of the web, along with the ability to cluster documents by topic. Northern Light also has a set of "special collection" documents that are not readily accessible to search engine spiders. There are documents from thousands of sources, including newswires, magazines and databases. Searching these documents is free, but there is a charge of up to $4 to view them. There is no charge to view documents on the public web -- only for those within the special collection. Northern Light opened to general use in August 1997.
IWon [New Window]
Uses Inktomi's database and provides listings of average comprehensivness and accuracy. If you use this search engine you can win millions. If you're the type that spends money on Lotto tickets then this search engine is for you. You won't lose money here, but we suggest you use a search tool that can survive on its merits rather than need hold out the carrot of winning millions.
Oingo [New Window]
Attempts to improve relevance via menu options provided with each search term on your results page. This allows you to refine your search further based on a specific meaning for each search term. A great idea, but hindered by the quality of the initial database and search algorithm which results in far too many irrelevant listings. While not perfect w applaud this effort to improve on the standard search process. Worth a look!
Basic Text Only Search [New Window]
The new Text Search provides all of the features of Raging Search, and more: Viewable on text-only browsers, World-class relevance, Lightning-fast performance, News related to your query.
Splat searches the millions of web sites splattered all over the internet to find exactly what you need, so you dont have to. Splat! is a privately held company in northwest Indiana, United States.
Twirlix [New Window]
Twirlix is working hard to make the search process easier. Where most search and metasearch engines are almost identical Twirlix is trying something new and potentially better than the industry standard. Unique features include a 5 dot "quality" rating system, site home page thumbnail, and separation of listings into "Top Matches' and "Further Matches". For those of us who are more visually oriented, this search engine's organization is a big step forward.
Search this Directory or just a Section (Category) of the Directory
This Site is Indexed Weekly by the FreeFind Search Engine
News & Weather
Hybrid Search Engines
News Search Services
FTP Search Tools
Search Google, Lycos, WebCrawler, Excite, AltaVista, or HotBot using the search form Below.
To submit a site to this Directory:
Find the page your where your listing should be and send to firstname.lastname@example.org. Send Page Title (the page(s) in this directory you want your listing to be on), URL (the URL to your page) and Description (don't be too breif. One or two paragraphs.). You will be notified when your page has been added to this directory. Thank you in advance for helping this Directory grow.