Tom's Subject Directory & City Building Fan Site

Search a Section of this site or the Entire Site
site search by freefind

The Invisible Web

Directory Top > Search Services > Invisible Web (19)

Bookmarks: Articles | Search Services | Searchable Databases

The Invisible (Deep) Web Introduction

Skip Intro

For years databases on the Web were referred to as specialty databases, subject-specific databases, virtual libraries, and other similar terms. The term "Invisible Web" was coined and these databases were referred to (and still are) as the invisible web. Recently the term "deep Web" was coined and both invisible and deep web are used today.

The "Invisible Web" refers to Internet resources whose contents cannot be indexed by search engines (this was written in 2001 and to date, it is very possible the Invisible Web no longer exists). These include databases and archived material, as well as general reference web sites.

The Invisible Web consists of hundreds of millions of web pages that are not indexed by the traditional search engines. The Invisible Web makes up a significant proportion of the web with an estimated size of between a half and two thirds of the total. Typical content of invisible web pages include interactive tools, archived material, statistical sources, news sources and a whole variety of full text databases such as dictionaries, encyclopedias and online books.

Deep and several other names are used to describe the Invisible Web, but "Google's deep Web surfacing system pre-computes submissions for each HTML form and adds the resulting HTML pages into the Google search engine index. The surfaced results account for a thousand queries per second to deep Web content.", according to WikiPedia. Then I should add, the Invisible Web may no longer be invisible, the deep Web may not be so deep.

Invisible (Deep) Web Search Services

"BrightPlanet has uncovered the "deep" Web a vast reservoir of Internet content that is 500 times larger than the known "surface" World Wide Web. The deep Web contains billions of high-quality documents in about 350,000 specialty databases -- all hidden from view from standard search engines."

The Librarians' Index to the Internet is a searchable, annotated subject directory of more than 7,000 internet resources selected and evaluated by librarians for their usefulness to users of public libraries. LII only includes links to the very best net content. While not a "pure" Invisible Web pathfinder, LII categorises each resource as best of, directories, databases, and specific resources. Databases, of course, are Invisible Web resources. By using LII's advanced search feature, you can limit your search to return only databases in the results list. Advanced search also lets you restrict your results to specific fields of the directory (author name, description, title, URL). The Librarians' Index to the Internet is a laser-sharp searching tool for finding Invisible Web databases.

"PubMed, a National Library of Medicine service, provides access to over 11 million MEDLINE citations back to the mid-1960's...including links to sites providing full text articles and other related resources."This decade, this is no longer true, since this figure has doubled.

PubMed is a free resource that is developed and maintained by the National Center for Biotechnology Information (NCBI), at the U.S. National Library of Medicine (NLM), located at the National Institutes of Health (NIH).

This site claims more than "22 million citations for biomedical literature from MEDLINE, life science journals, and online books". A "Quick Start Guide" along with full text articles, Frequently Asked Questions and Tutorials to get your research started. There are clinical and topic-specific query tools available along with MeSH (Medical Subject Headings) Database and Journals in the NCBI Databases.

Adobe and AltaVista teamed up to enable us to "search through more than a million summaries of Adobe Portable Document Format (PDF) files on the Web..." Now you can search through more than a million summaries of Adobe Portable Document Format (PDF) files on the Web. You can see the summaries in the search results before deciding to view the original Adobe PDF.

Searchable Databases


DeepPeep - database form search     [New Window]
DeepPeep is a search engine specialized in Web forms. The current beta version tracks 13,000 forms across 7 domains. The domains are Auto, Airfare, Biology, B ook, Hotel, Job and Rental. An advanced interface supports complex queries over the form repository (e.g., show the top-k labels in a domain, or top-k values for specific label in a domain. Advanced search allows you to perform more specific queries. Besides specifying keywords, you can also search for specific form element labels, i.e., the description of the form attributes.

Database for family-safe animal-related sites.

Searchable database of the Chicago Police Department's records of reported crime. You will be able to see maps, graphs, and tables of reported crime. The database contains 90 days of information which you can access in blocks of up to 14 days. Data is refreshed daily. However, the most recent information is back-dated 7 days from today's date.

Education Resources Information Center. An enormous database of education resources maintained by the United States federal Department of Education.

Provides access to statistics maintained by United States government agencies; most statistical data from the United States Census Bureau (and the equivalent bureau of other countries).

Online Encyclopedia of Life

This database provides "authoritative conservation information on more than 50,000 plants, animals, and ecological communities" of the USA and Canada.

"Australian Geological Survey Organisation maintains a database of nuclear explosions with the location, time and size of explosions around the world since 1945."

Most people use this web site to get information about a particular number sequence. At the present time the table contains over 150000 sequences.

"Type in a series of numbers and this database will complete the sequence and provide the sequence name, along with its mathematical formula, structure, references, and links."

The Web version of the important Medline medical database.

Updates financial/currency rates once a minute.

Find the total population, the population density, and the other zip codes within the given radius of a particular zip code.

Invisible (Deep) Web Articles

  The Deep Web: Surfacing Hidden Value
  by Michael K. Bergman

  Vol. 29, #4, Apr. 2002: The Invisible Web   by Anne Clyde

  Invisible Web Gets Deeper
  by Danny Sullivan
  Aug. 2, 2000
"Invisible Web" Revealed   [New Window]
  From The Search Engine Report
  July 6, 1999