Go Deep, Young Man, and Find It

Wendy Boswell is a genius who knows all about web stuff. Below is her advice on finding the “Deep Web,” with 500 times the amount of web pages catalogued by Google.

“That’s one of the tricks you can use to find Invisible Web content – just put the word “database” in your query and more often than not you’ll come back lucky. This takes you to an academic, accredited, footnote-able institution. Way more worthy of a citation than say Billy Bob’s Guide To That Them There Hogs.

Let’s try another query: how about you’re doing an in-depth report on the past ten years of plane crashes in Argentina. Try a query for “plane crash Argentina” in Yahoo and you’ll get mostly news items, which would take a long time to comb through. Let’s try this query again: “aviation database”, and then we’ll work our way down to the plane crashes.

The fifth site on our list is the winner, and that is the NTSB Aviation Accident Database. It took a bit of work to get there, but with the depth of search and information that this particular database offers, it was worth it.

Invisible Web Gateways

Maybe you don’t want to dink around with finding databases on the Web; you would like to go straight to the databases themselves. There are sites that serve as invisible Web “gateways” that will help you do this. Here are just a few:

Invisible Web: Gary Price and Chris Sherman have put together a searchable directory of various databases on the Invisible Web.

Librarians Index to the Internet: A directory of various sites on both the visible and invisible Web put together by librarians; all are reviewed before inclusion and have the Librarian Stamp Of Approval.

GPO Access: An amazing site. You can search hundreds of US Government databases at the same time.

WebLens Scholarly and Academic Research Resources: Includes databases as diverse as the Stanford University High Wire Press Archives, a searchable index of past issues of the New England Journal of Medicine, and the Smithsonian Institution Libraries, a “galaxy of knowledge.”