Getting your site Indexed. Website Builders- Web Site promotion help. What you really need to begin. Free Website advice from Webdisplays

Your Site

Page Design
your.com
Content
Layout
Compatibility
Navigation
Front Door
Ecommerce
Security
Multimedia
Images
Connections

Dynamic HTML
Introduction
Graphics
Animation!

Maintenance
Overview
Tools
Promotion
Indexing
Search Engines
Reports

Your Name
Domains
Hosts
Free Pages
$ Value
Designers

HTML Tips
Trial & Error
Editors
RGB Color Chart

Contact Us!

Getting Indexed

One of the biggest races on the web, is the desire to get indexed, or listed in the results pages of website search engines at or near the top of the results pages for particular search requests. The most vociferous battles rage over popular search items like the words mp3, sex and so on. Needless to say, the search engines have noticed the battle and become rather annoyed at the tricks used by webmasters to get their sites listed higher than others.

Even if your site is not a common sort, you are likely to be affected by the measures search engines have taken to eliminate 'spam' from their indexes of webpage content. The open text indexing process is really a simple one, and here is how it should work:

Submit your main, front page URL by visiting the AddURL page at the SE.

Engine checks if the page exists and schedules it for indexing.

Indexing program called a 'robot', 'bot' or 'spider' visits the site.

Bot reads robots.txt file on your server to see what it may be forbidden to index.

Bot loads the first document it was sent for and notes it's title.

Bot may check for document language and/or character set, last modified and expiry meta information.

Bot checks for controls, keywords and (maybe) descriptions in the meta tags.

Bot may read all text or just the first few paragraphs of text instead, to summarize or store the 'content'.

Bot may also check any other allowed documents that appear as local links from the first and succeeding pages, called 'spidering'.

Bot saves information (nouns, word or phrase list) for entry into the search engine database (indexing).

Database indexer catalogs words and rates site based on relevancy of the frequency, positions and words recorded, and it's content rating.

Relevancy may also be affected by website name, path names and the filename of the document.

Site appears somewhere in search results listings for relevant words and phrases.

Bot may schedule a revisit if instructions are given in meta tags, or not, as it pleases.

In between steps 4 and 5 above, and during step 9, an additional 'cloaking' process may be implimented by the webmaster that customizes the document especially for the sort of visitor (robot) in question! This can be accomplished by several means, but the most reliable use both the IP address (network information) and User-Agent (software 'name' of the requestor) to determine how the page should 'look' to particular robot. This is a complex process involving high maintenance which you may wish to explore further at Fantomaster, but it is despised by search engines and surely not for the faint hearted!

This process varies considerably depending on the search site in question, and a few (like Yahoo and Netscape's Open Directory) use no robots at all, but rather humans do this grunt work manually (far better). Any good site design has to include meta tags, which are an industry standard method and format for you to include invisible information on your pages to supplement or take the place of text that may or may not actually be on the page in question and provide additional information and preferences about each document. Indeed, some bots do not even follow links on their own, and they must be told about each page individually.

Now, let's examine the process and the methods that the bots and indexers use to try to understand your pages.

Continued... Read the rest of this first, or:

Here it is! Webdisplays Free Meta Tag Generator...

Back To Main