Use of "Robots" META-tags

Unlike 'robots.txt' files, describing site indexing as a whole, tag '' manages indexing of a certain web-page. And it is possible to deny indexing not only the document itself, but its links as well.

Indexing rule are written to 'content' field. The following rule can be used:

Default value: <meta name="Robots" content="ALL">.

Do not list values separated by comma. For example, the following will not work:

<META name="ROBOTS" content="noindex, nofollow">

Correct variant looks like this:

<META name="ROBOTS" content="none">

In the next example the indexer can analyse a document does not pick out links for further search:

<META name="ROBOTS" content="nofollow">

Name of the tag, names and values of fields are not register sensitive. As a matter of fact, the indexer checks only for 3 values: NOINDEX, NOFOLLOW and NONE, as far as FOLLOW and INDEX are default values.

Up

Back | Contents | Proceed