This robot crawls your web site to create an index, available at : websearch.kompass.com
Below you will find the answers to the most frequently asked questions about this robot.
Why are you crawling my site ?
Kompass indexes the pages of your site, as do other search engines, in order to create an index.
This index serves to respond to requests made by the users of the Kompass information system.
This search engine includes all of the client sites of the Kompass Weblinks service.
How can I prevent your robot from crawling all or part of my site ?
You may refuse the indexation by our robot of part or all of your site by specifying a file robots.txt and/or appropriate META markers, like for the other major search engines.
Your robot requests a robots.txt file which does not exist on my server. Why ?
The robots.txt file is a standard file described on this page : Robots Exclusion Protocol
If you would just like to avoid the "file not found" error, then create an empty robots.txt file.
Why have the crawled pages of my site not appeared yet in the index ?
Don't worry if your crawled pages do not appear immediately in the index. Your pages will be included soon as long as your site is still part of the Kompass Weblinks service.
Kompass reserves the right to not index all or part of your internet site.
What are the types of links followed by your robot ?
Our robot follows the HREF and SRC links except those of the type REL= "NOFOLLOW".
Why haven't all of the pages of my site been crawled ?
Our index currently contains numerous pages from all of the sites of the Kompass Weblinks Service. The first pages of your site are prioritized over the back pages. Thus it is possible that some of your pages are not included in our index.