Dear Community
I have spent a little bit of time researching this on the forum and found patches of information but I felt I would like to open a thread.
I want to completely, to the best I possibly can block any search engines from finding my application.
Under the SEO tools in the box it currently says this
User-agent: *
Disallow: /version-test/
I assume that there is something else I can type in here to not allow Bots to crawl my live site- can anyone suggest how i can achieve this?
Many thanks in advance
1 Like
That’s usually what one writes in the robots.txt file to prevent bots from crawling your site.
dan1
3
See: http://www.robotstxt.org/robotstxt.html
To exclude all robots from the entire server
User-agent: *
Disallow: /
Add another disallow.
Disallow: /
Another option is to use meta tags. http://www.robotstxt.org/meta.html
So it really is that simple?
Far too easy guys thanks a lot!
1 Like
dan1
6
Yep, be sure to mark as solved. Others may have this question down the road. So good to flag it accordingly.
system
Closed
7
This topic was automatically closed after 70 days. New replies are no longer allowed.
