Hide website from Google - how it works
If you hide a website from Google, it will not appear in the results pages of this and many other search engines. We show you how this works in this guide.
Hide homepage from Google
Google and other search engines scour the Internet with so-called crawlers, which are also known as "robots" or "spiders". These index files for different types of search services. You can prohibit crawlers from accessing certain files, paths, or the entire website.
- In the main directory of your URL, create a file called "robots.txt".
- Write in this "User-agent: *" to address all crawlers. In the second paragraph we explain how you only address certain crawlers.
- If you want to exclude a file from search engine results, write in a line "Disallow: \ path \ file name. file extension". Write the command without quotation marks and replace "path", "file name" and "file extension" according to your requirements.
- To exclude a folder, write "Disallow: \ path \". End the path name as in the example with a backslash.
- To hide the entire URL from search engines, enter "Disallow: \"
- Some crawlers follow the additional instruction "". You can enter this as the first line of "robots.txt".
Hide website from certain crawlers
You can specify the following user agents to keep only certain crawlers away from your website:
- "Mozilla / 5.0" and "Googlebot / 2.1" for normal Google searches.
- "Googlebot-Image / 1.0" for Google image search.
- "Googlebot-Video / 1.0" for the Google video crawler.
- "Googlebot-Mobile / 2.1" for Google search on mobile devices.
- "EmailCollector" so that email addresses are not recorded.
- "Lycos / xx" for the shower from Lycos.
- "Bingbot" for Bing's search.
- Google and Bing list other crawlers.
- On the Abakus Internet Marketing website you will find a sample file that contains many other crawlers.