Home› General Chat
If you need urgent support, call 999 or go to your nearest A&E. To contact our Crisis Messenger (open 24/7) text THEMIX to 85258.
Options

Help With Stopping Search Engines Indexing a Page

BillieTheBotBillieTheBot Posts: 8,721 Bot
in General Chat
I've created a website but one page needs to be not found on search engines, as it's a download page.

Is there a way for me to stop search engines from indexing this specific page?
Beep boop. I'm a bot.

Comments

  • Options
    JsTJsT Posts: 18,268 Skive's The Limit
    Should be able to use the robots.txt file

    http://www.robotstxt.org/orig.html
  • Options
    Former MemberFormer Member Posts: 1,876,323 The Mix Honorary Guru
    .htaccess

    robots.txt files are still scanned by bots, so not only does the bot not farm it the directory, but it farms the robots.txt file giving everyone the pleasure of viewing it.

    http://www.kxs.net/support/htaccess_pw.html

    Add a password to the page, it's easier.
  • Options
    Former MemberFormer Member Posts: 1,876,323 The Mix Honorary Guru
    Yeah some robots will find the page whatever you do. The more polite ones obey robots.txt but others don't.

    Your best bet may be to use server side scripting to dynamically create the page depending on various conditions (IP, broswer header, login cookie, etc..)
  • Options
    Former MemberFormer Member Posts: 1,876,323 The Mix Honorary Guru
    I solved it another way, forced people to enter their paypal email address into a contact form so that I can verify they are a paying customer before sending their product.
Sign In or Register to comment.