diff options
| author | francis <francis> | 2009-09-02 14:22:42 +0000 |
|---|---|---|
| committer | francis <francis> | 2009-09-02 14:22:42 +0000 |
| commit | eed6add13064d06e21d3096f0f78abac0c2eaebf (patch) | |
| tree | 9b6675f523d33ee6289b67ff794a5bdbf3c88a6b | |
| parent | 47ec637052244bacf6a90357a276430349757da4 (diff) | |
Block a few more URLs that don't need to go in search engines.
| -rw-r--r-- | public/robots.txt | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/public/robots.txt b/public/robots.txt index d24e1fe0f..8e47db0f7 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -1,7 +1,11 @@ # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-agent: * +Disallow: /annotate/ +Disallow: /new/ +Disallow: /search/ Disallow: /similar/ +Disallow: /track/ Disallow: /upload/ Disallow: /signin |