d7b6 robots

Hi,

Please create a default robots.txt file...

I think user don't want flash/modules/chat/plugins/admin.swf and /administration/modules.php in search engine..all already indexed by google!

Thank

I think that's one of those things I would rather do myself. Managing what pages are available for indexing is all part of optimizing your SEO specifically to meet the needs of your site.

I think some people don't know how to do it and what folder disallow...

I can see your point, sort of.

A robots.txt file is not a necessity, just another tool to help keep certain pages from being indexed. Certain pages should not be indexed, of this, you are right but typically speaking, those pages cannot get passed the security anyway (pages that need passwords). By default, all pages that have guest/read only access will be indexed.

I have absolutely zero php skills but am fairly accomplished at basic html (and a little asp.). I know doing this will be easy for me as the skills required to complete the task are minimal. If someone doesn't have the skills to complete such an ordinary task as this, they should question whether they should undertake a Dolphin install or not. If they do decide to go ahead, they will most likely need some help from the community here or there and may even need to hire someone to custom fine tune their site for them.

I would be more interested in seeing an RC release made available than to keep pushing for the little things that can clog the wheels of progress. We can all deal with installing our own robots.txt file for now and maybe at a later date, when things have calmed down a bit, a default robots.txt file could be created and added to a future upgrade.

A robots.txt file can do more harm than good.  Bad bots will just ignore it, and it makes it easier to find what you are trying to protect.  It's much better to block a list of user agents in a .htaccess file.

Ex: http://www.htaccess-guide.com/index.php?a=9


You'd have to add the regular search engine bots.

Thank you houstonlively!