Public can rate comments

I created a blog post and posted a comment on that blog post. I logged out, viewed the blog and want able to rate the comment itself by giving it plus one point. I created another blog post but this time set allow to rate to members only to see if it made any difference. Again posted a comment on the blog and logged out. When to the new blog but was still able to rate the comment by giving it one point. Is there anything in admin that allows you stop visitors rating comments etc. Something in admin perhaps?