Changing Permissions makes pencils.

System

If I make a change to the permissions on a file, folder or even the workarea it flags my files as modified. Just the act of adding a group or user to the permissions changes them. I think I've seen others comment about this. Is there any way to prevent this?