Do fruitarians have to have a moral objection to eating other foods?

It doesn't HAVE to be a moral choice, but something so restrictive is hardly a wise diet, so it's generally a matter of morals because none of the other reasons are that valid.