This is mostly preference, but here are some positives and negatives:
High: knock it into things, can be irritating when you wear gloves, but makes stone look larger
Low: less likely to bump into things, often more integrated with shank, depending on the setting it may be harder to fit a wedding band flush, and the stone looks slightly smaller from the top view.