(No spoilers for OotP here. Please don't put any in the comments.)
There's a fair bit about mind control in the HPverse. We're told about the Imperius curse, which gives the caster complete control over the castee, unless the castee can resist. This is one of the Unforgivable Curses and "the use of any one of them on a fellow human being is enough to earn a life sentence in Azkaban".
So, now we know: Imperius == bad.
But what about Memory Charms? Wizards are forever casting these on Muggles. These charms modify your memory. They alter it. And the wizards seem to think this is just fine.
It's not for the good of the Muggles; it's all to protect the secrecy of the wizarding world. Muggle sees some magic? Memory charm. Nobody thinks twice.
At the World Cup in GoF, there's a Muggle family running the campground. When Death Eaters torment them by levitating and jeering at them, the "good" wizards are horrified. But when a wizard casts a memory charm on the man over and over and over again, nobody seems to think this is a problem.
But it is a problem. It's an incredible invasion.
And it's not just Muggles who have their memories modified. Lockhart uses them left and right to make wizards think he's done great deeds instead of them.
So how can anyone trust what they know?
It's all so fucking creepy.