The dementors affect you worse than the others because there are horrors in your past that others don't have.

Dementors are among the foulest creatures that walk this earth. They infest the darkest, filthiest places, they glory in decay and despair, they drain peace, hope, and happiness out of the air around them. Even Muggles feel their presence, though they can’t see them. Get too near a Dementor and every good feeling, every happy memory will be sucked out of you. If it can, the Dementor will feed on you long enough to reduce you to something like itself — soul-less and evil. You’ll be left with nothing but the worst experiences of your life.

You can exist without your soul, you know, as long as your brain and heart are still working. But you’ll have no sense of self anymore, no memory, no . . . anything. There’s no chance at all of recovery. You’ll just — exist. As an empty shell. And your soul is gone forever . . . lost.

Sure you can manage that broom, Potter?" said a cold, drawling voice.
Draco Malfoy had arrived for a closer look, Crabbe and Goyle right behind him.
"Yeah, reckon so," said Harry casually.
"Got plenty of special features, hasn't it?" said Malfoy, eyes glittering maliciously. "Shame it doesn't come with a parachute - in case you get too near a Dementor."
Crabbe and Goyle sniggered.
"Pity you can't attach an extra arm to yours, Malfoy," said Harry. "Then it could catch the Snitch for you.

What's most important in a friendship? Tolerance and loyalty.

Where your treasure is, there will your heart be also