I agree with what WisdomBeing said, that we shouldn't bounce back any spells or curses that has been inflicted upon us.
I remember reading that Tulku Dragpa Gyeltsen (not sure if it's the right lineage lama, that this occurred to - do correct me if I'm wrong) was traveling with his entourage from one area to another. When they arrived at their destination, Tulku Dragpa Gyelten (TDG) went into his tent and asked everyone not to enter. From the outside of his tent, his students and followers could see bolts of lightning hitting the tent continuously, and when TDG walked out, he passed a box to one of his students and asked him to throw it away as far as possible. When his student threw it, the box landed and there was a huge explosion from all the energy from the lightening.
From TDG actions and practice, I think that it goes to show that Dorje Shugden's prayer would not bounce back any curses or black magic, but instead absorbed and removed.
That's my take of the story anyways.
Going back to what TK explained, I have a question: If you feel like there was something (spirit or the like) disturbing you and you immediately visualize your Guru in your mind whilst reciting your Guru's mantra, if that doesn't repel the spirit away - does that mean that your samaya is not clean?