Mine doesn't spray out of the toilet at all. Even with the seat up. I'd double check that it is installed properly. When I first went to install it there is a suction cup on the side and I couldn't figure out how to make that flat on the edge of the toilet. I moved it slightly then I got it to suction (I don't have one that sprays out of the toilet so not sure if this is the issue or not - just a suggestion to why it may be happening).