After applying a mask, are you getting the inverse area masked ? ( Like in this sample)
This has an easy solution 
Just open any document or create a new one in Photoshop
And double click at the Quick Mask tool in the Tools bar ( where the red arrow points above). You will get the Quick mask options window, make sure to select Color indicates = Masked areas and accept.
Now you can apply the mask correctly ( sometimes you will need to delete the incorrect mask first in the layers palette).
