Yes.. it happens! Minerals, and marine growth.
I use On/Off to clean this, it's easy!! Even on a rag, but it's acid based so be careful around your trailer or other things and wear gloves and don't breath it!
http://www.amazon.com/MaryKate-Off-H.../dp/B0000AXNNA
Note: Parafunalia has this on their shelves.
I often use some on a rag and clean the waterline. Don't get it on your bottom paint if you have any as it will remove that too. It works great.
There are other products also, but this one works the easiest for me. If it's just a few hours, wash the boat, keep up with it, but after a season in the water, you can use a garden sprayer to make the bottom of your boat look like it just came out of the factory.
Again, don't get it on your trailer, cover it somehow, but it will make cleaning your boat very easy otherwise. It's powerful stuff.