I use basic Elmer's glue for the initial attachment of a sand layer. However I often had trouble with a diluted drip of glue over the top to lock stuff in place - it would often upset the sand and rocks, or wash things away even if I was careful.
Therefore I've started using liquid super glue to drip over the top of the sand to lock everything in place. A brand that works really well in my experience is Starbond, their super thin variety. It seriously runs into every crack. Once the glue sets, the basing materially is essentially indestructible and will never rub off. You won't have to worry about any amount of punishment during the painting process dislodging materials.
An added benefit of this technique is that small rocks and bits of fill-in sand can be sprinkled over the basic glued sand before dripping the super glue. It will all get locked down.
