Tom,
I regularly visit my brother on the south east coast and take my daughter on the beach. To overcome sand and on hot days sweat getting in my eyes and under the my lenses, I have dark safety glasses which are only £7 designed for outdoor use. O now have no problem with wind or dust in my eyes and they provide UV protection too.
At a few outdoor events I have recently attended I have had people ask where I got them and even sorted them out with a pair!
I also have what I call my emergency eye kit which has a small bottle of all the esential solutions and a mirror so if I have any problems I just take the offending lens out and find a suitable place to clean it such as a friendly cafe and clean it at the table. So far not dropped a lens in my coffee
As for swimming, my wife is six feet tall and my duaghter at five is very load, so I can usually locate them after a swim. If with my brother, he keeps an eye on me just in case I loose my bearings in the blur that is not wearing lenses. I have found cafes the ideal for taking lenses out prior to a swim and then for putting them in again. No wind or sand but a plentyful supply of water to clean your hands with.
This helps me, perhaps some will be of use to you too.