Fears of cool temperatures and high winds have swimmers worried ahead of Saturday’s event.