Seen plenty of sharks in my time kayak fishing and surfing in San Diego county and Baja, never even considered hauling a gun around. If a shark big enough to eat you decides to do so, it will be over before you know what hit you. When you paddle out in the ocean you take your chances in any number of ways, if you don't want to risk it, stay on shore. Probably drop the gun over the side anyways.
I do carry a sidearm when camping while kayak fishing in the Cascade Mountains because the fish smell attracts bears and cougars. Don't want to kill them but a couple rounds usually scares them off. Does that count?