Pair of board shorts and a T-shirt should be fine. Pick up a pair of waders (non-neoprene) if you really want to. Check out sports authority. Cotton has a tendency to stay wet and chill you in the cool air and wind. Synthetic fiber is the way to go(polyproplene). Little Marine Corps trick

....
paddling gear is spendy and really isn't needed 9 months out of the year. I think a new pair of waders run around $80 - $120...