Totally agree with what Bruce said.

You could try Koolewong or Green Point in a N/E'er too. Usually a few sailors at either or both.
I prefer Umina in a Southerly but it is more Freeride or Wave style than Slalom due to waves. Also you have to look out for Learner Kiters who sometimes litter the water. They tend to stay closer in. We go a bit further out, where the grey suited ones sometimes pass. Waves near the Box are usually nice.