I'm a girl and your weight - but not sure that qualifies me to answer!
I don't get to sail nearly as often as I would like so find that fitness is definitely an issue. However, I don't find or think it should be linked to sail size or struggling to hold a sail down. Assuming wind strength stays the same, holding a sail down is primarily about your weight and commitment to the harness. This should not change as you tire.
A few things that spring to mind...
1) As tough as we think we are, we simply don't have the muscles of windsurfing dudes. If you consistently feel uncomfortable, are you applying enough downhaul? Not uncommon I think for women to not have enough. (Expressions of interest for lovely man to follow me round and tighten my downhaul are welcome

).
2) Is your boom high enough? Without making yourself over powered, a low boom can make you feel like you're struggling to hold down the sail when you're light and girly.
3) As for many responses above - play with line length and position. I think you've mentioned before that you use a seat harness? If you're uncomfortable, maybe try a waisty for something different. I know it's a bit 'each to their own', but I find a waisty helps me to hold down a sail much more comfortably. Also, lines further back helps heaps for early planing and keeping the rig more upright, but I find it does make it harder/more uncomfortable to go upwind.
Fitness is certainly a big issue for me and limits my sailing. Mostly my arms. As the session gets longer, I find it harder to trim/adjust the sail and therefore more uncomfortable. Also harder to gybe. As soon as I hook out and my arms have to do some work, I fall apart.
Hope all these responses help! Windsurfing - the one place chicks can quite comfortably talk about their weight!