Another solution without doing something drastic or rash would be to try Mupen64plus. Sixtyforce is quite out of date it literally hasn't been updated in years with updates then also only coming once in a blue moon. I've even used mupen64plus to run the rice plugin and high res texture packs on the Mac.
The emulator itself:
http://code.google.com/p/mupen64plus/
If you want a front end:
http://www.speedofma...mupen64plus.php
On the other hand your weakest link in my opinion is probably your intel integrated graphics more than anything. But yeah, sixtyforce is just too old by now... to put things in perspective for people here it'd be like wondering why jfduke from years back doesn't render the polymer HRP properly. Sixtyforce being an out of date emulator also has quite the low compatibility base with various games and mostly just supports the first games that where emulated like Mario 64.
I also wonder what windows running programs you tried on your Mac. Parallels? That should have ran just about any emulator you throw at it just fine. Wine takes more of a person knowing what they are doing to get it right but there's a community dedicated to that "porting team". Crossover is pretty much the same thing as Wine. Also, one way isn't always nessasarily better than the other and I've used for example parallels to run one game and Wine to run another.
But yeah, I'd suggest to look into mupen64plus for N64 emulation on the Mac.... example shot.. *edit* with the aformentioned rice plugin and a custom high res texture pack... (some textures there are by me)