Been a while... if I stay netural, can I have anyone in my party?

I remeber being too good makes you lose evil characters, being too evil makes you lose good characters (and neutrals as well). if I stay in the neutral area, can I have anyone in my party (Except people who conflict?). Trying to optimize my party this time (rather than roleplay) so i just wanted to check.

Thanks!