Two transcription factor genes, fruitless (fru) and doublesex (dsx), are the primary factors that direct the development of brain sex differences in Drosophila. In the nervous system, the dsx gene produces different proteins, DsxM and DsxF, respectively, in males and females, whereas the fru gene produces proteins (FruM) only in males. Thus, the dsx-dependent sex differences in the nervous system likely reflect the distinct target specificity of DsxM and DsxF, whereas the fru-dependent sex differences rely on the presence and absence of FruM. Some neurons express both fru and dsx and others express either fru or dsx, while the majority of neurons express neither. By studying the sexual dimorphism of single neurons, insights into the molecular mechanism whereby FruM specifies the neuronal sex have been obtained. The sexually dimorphic morphologies of a fru-expressing neural cluster in males are completely feminized when FruM is lost, whereas the same cluster has an intersexual appearance under moderate reductions in FruM in fru hypomorphic mutants. Single-cell labeling demonstrates that even in such fru hypomorphic mutants, each neuron in the cluster has a complete male-type structure or a complete female-type structure, indicating that the intersexual appearance of the cluster results from the mixed presence of the male-type and female-type neurons. This all-or-none mode of sex-type determination by FruM is mediated by a competitive recruitment to the FruM-target genomic sites of two antagonistic chromatin regulators, histone deacetylase 1 and heterochromatin protein 1a, each of which masculinizes or demasculinizes single neurons in concert with FruM. These findings open up a new avenue for the study of epigenetic bases for sexual differentiation.