First of all, thank you for a brave exposition. I am sure that there are those who would ban it if they could.
I would say that the problem of free speech can be viewed from two perspectives. One is that absolute freedom of speech is a good that in and of itself must be allowed. The other is that freedom of speech has a positive purpose for society, and that purpose must be recognized and preserved for the good of society, but allowing intolerant speech can actually be bad for society.
That framework suggests individualism vs. those who accept the reality that humans are social beings, that we have never lived as isolated individuals, but have always co-existed in groups we call societies. A person can certainly be of the latter category, calling for actions in the best interests of society as whole, without being a ‘socialist’. Those who tout individualism will tend to favor unconstrained free speech, even to the point of allowing hate to be spewed into society; those who accept reality will tend to allow limitations on speech.