I think we have to distinguish between political speech and other speech. Political speech would be speech proposing or supporting or opposing any actual action of government: any law, policy, program, etc.

Political speech is necessary because it is part of a just political process. To constrain it is unjust on the face of it. Outside that, speech can be justly constrained--though delineating speech that should be constrained in a nation in which freedom is the ideal is not easy to do.

True freedom, though, is not just freedom from the power of government, but also freedom from undue interference in our lives by other people. We can and should separate 'speech' from physical confrontation, harassment, etc. Having words written as a sign of some kind or chanting some favored phrases can't sanctify such behavior.