That is understandable, but no one should be harmed for saying “group” are dumb, “group” are evil. Inciting violence against a person or group is wrong depending on how much influence the inciter has to cause real physical harm in the near future. Saying someone stole or SA’d with no proof or not good proof isnt part of free speech too.
If it’s ok to punch nazis then it can open the door to punch other groups with different opinions.