From my reading, I think John Brown scared a whole bunch of the so-called Black activists of that time.
.. I think John Brown must have been saying to himself: "this Negro Frederick Douglas is a coward! All he wants to do is go around preachifying to white folks - he's afraid of true revolution."