I understand how you dad might have felt, but he might have been correct that challenging their behavior could have made things worse. Progressive authors today (& then) criticize Lincoln for not going far enough to liberate slaves, but fail to recall that the Emancipation Proclamation, which hardly liberated any at the time, resulted in the NY Draft Riots. Standing up to a racist majority must be done with care & consideration—are you really helping?
He sounds like a wise & generous human being. The world is poorer for his absence.