I hear this stuff about apartheid too and I usually like to agree with these folks but on the condition that the roles get reversed; if colonization and slavery were a net good for the country, then the whites can be the slaves (and in the case of South African apartheid, they get to live in the bantustans and open air prisons under the control of the black population).
I'm sure these people aren't racist and it's the principle of the matter, hence role reversal should be fine.