When I lived in El Paso my policy covered me in Mexico for up to 20 miles from the border which is common if you live next to the border, but not so much if you don't. Some insurance policies cover you in Canada. For anywhere else you generally need an overseas insurance policy which you can buy from most of the bigger insurance companies.

far as I can tell.

I've bought overseas policies before. Even if the rental car company offers it, you'll be better off going through your own agent. It should be considerably cheaper.

Doesn't sound great.
http://carinsurance.arrivealive.co.za/car-insurance-for-foreign-travelers-to-south-africa-mission-impossible.php

One thing I have is a credit card with a small credit limit that I use just for renting cars. If you read the fine print in the rental agreement, they can charge your credit card for any damage. I had a rental car company do this to me. So now I have a card with a $500 limit.

Another thing I've ran into is that if you travel to a country with a high accident rate, they often require that you get coverage from the renter.

When I used the card to rent a car, I got the insurance.

Might want to check that out.

It's hard to imagine that a S. African car rental agency doesn't offer liability insurance.

The car company only offers damage, same as credit card, medical, and theft. I cant find how liability is covered although I am sure it's there somewhere.