if we bury it in some sort of casks in salt mines, etc... so long as we don't forget where we buried it, can we not retrieve it again later if we need it?

I will assume we don't bury it in leaky containers that corrode and poison our own ground water.

They are all valuable, precisely because they are radioactive.

Were we to do so, and "forget where we buried it," it would be of very low consequence.

As for radium, the planet has always existed with "buried" radium, largely radium-226, which is secular equilibrium with uranium, and has been so for billions of years, since the formation of this planet. Radium-224 is formed in the decay series of thorium, but the concentration is very low, precluding isolation.

so long as we remember where we buried it.

I'll take it that as yeah, there isn't any problem with putting it underground for storage.

It's the waste of valuable material.