..of the genre.. I read and re-read that book so many times it literally fell apart in my hands...

Not sure if it's the one you're thinking of, but it's one I would highly recommend.

Not the one I'm looking for.

It involves a man,
walking home.

Mid 60s novel, but set in Northeast US. It takes place several centuries after a devastating nuclear war. Society is like medieval Europe with a religious order to suppress any technology that might contain “atoms” since that brought about the downfall
And I will second the endorsement for Lucifer’s Hammer. I think it would be relevant today.

The one I am looking for is set in southern CA
and occurs immediately after the event.

I think everyone here would consider it quite right-wing. It is basically a conflict between technology and authoritarianism.
The writer was a major influence on Reagan’s SDI and a long-time associate of Newt Gingrich.

There's a degree of racism in it that is truly horrifying.

But you are right about the racism.
If Trump could read I bet he would love the philosophy behind the story.

As were the other Niven / Pournelle collaborations. I can let the politics slide, since they were a product of Pournelle. Niven was more Liberal. The Mote in God's Eye is the greatest First Contact story I have ever read. The Moties were truly alien.

The Mote in God's Eye I likewise thought it very original, and the Moties truly alien. On a reread, it seemed as if they were modelled very much on Japanese samurai culture.

Nonetheless, it's a good read and I'd recommend it to anyone.

But not the book I'm looking for about one man walking home.

Good luck!

https://en.m.wikipedia.org/wiki/Apocalyptic_and_post-apocalyptic_fiction

Bookmarked it

but Summer of the Apocalypse by James van Pelt is almost that, except it takes place in Colorado, not California.

This description is taken from Amazon:

The stories of the two journeys, separated by more than half a century, is very well told.

His YA novel Pandora's Gun is excellent, and I understand he's planning a sequel.

Van Pelt mostly writes short stories and so far has five collections out there. I like what he writes. Here's a link to the online publication of one of my favorites. http://www.flashfictiononline.com/f20080301-just-before-recess-jim-van-pelt.html

He's also a really nice guy. I've met him at some science fiction things I go to.

https://www.amazon.com/Walk-Lee-Goldberg/dp/1453728988/ref=sr_1_9?s=books&ie=UTF8&qid=1526841172&sr=1-9&keywords=the+walk

That's it.
Much appreciated.