Johnson was voted off the CBS show, which takes place this season in Nicaragua, in an episode that aired on Wednesday night.
"I had fun, but I was miserable the whole time," Johnson told The Associated Press.
"I still love the game, and it's been a great adventure. This is the most stressful time I've ever gone through in my life, and I'm including Super Bowls and collegiate national championships. I initially said, 'Keep your strongest members.' I obviously wasn't one of the strongest members."
This season has seen the tribes divided by age. The 67-year-old coach had long been a fan of the show and had lobbied hard for a chance to compete. But Johnson wasn't able to lead his tribe to victory in an immunity challenge that involved rolling barrels along the beach then landing sand bags on top of them.
Contestant Marty Piombo, led the campaign to oust Johnson, saying he needed "to remove him so people will lose their daddy."