At 73, James Patterson has sold more than 425 million copies of his 200 novels globally, making him one of the highest paid authors in the world.

Patterson had annual earnings topping more than $80 million between June 2019 to June 2020 and sold nearly 5 million in books in the U.S. during that time, according to Forbes.

In July, Patterson signed a multi-project deal with Amazon's Audible for an undisclosed amount, with his first Audible Original, "The Coldest Case" out Thursday.

"I do not work for a living, I play for a living. I love doing it," Patterson tells CNBC Make It.

But Patterson's road to success didn't happen overnight. He started writing as a side hustle and he faced a lot of rejection before getting his first book published.

Patterson grew up in the woods in Newburgh, New York with an insurance salesman father and a school teacher mother. Despite having dreams of being a writer, the idea "always seemed presumptuous," Patterson says.

But after reading books by James Joyce, "it really turned me on to reading, and then I started scribbling," he says.

After college, Patterson moved to New York and got his first job as a junior copywriter at the advertising agency J. Walter Thompson in the early 1970s. While climbing the corporate ladder, Patterson wrote his first book, "The Thomas Berryman Number" in the mid-70s.

The book "got turned down by 31 publishers," Patterson says.

But after it was finally published in 1976, "then it won an Edgar Award for best first mystery, so go figure that one out." (The Edgar Award, which is named after Edgar Allan Poe, honors the best writers in mystery fiction and non-fiction.)

Despite publishing his first book at age 26, Patterson didn't have a bestseller until he was 40. "It wasn't like all of a sudden," he says.

In fact, Patterson kept his day job for decades and ascended to CEO of J. Walter Thompson in 1988.

During that time, Patterson says he would read well over 100 books a year to help spark his creativity.

"I have a huge imagination and I'm constantly [asking], what about this? What about that?" he says.

Nowadays, due to his hectic writing schedule, he reads a book a week, he says.

Despite his success, Patterson says he doesn't like to give advice, especially about writing. But he credits his own success to his work ethic and passion.

"I also don't take myself too seriously," he says. "Yeah, I sell a lot of books, so what?"

Don't miss:

The best credit cards for building credit of 2021

From growing up in poverty to battling the Covid pandemic: How Northwell Health’s CEO found success