Paul Shaffer is ready to disband 'Late Night's' CBS Orchestra

This was such a long run, so if I'm meant to get a gig in a piano bar in Palm Springs .... I'm not too worried at this point," he says. "I feel like I've had a lot of fun. How can you keep having all the fun? Eventually, you've got to let others have some fun."

Hidden in the attention being paid to David Letterman leading to his retirement Wednesday is the knowledge that it will also mean the end of a regular television role for America's bandleader, Paul Shaffer.

The gravel-voiced sidekick, who's 65, isn't ready to leave show business even with the gig of a lifetime coming to a close after 33 years.

"Of course, I had the old mixed feelings, sadness, etcetera," Shaffer says during an interview in his office behind the Ed Sullivan Theater. "Now I have come around to just being absolutely thankful for such a wonderful run, such a long run, working for a guy who has been nothing but encouraging to me."

Shaffer is a walking trivia answer of show biz credits: bandleader for the original "Saturday Night Live" troupe, same role for the Blues Brothers, part composer of the 1980s hit "It's Raining Men" and the "Late Show" theme, cringe-worthy record executive Artie Fufkin in "This is Spinal Tap" and music director for the annual Rock and Roll Hall of Fame inductions. If he hadn't neglected to return Jerry Seinfeld's call, you may have known him as George Costanza, too.

He'll always be linked in public consciousness with Letterman, however, like Ed McMahon and Johnny Carson, or Andy Richter and Conan O'Brien.

Until Shaffer interviewed for the bandleader job that began in 1982, he had never met Letterman. Shaffer was asked what he saw for Letterman's post-midnight "Late Night" on NBC, keeping in mind they could only afford a few musicians. He envisioned something like the lounge bands he saw at the beginning of his career in Toronto, a lead keyboard player and a couple of other instruments, interpreting Stax and Motown hits.

In this 1986 photo, bandleader Paul Shaffer directs his band during a rehearsal for "Late Night with David Letterman" in New York. At far right is a recent photo of Shaffer, now 65 years old.

Letterman says that sounded great, that he'd always thought of himself as the Wayne Cochran of comedy. Shaffer cracked up at the reference to an obscure 1960s soul singer who looked a little like Jay Leno with an oversized, platinum blond pompadour.

The time slot after Carson "sounded like the hippest opportunity, like in Las Vegas when a performer would do a late late show for the other performers or cab drivers on the strip. That's how I related to it. It was all perfect for me."

Starting at a time when his five years at one job, on "Saturday Night Live," seemed like an eternity, Shaffer says he never felt tempted to leave Letterman. He's had the freedom to do other things while the "Late Show" let him lead a band -- perform in front of an audience, do sketch comedy and match wits with TV's hottest host. That's not to say it was always easy.

"I spent time preparing funny, off-the-wall lines, something Jerry Lewis had says, for example," he says. "I was doing quite well with it, I thought. (Letterman) says, 'I would rather we just have a conversation and try to talk.' Well, that was daunting to me, but I did. That's when I began to see what he really needed from me."

Through the years, Shaffer has become like a security blanket to Letterman. As anyone who's been in his frigid studio knows, he's a host who likes things Just So. Shaffer will interject quick remarks -- "Instagram that right away," he says after his boss took a selfie Tuesday night -- or lead the band into a snippet of "Tequila" for a Top Ten list about Cinco de Mayo parties. Sometimes it's as simple as breaking the silence, an "ahh" or slight cackle.

Letterman has a habit of calling a pre-show meeting just when there's too little time to seriously discuss things. And after his heart bypass surgery a decade ago, Letterman stopped rehearsing regularly.

"The show got way more fun at that point, way more spontaneous," Shaffer says.

The bandleader worked with plenty of heroes; James Brown asked to come on after he heard Shaffer's band playing some of his music on the show. This added another, delicate task to his duties: it was Shaffer's job to sometimes tell performers they'd have to cut a portion of their song because of television time constraints.

"I've gotten better at what I do over the years, if I may humbly say," he says. "When I started I may not have been able to work with artists as well as I do now. If you work with people, you get to understand what they need, when they need it, what you should or shouldn't say."

When the CBS Orchestra, expanded beyond the bare-bones band Shaffer had in the NBC days, disbands, Shaffer has his options; he plays a little jazz and wants to keep in comedy. Maybe a TV drama will offer a story arc. The biggest adjustment will be the loss of the routine, knowing where he had to be on weekday afternoons.

"This was such a long run, so if I'm meant to get a gig in a piano bar in Palm Springs .... I'm not too worried at this point," he says. "I feel like I've had a lot of fun. How can you keep having all the fun? Eventually, you've got to let others have some fun."