"What really bothered me about that conversation was, after I left that conversation I felt like it wasn't productive," he began. "I have no problem talking to people from the other side but I just have to feel like that's really your truth. I don't know if I got that from Rush, it felt like more of a performance."
Colbert interjected, saying he felt the same way about Donald Trump when he interviewed him.
Charlamagne then went on to say he listened to Rush "eloquently" describe white supremacy and white privilege to a fan on Rush's own show later in the day on Monday, which surprised Charlamagne as Rush wouldn't acknowledge either of those concepts during their talk.
"It didn't feel productive, it felt like a waste of my time," Charlamagne said.
During Monday's exchange, Rush said, "I don't buy into the notion of white privilege. I think that's a liberal, political construct right along the lines of political correctness."
"It's designed to intimidate and get people to shut up and admit they're guilty of doing things they haven't done. I don't have any white privilege."
Charlamagne had fired back, "You're being delusional."
Their conversation came after protests have erupted across the country denouncing police brutality in the wake of Floyd's death, who died while being forcefully detained by a white police officer in Minneapolis, Minnesota on May 25.
On Colbert's show, Charlamagne said the protests didn't surprise him.
"I always say that right now, America's being exposed for the illusion it's always been," he explained. "America advertises itself as an all-inclusive resort, but it's really a private club founded by old rich white men for rich white men, and they make the rules."
"We're all caught up in this web of white supremacy and nothing is gon' get us out of that web until you have more privileged white men who are willing to dismantle that system."
On Friday, Derek Chauvin was arrested after video captured him with his knee on Floyd's neck for more than eight minutes during Floyd's arrest for allegedly passing a counterfeit $20 bill. Floyd was unresponsive when paramedics arrived and was later pronounced dead.
Chauvin faces 2nd-degree murder and 2nd-degree manslaughter charges.
The three other officers in the arrest -- Thomas Lane, J.A. Keung and Tou Thao -- were charged with aiding and abetting the homicide.
On Tuesday, the state of Minnesota filed a human rights complaint against the Minneapolis Police Department.