The comedian also shares what he considers really "scary" about the Trump presidency.
"I'm not mad at Kanye. That's my brother -- I love him, I support him," Chappelle told CNN's Van Jones in an interview tape before the rapper's wild Oval Office meeting with Trump on Thursday. "But, you know, I don't have to agree with everything that he says."
When Jones brought up the controversial hip-hop star, Chappelle stood right up and threatened to walk out of the studio as a joke.
"I just trust him as a person of intent," the comedian added. "But yeah, he shouldn't say all that shit."
Kanye has been publicly supporting Trump since the former "Apprentice" star beat Hillary Clinton in the 2016 presidential election. He visited the real estate billionaire in Trump Tower shortly after the unexpected election results, and has been vocal about his love for Trump ever since. On Thursday, he talked the president's ear off for 10 minutes inside the Oval Office, unleashing another one of his signature rambling rants that we've previously seen on "TMZ Live" and "Jimmy Kimmel Live!"
"I tell you what, that was pretty impressive. That was quite something," the president said after Kanye was done expressing himself.
Chappelle praised Kanye as "a genius" and found the silver lining in the rapper's often muddled message.
"I think the angle he's seeing things from is about the division that he sees. And -- and he's not inconsistent with what he's saying," Chappelle said, and cited Kanye once encouraging African Americans to reappropriate the Confederate flag.
In the middle of his answer to a question about his stance on Kanye, the comedian said he's scared of what America will be like after Trump leaves office.
"The thing that's scary about this presidency is after it. I don't know if you've been married before or had a girlfriend and said something in a fight that was so wrong," he said. "And then after that ... we're still family, we're still around each other. But man, I sure did say all that shit, didn't I?"
Dave Chappelle can currently be seen in theaters again alongside Bradley Cooper and Lady Gaga in "A Star Is Born."