The actor recently underwent open heart surgery and died from complications from the operation, according to TMZ.
Avery's wife Barbara had been at his bedside all day at the hospital, but left for a short time to grab some food. When she returned, she learned he had passed away.
Alfonso Ribeiro, who played Avery's son Carlton on the popular sitcom, remembered the late actor this morning on Facebook.
"The world has lost a truly special man," he wrote. "I am very saddened to say that James Avery has passed. Even though he played my father on TV, he was a wonderful father figure to me in life. He will be deeply missed."