"Any time you lose a loved one, especially when you're younger, it alters the trajectory of your life," Anderson told PEOPLE.
"It's inconceivable to me that my brother died in 1988 and I've lived more of my life without him than I have with him," he continued. "There's not a day that goes by that I don't think about what he would be doing, who he would be, and I still think about his death and have questions about it."
The CNN anchor revealed that his mother Gloria continued to be affected by Carter's death until she passed away at the age of 95 from stomach cancer in 2019.
"It was the same with my mom," Anderson explained. "Till the day she died (in 2019), we were both still stunned by what happened. There are some things that never can be answered and you have to find a way to live in that space of not knowing, or not fully understanding."
Ten years prior to Carter's passing, in January 1978, the brothers also lost their father, Wyatt Emory Cooper, who died during open heart surgery. Anderson's son Wyatt -- whom he welcomed via surrogate in April 2020 and co-parents with his ex-partner Benjamin Maisani -- is named after his late grandad.
The "60 Minutes" co-host said that his upcoming book, "Vanderbilt: The Rise and Fall of an American Dynasty," is a "letter" to his son.
"My dad wrote a book before he died (during heart surgery when Anderson was 10) about his family growing up in Mississippi," Anderson told PEOPLE. "And because he died when I was so young, a lot that I know of him came from that book. I wanted to write a letter to Wyatt about this crazy and unusual part of his family's past."
The book -- which Anderson co-wrote with historian and novelist Katherine Howe -- "explore[s] the story of his legendary family and their outsized influence" and "chronicles the rise and fall" of the Vanderbilt dynasty.
"Vanderbilt: The Rise and Fall of an American Dynasty" is set to be released on September 21.
If you or someone you know is struggling with depression or has had thoughts of harming themselves or taking their own life, get help. The National Suicide Prevention Lifeline (1-800-273-8255) provides 24/7, free, confidential support for people in distress.