The demise of their 13-year marriage came up after the radio host wondered, after seeing how much tabloid attention his relationship with Jennifer Lopez earned him the first time around, whether Affleck had any concerns about dating her again -- especially now that he's a father this time.
"It crossed my mind for sure. My responsibility to my children is the highest responsibility I have," he explained. "I don't want to do anything that is painful or destructive to them if I can help it."
"That being said, my life affects them. I've already inflicted that on them. Me and their mom are celebrities and that's hard," he added. "Let's not bulls--- each other. That is a cross to bear. That's an albatross already."
After saying that there were a lot of "f---ing horrible lies" printed about him amid the couple's divorce, calling them "bulls---," he explained what went wrong.
"The truth was we took our time, we made the decision ... we grew apart. We had a marriage that didn't work. This happens. It's somebody I love and respect but to whom I shouldn't be married anymore," said Affleck. "We tried, we tried, we tried, because we had kids. Both of us felt like, we don't want this to be the model that our kids see of marriage."
Saying that he'd "probably still be drinking" if he and Garner were still together, the now-sober star added, "Part of why I was drinking alcohol is I was trapped. I can't leave because of my kids, I'm not happy, what do I do?"
"What I did was drink a bottle of scotch and fall asleep on the couch," he answered, adding, "that turned out not to be the solution."
Going back to Garner, he added, "We did our best. Did we have moments of tension? Did we have disagreements over custody? Was stuff difficult for us? Did we get angry? Yes, but fundamentally it was always underpinned with respect. I knew she was a good mom and I hoped she knew I was a good dad. I knew I was. I had to get sober, I knew that too and I acknowledge that."
Stern asked whether Affleck was ever suicidal during any of their relationship drama and subsequent drinking -- with Ben saying he never was. "I have had depression. I've been on anti-depressants for 25 years" he added, "I really believe in them."
The host also wondered if it has been hard for Affleck to stay sober since.
"It depends. At the beginning, it's very, very hard," he said, before saying that, for him, "the cure for addiction is suffering."
"You suffer enough before something inside you goes, 'I'm done.' I'm lucky, because I hit that point before I lost really the things that were the most important," he continued. "When I felt as if it impacted [my kids], I recognized it, it was the worst day of my life. I made amends and for a while, I thought, maybe this is temporary, maybe this feeling's gonna go away. But since that day, I swear to Christ, I have not ever wanted to drink once. Ever. So it's the easiest thing in the world."
Admitting he's "not cured" and joking he's definitely "not a preacher," Affleck added that "there is a happy ending if you can get there." He said his alcoholism isn't something for which he's ashamed -- and concluded, "I don't even think it's that interesting."
Affleck and Garner were married from 2005-2018 and share three children; Violet, 16, Seraphina, 12, and son Samuel, 9. Following Lopez's split from Alex Rodriguez, she and Affleck rekindled their romance -- after previously dating back in the early 2000s.