"I didn't really grow up with my dad," says Brody, who adds he didn't "really see much" of Caitlyn when she married into the Kardashian family.
During a segment on the show where he referred to Caitlyn by her birth name and gender she was assigned at birth, Brody spoke about their ups and downs over the years.
"I didn't really grow up with my dad," said Brody in conversation with Brandon Lee and Frankie Delgado. "When I was really little we got to spend a little bit of time [together]."
"When he started, obviously with the family, with the Kardashian family and all that, I didn't really see much of him," he continued. Caitlyn married Kris Jenner in 1991, they're parents to Kendall and Kylie Jenner.
In a confessional, Brody added, "Bruce was just never really around. Every so often, maybe once every couple years. He wasn't around for my graduation, he wasn't around for most of my birthdays. There really wasn't any relationship, it was very surface."
"Eventually, we were starting to become closer and then all of a sudden it's like [snaps fingers, as a photo of Caitlyn was shown] I was just getting to know Bruce and then Bruce became Caitlyn," Brody explained.
Speaking with his friends again, Brody said that "Caitlyn is a totally different person than Bruce. It's her true self but Bruce was always holding a lot back and being somebody totally different."
Once again talking to the camera about Caitlyn following her 2015 transition, Jenner said, "It's awesome, she's great, she's happy, she's off living her life, so what I've learned is to not expect too much from her."
Brody already came under fire for referring to Caitlyn as "he" when speaking of her past in the premiere, though his brother Brandon came to his defense. In an interview with Channel Q Radio, Brandon said people should "teach through encouragement" instead of "tearing somebody down."
"I do that even with my family -- when my brothers and sisters slip up here and there -- I do encourage them to use the pronoun that my dad would prefer," he said.
"Accidents happen and people slip up and you shouldn't shame them for it," Brandon added, "There's not an ill intention." Brandon also explained that Caitlyn told them all they could still refer to her as "Dad," telling them, "There's a part of me that relates to being your father at one point in my life."
"The Hills: New Beginnings" airs Mondays on MTV.