Travis Kelce and Taylor Swift have been the talk of pop culture in the United States over the past nine months, forging a romance that transcends boundaries between sports and music and appeals to a very broad demographic. But being in the spotlight all the time comes with certain disadvantages.

Kelce has since become more keenly aware of the trappings of fame since he embarked on his romance with Swift, who has spent much of the past year on her worldwide Eras Tour. Kelce, the star tight end for the two-time defending Super Bowl champion Kansas City Chiefs, divulged these surprising details during a recent podcast appearance.

Kelce and Swift’s problems with privacy

During a conversation on the “Bussin with the Boys” podcast, Kelce said that the hardest thing about dating a global superstar like Swift is the wave of fans and admirers who have shown up to his home, just to catch a glimpse of the couple.

“That’s been the craziest (aspect of the relationship), definitely the pulling up to the house,” Kelce said.

A source contacted by Us Weekly shed more light on the situation, and they described Kelce as “freaked out” when new fans did not observe his privacy.

“People would show up at his house and leave things on his doorstep. … He didn’t feel safe,” the source said.

But such is life when dating a woman who is arguably the world’s most famous. Travis‘ brother Jason revealed earlier this year that the Chiefs star eventually had to abandon his house entirely, but fortunately, it hasn’t threatened the couple’s relationship — which became public knowledge nearly a year ago, at this point.

“I’ve had fun with just about every aspect of it,” Kelce said about dating the 14-time Grammy-winning singer-songwriter. “It’s just, when you’re at home, you want privacy, and you don’t always get that.”