It has everything to do with mockery, bullying and him eating so much food when he got to London at age 18.
The 22-year-old Grammy winner tells Australia's '60 Minutes' that people calling him 'fat' was more hurtful than the gay slurs.
"If someone called me fat, that affects me way more than someone calling me a f----t," he admits, referring to his sexuality. "I think just because I've accepted that, if someone calls me a f----t, it's like, I am gay and I'm proud to be gay so there's no issues there. If something calls you fat, that's something I want to change."
Sam however blamed his weight gain on loneliness stating that: "I genuinely think I gained about two stones [28 pounds] when I moved here [London]. I just get loads of chicken and watched Lost every night after work."
He jokingly added that the title of his critically-acclaimed album, "The Lonely Hour, came from that moment of his life. But insists food got him fat: "When I was at school and wasn't having a great time or when music wasn't going very well. I would eat, eat... Eating would make me feel better, when I felt lonely I would eat."
These days, the "Stay With Me" singer is looking better than ever. Last week, he shared that he's only four pounds away from his weight goal.