(Kadamba Kanana Swami, 25 December 2013, Mayapur, India, Srimad Bhagavatam 5.9.3)
Srila Prabhupada said that we can measure our advancement by how much we are becoming free from material desires. The tenth offence, in the matter of chanting the holy name, is to maintain material desires. Having material desires is one thing but we should not feed them. If we keep our material desires like a pet and you feed it, the thing becomes big and fat like some ugly fat little dog!
So, we do not feed our material desires, then they will go away very quickly. If you want to get rid of unwanted guest, then do not feed him and then he will go away very quickly. So the problem is not so much having material desires, the problem is feeding material desires.