But of course you are quite right if you mean that giving up fun for no reason except that you think it's "good" to give it up is all nonsense. Don't the ordinary old rules about telling the truth and doing as you would be done by tell one pretty well which kind of fun one may have and which not? But provided the thing is in itself right, the more one likes it and the less one has to "try to be good", the better. A perfect man would never act from a sense of duty; he'd always want the right thing more than the wrong one. Duty is only a substitute for love (of God and of other people), like a crutch, which is a substitute for a leg. Most of us need the crutch at times; but its idiotic to use the crutch when our own legs (our own loves, tastes, habits etc.) can do the journey on their own.
C S Lewis from Letters to Children
This same thought, although expressed with none of the same clarity, is found in Kant:
We have now to elucidate the concept of a will estimable in itself and good apart from any further end. This concept, which is already present in a sound natural understanding and requires not so much to be taught as merely to be clarified, always holds the highest place in estimating the total worth of our actions and constitutes the condition of all the rest. We will therefore take up the concept of duty, which includes that of a good will, exposed however, to certain subjective limitations and obstacles. These so far from hiding a good will or disguising it, rather bring it out by contrast and make it shine forth more brightly.
Immanuel Kant from the Groundwork of the Metaphysic of MoralsThis thought is important to any sound understanding of ethics and theology. Doing right is not the same as acting from a sense of duty. The motive to act rightly is only necessary when doing right isn't what we want to do. This makes deontological ethics - the view that there are real and fundamental duties that humans must obey - much less prissy and much more human. Sometimes we can be doing right just by having fun. Sometimes.
It is also important because it helps explain the connection between God as Law Giver and God as Lover. It is the same God, the God of the Old Testament and the God of the New. The Law is Love, but experienced as obligation. Both Will us to be the best we can be: Love and Discipline.
We are lucky if our circumstances and our nature mean that doing the right thing is fun - but sadly this is often not the case.