For me I think it was that "cool" ingredient or effect that didn't agree with me. I can smell the nice sandalwood musk underneath it, but something goes a bit perfume-y on me, wonder if it has a flower note? For me what makes a perfume smell old ladyish is when they are heavy on the florals, especially rose, it reminds me of those old school perfumed talcs that my grandma used to wear, or those heavy perfumed creams. But different stokes for different folks, there's beauty in having different tastes.
I LOVE the idea of a sweet musk, sandalwood, vanilla and sugar though, That's probably going to be my first PE. Maybe add a bit of Tonka bean and/or sweet pipe tobacco.