Depression, stress, alcoholism, illicit drug use and fatigue often can be factors in loss of sex drive in men. Sometimes the culprit is a decrease in male sex hormones due to an endocrine disorder. In other cases, loss of sex drive may be a medication side effect.
