Female ginkgo trees are the ones that bear fruit and hence produce the offensive odor. But branches of a male ginko tree can (rarely) change sex and bare fruit.

I've never tried Ginko kernels roasted to the point they were crunchy. But I find the ones roasted (some times on a skewer) to the consistency of firm cooked lima beans have a flavor similar to lima beans. Preparing lima beans is much easier though they produce very different medicinal effects.