remove detect_math_libs (#4383)

Darwin may not require libm, but it doesn't hurt to link it and simplifies configuration logic.
1 file changed