I understand where they're coming from from a biological perspective. I once heard it explained as 'humans are the only animals who drink the genetic material of other animals'. I'm not sure about that, because I figure eating another animal's meat or raw eggs also qualifies as eating genetic material, but yeah, humans are the only animals that still drink mother's milk as adults, and drink it from other animals, to boot.
Then again, as I stated earlier, humans have been doing this for thousands of years. I'm pretty sure that, if there are any ill effects, they're negligible.
Also, I just love milk too much to give it up for a 'possible decreased health risk'.