I do. I care about keeping my cute little hardware thingies safe! It would be cruel and unjust and evil and vile to allow any harm to befall a USB device.

To be honest,
I actually don't.

I've never had any problems caused by removing a USB device without using that function.

But, I know how to tell if a device is inactive without using that function.

I'm mildly freaked out by the thought of the slim chance of accidentally harming a flash drive by not doing the safely remove thing, so pretty much yes - but it depends on the computer. I haven't seen this on anything newer, but I've dealt with computers where safely removing hardware disabled that entire USB port until you restarted the computer. (Wasn't a matter of picking the wrong thing to safely remove, for the record, though that can do that too.)
---
Edited by: Miles, Oct 27th, 2015 @ 10:41 am