I understand why this bothers some people, but no, I’m not one of them. At some point, you have to decide if there’s some nefarious nature to the technology you invite into your life.
I tend to sway towards the side of these gadgets potentially making my life better (even if just by way of convenience) rather than making it worse (through, again, some nefarious means or even just bad security or the like). But I know this bothers many people still. My guess is that right or wrong, society gets more used to the notion of these open mics around us over time.