Ultrasonic to a phone or Alexa/Siri/etc, connect to an unsecured network, send data to a neighbor’s smart TV which is connected to Internet, Bluetooth or other to a phone
Because it would result in a scandal and it seems easily discoverable (by professional investigators/engineers). I don’t know. It’s likely done on a small, targeted, scale, but can’t imagine this rolled out on a large scale. Too little gain for the potential lashback, quite some factors need to be right, too.
You could be right, but I’m not alleging they would use cell.
Presumably the smart appliance already has wireless capabilities like WiFi/BLE. And then it’s just a software exercise on how to code an interface between devices of the same manufacturer.
That would add a ludicrous amount of cost to the device in both material cost and R&D. It’s so incredibly unlikely that any company would make that investment just to spy on the conversations of ordinary citizens when there are far cheaper and easier ways for them to build and sell advertising profiles.
Ultrasound is used by Microsoft teams, some apps use it to transmit data between phones. Back in the day there was a chrome app to transfer links.
Amazon sidewalk already connects devices together. Samsung Smart things already bridges Samsung devices. Apple Air tags already use “primary” Internet connected devices to transmit data about “secondary” devices.
It would show the encrypted out bound traffic right? You wouldn’t be able to identify it by reading the bits, but you could by the volume and not doing anything else.
Maybe. They might do some processing locally and just upload as text so it might be easy to batch the data, making the upload volume and pattern less obvious.
It also saves them network bandwidth so I’m sure that would motivate them too. Uploading raw mic data from all TVs would be expensive.
DNS will tell you the server name and address, which would just be some server owned by the company. Nothing weird there unless they have the chutzpah to name it something telling. They could even bypass DNS entirely with hardcoded IP addresses.
Timing wouldn’t be a great indicator either if they aggregate requests.
They could slide anything nefarious in with daily software update checks or whatever other phone-homing they normally do, and without deep packet inspection or reverse engineering the software, it would be very difficult to tell.
I don’t think Wireshark can do deep packet inspection, can it? Assuming the client is using SSL and verifying certs, maybe even using cert pinning?
Size would be a big indicator if they’re sending full voice recordings, but not if they’re doing voice recognition locally and only sending transcripts, metadata, or keywords.
I’ve never actually done this kind of work in earnest, and my experience with Wireshark is at least a decade out of date. I’m just approaching this from the perspective of “if I were a corporate shitbag, how would I implement my shitbaggery?”
Just spitballing here but you might be able to try and correlate the amount of data sent with how much real life activity there was. Say, have silence for a week around the TV then play recorded speech near it for a week and see if that changes the frequency or size of the data being sent back home. Then do this for random 1/2/3 day periods. If offline text to speech is as crap as I’ve heard then the increased data transfer should stick out pretty clearly.
That’s a completely unhinged level effort for what would still ultimately boil down to speculation lmao. Smart TVs phone home frequently, semi randomly, with varying data amounts, both when used regularly and when off for months at a time, both when you’re walking and talking around it, and if you’re on vacation for two weeks. If despite all that you tried to control the environment around it you’d somehow need to… ensure absolute silence in the room that it’s in for DAYS at a time? Unless you live in the middle of the woods that’s not very likely, and even then, all it would be is guessing lmao
First, someone would be able to prove that communication is happening. Second, if the keys are stored locally, and the original packets saved, the encryption can be reverse engineered.
Encryption prevents man in the middle attacks. If you have one of the ends, you can usually get the data. If you have the device that’s doing the encryption of the data, and you have the encrypted data, you can decode the data. It’s just a matter of getting through obfuscation at that point.
The reason this hasn’t been done yet is that it’s not happening yet. CMG was lying in their advertising.
Try it out. Setup dnsmasq and connect your phone to the network. You’ll see a ton of requests initially, that gives you some idea of what apps/services/accounts are on the phone. Let the phone go to sleep, and watch what is sending requests in the background. Many services use very specific host names which indicate what is being processed.
On the TV, it would be similar. You walk into the room and it starts sending packets? You say something unrelated to its trigger word yet Wireshark shows activity? Suspicious. If you can get a certificate onto the TV you can use mitmproxy to view the HTTPS traffic, but that’s probably kinda difficult.
I do not use smart TVs but I have been doing stuff like the above for a while. If they are recording and storing stuff some engineer eventually figures out, it’s not an NSA backdoor.
I’m not saying they are/aren’t, I do not know, it just seems very unlikely and improbable especially given smart phone ubiquity. What is known to be actually occuring is a complete violation of consumer privacy for marketing purposes, but OPs form of spying is so far unsubstantiated.
Now, can that TV be hacked and used by your neighbor to spy on you? Or can your government access your mic/camera? That’s an entirely different question and field of expertise.
“if I were a corporate shitbag, how would I implement my shitbaggery?”
In this case, it would be pretty hard. We have wiretap laws, which would mean you have to tell the user you’re doing this. Even though no one reads the ToS, someone does, and it would be news if someone was doing this.
Even then, it would be a hard enough problem that companies would think twice about it for a few reasons. Number one, processing 24/7 of all audio in your home is going to be rather difficult/expensive, so you’d have to go with something like keyword-triggers-processing the way that your phone listens for “hey google/siri” or Amazon listens for “Alexa.” It works kinda like game video sharing - they are always listening and recording for a short time frame* but they only send the data somewhere if they hear the trigger phrase. That’s not easy in itself, they’ve spent a ton of time getting the right algorithm so that it correctly hears the right trigger phrase and you don’t get a ton of false positives to varying degrees of success. And keeping in mind these are companies that are best suited to it, they still struggle sometimes with even that. The ad companies would have to listen for dozens/hundreds/thousands of triggers…
And then you get to the data retention policies. Google is an ad company, Apple is not. One of the reasons that Apple can tout privacy as a feature is simply that they don’t need the data, so they don’t collect nearly as much, and they save even less. They get the bonus of not dealing with law enforcement and all that.
So, assuming they solve that, solve some big issues with the laws of the land and physics, now we’re to the point where they have to think about network traffic. Which is going to be trivially easy for nerds to figure out and circumvent, so they would have to have their own ad-hoc network which comes with another 137 or so difficulties.
I dont add it [edit: smart tv] to the wifi or drop a cat 5 cable to it and my smart phone will still see it in the house and ask if I want the two devices to connect. I miss when TVs were a bit thicker and easier to take apart so you could easily take out the wifi and Bluetooth cards.
If it were, it would be pretty common knowledge and there would be several news cycles about it. I don’t doubt that they could bury it in the terms of service, but we have wiretap laws in enough places that are two-party consent that it would have had to come out by now. Not to mention nerds like me running pi-hole and monitoring their traffic, repair people who could easily regonize a mic in the device, etc.
The privacy agreement in them covers it, just like Alexa.
Check yours, if you don’t agree to the privacy agreement, things like cable and broadcast channel recognition don’t work.
It also breaks Automatic Content Recognition, which enables the manufacturer to monitor what you’re watching.
Granted that’s not the same as listening, but it’s close enough. And we know Google employees have been caught listening/watching people. There was another article just the other day of another company caught doing the same.
Just because something’s illegal doesn’t stop people from doing it.
As for catching it with monitoring… We know Microsoft has hard coded domain names into certain DLL’s since XP, so you can’t block the domains with a hosts file. There’s some talk in the Pihole community about smart tv’s being able to bypass your DNS with hard-coded IP destinations - they only need one to be able to then deliver their own DNS.
Some smart TV’s will connect to others via wifi if they don’t have connectivity, yet another way to bypass our efforts to block their connections.
That manufacturers are so blatantly adversarial makes it pretty clear they’ll try to get away with anything they can. And anything I can think of, surely their dedicated teams of engineers thought of it long before me.
Edit: then there’s apps like Netflix, Prime, Peacock, Hulu, YouTube, etc, that make encrypted connections to home. It would be trivial to permit those apps to deliver alternative name resolution for the entire OS on TV’s since we don’t control the OS.
I’m confident this is built in to many smart TVs these days.
Well. Wireshark would confirm that if it were true.
I’m sure it will show HTTPS traffic outbound from your TV.
I’m sure it will show no traffic whatsoever if you don’t connect your TV to your network
deleted
Source?
Either way, open networks are very uncommon in residential areas (and honestly in general)
deleted
Source that it happens obviously.
You claimed that they connected to open networks.
deleted
There’s a dozen ways they could jump the air gap.
Ultrasonic to a phone or Alexa/Siri/etc, connect to an unsecured network, send data to a neighbor’s smart TV which is connected to Internet, Bluetooth or other to a phone
But this would be proven then?
Something that can be done easily and may be done in the future, if it hasn’t been discovered yet
Clandestine methods have been known since the 2000s. We know they’re scummy and want our data. Why does this seem too crazy?
Because it would result in a scandal and it seems easily discoverable (by professional investigators/engineers). I don’t know. It’s likely done on a small, targeted, scale, but can’t imagine this rolled out on a large scale. Too little gain for the potential lashback, quite some factors need to be right, too.
The economics aren’t there. A cellular chip and a subscription will not pay for the private conversations of a random house.
You could be right, but I’m not alleging they would use cell.
Presumably the smart appliance already has wireless capabilities like WiFi/BLE. And then it’s just a software exercise on how to code an interface between devices of the same manufacturer.
deleted
That would add a ludicrous amount of cost to the device in both material cost and R&D. It’s so incredibly unlikely that any company would make that investment just to spy on the conversations of ordinary citizens when there are far cheaper and easier ways for them to build and sell advertising profiles.
Ludicrous R&D?
Ultrasound is used by Microsoft teams, some apps use it to transmit data between phones. Back in the day there was a chrome app to transfer links.
Amazon sidewalk already connects devices together. Samsung Smart things already bridges Samsung devices. Apple Air tags already use “primary” Internet connected devices to transmit data about “secondary” devices.
None of this is new tech, it’s all feasible.
its not unlikely, devices were already shown doing shit like this
Low-bandwidth cellular chip…
At that point the customer acquisition cost is t worth it.
It would show the encrypted out bound traffic right? You wouldn’t be able to identify it by reading the bits, but you could by the volume and not doing anything else.
Maybe. They might do some processing locally and just upload as text so it might be easy to batch the data, making the upload volume and pattern less obvious.
It also saves them network bandwidth so I’m sure that would motivate them too. Uploading raw mic data from all TVs would be expensive.
You’re getting down voted, but this seems the most likely. TTS is trivial anymore.
And with DNS requests and timing you should be able to figure whats in those packets.
Sorry if this is a noob question, but…how?
DNS will tell you the server name and address, which would just be some server owned by the company. Nothing weird there unless they have the chutzpah to name it something telling. They could even bypass DNS entirely with hardcoded IP addresses.
Timing wouldn’t be a great indicator either if they aggregate requests.
They could slide anything nefarious in with daily software update checks or whatever other phone-homing they normally do, and without deep packet inspection or reverse engineering the software, it would be very difficult to tell.
I don’t think Wireshark can do deep packet inspection, can it? Assuming the client is using SSL and verifying certs, maybe even using cert pinning?
Size would be a big indicator if they’re sending full voice recordings, but not if they’re doing voice recognition locally and only sending transcripts, metadata, or keywords.
I’ve never actually done this kind of work in earnest, and my experience with Wireshark is at least a decade out of date. I’m just approaching this from the perspective of “if I were a corporate shitbag, how would I implement my shitbaggery?”
The answer is: it wouldn’t. You’re right on the money, you couldn’t do anything other than speculation.
Just spitballing here but you might be able to try and correlate the amount of data sent with how much real life activity there was. Say, have silence for a week around the TV then play recorded speech near it for a week and see if that changes the frequency or size of the data being sent back home. Then do this for random 1/2/3 day periods. If offline text to speech is as crap as I’ve heard then the increased data transfer should stick out pretty clearly.
That’s a completely unhinged level effort for what would still ultimately boil down to speculation lmao. Smart TVs phone home frequently, semi randomly, with varying data amounts, both when used regularly and when off for months at a time, both when you’re walking and talking around it, and if you’re on vacation for two weeks. If despite all that you tried to control the environment around it you’d somehow need to… ensure absolute silence in the room that it’s in for DAYS at a time? Unless you live in the middle of the woods that’s not very likely, and even then, all it would be is guessing lmao
Oh entirely, but it’s the best I could come up without disassembly. (And I’m fairly sure I’ve done worse debugging a prod environment)
First, someone would be able to prove that communication is happening. Second, if the keys are stored locally, and the original packets saved, the encryption can be reverse engineered.
Encryption prevents man in the middle attacks. If you have one of the ends, you can usually get the data. If you have the device that’s doing the encryption of the data, and you have the encrypted data, you can decode the data. It’s just a matter of getting through obfuscation at that point.
The reason this hasn’t been done yet is that it’s not happening yet. CMG was lying in their advertising.
Try it out. Setup dnsmasq and connect your phone to the network. You’ll see a ton of requests initially, that gives you some idea of what apps/services/accounts are on the phone. Let the phone go to sleep, and watch what is sending requests in the background. Many services use very specific host names which indicate what is being processed.
On the TV, it would be similar. You walk into the room and it starts sending packets? You say something unrelated to its trigger word yet Wireshark shows activity? Suspicious. If you can get a certificate onto the TV you can use mitmproxy to view the HTTPS traffic, but that’s probably kinda difficult.
I do not use smart TVs but I have been doing stuff like the above for a while. If they are recording and storing stuff some engineer eventually figures out, it’s not an NSA backdoor.
I’m not saying they are/aren’t, I do not know, it just seems very unlikely and improbable especially given smart phone ubiquity. What is known to be actually occuring is a complete violation of consumer privacy for marketing purposes, but OPs form of spying is so far unsubstantiated.
Now, can that TV be hacked and used by your neighbor to spy on you? Or can your government access your mic/camera? That’s an entirely different question and field of expertise.
More info
In this case, it would be pretty hard. We have wiretap laws, which would mean you have to tell the user you’re doing this. Even though no one reads the ToS, someone does, and it would be news if someone was doing this.
Even then, it would be a hard enough problem that companies would think twice about it for a few reasons. Number one, processing 24/7 of all audio in your home is going to be rather difficult/expensive, so you’d have to go with something like keyword-triggers-processing the way that your phone listens for “hey google/siri” or Amazon listens for “Alexa.” It works kinda like game video sharing - they are always listening and recording for a short time frame* but they only send the data somewhere if they hear the trigger phrase. That’s not easy in itself, they’ve spent a ton of time getting the right algorithm so that it correctly hears the right trigger phrase and you don’t get a ton of false positives to varying degrees of success. And keeping in mind these are companies that are best suited to it, they still struggle sometimes with even that. The ad companies would have to listen for dozens/hundreds/thousands of triggers…
And then you get to the data retention policies. Google is an ad company, Apple is not. One of the reasons that Apple can tout privacy as a feature is simply that they don’t need the data, so they don’t collect nearly as much, and they save even less. They get the bonus of not dealing with law enforcement and all that.
So, assuming they solve that, solve some big issues with the laws of the land and physics, now we’re to the point where they have to think about network traffic. Which is going to be trivially easy for nerds to figure out and circumvent, so they would have to have their own ad-hoc network which comes with another 137 or so difficulties.
That’s not how that works lol
I dont add it [edit: smart tv] to the wifi or drop a cat 5 cable to it and my smart phone will still see it in the house and ask if I want the two devices to connect. I miss when TVs were a bit thicker and easier to take apart so you could easily take out the wifi and Bluetooth cards.
If it were, it would be pretty common knowledge and there would be several news cycles about it. I don’t doubt that they could bury it in the terms of service, but we have wiretap laws in enough places that are two-party consent that it would have had to come out by now. Not to mention nerds like me running pi-hole and monitoring their traffic, repair people who could easily regonize a mic in the device, etc.
The privacy agreement in them covers it, just like Alexa.
Check yours, if you don’t agree to the privacy agreement, things like cable and broadcast channel recognition don’t work.
It also breaks Automatic Content Recognition, which enables the manufacturer to monitor what you’re watching.
Granted that’s not the same as listening, but it’s close enough. And we know Google employees have been caught listening/watching people. There was another article just the other day of another company caught doing the same.
Just because something’s illegal doesn’t stop people from doing it.
As for catching it with monitoring… We know Microsoft has hard coded domain names into certain DLL’s since XP, so you can’t block the domains with a hosts file. There’s some talk in the Pihole community about smart tv’s being able to bypass your DNS with hard-coded IP destinations - they only need one to be able to then deliver their own DNS.
Some smart TV’s will connect to others via wifi if they don’t have connectivity, yet another way to bypass our efforts to block their connections.
That manufacturers are so blatantly adversarial makes it pretty clear they’ll try to get away with anything they can. And anything I can think of, surely their dedicated teams of engineers thought of it long before me.
Edit: then there’s apps like Netflix, Prime, Peacock, Hulu, YouTube, etc, that make encrypted connections to home. It would be trivial to permit those apps to deliver alternative name resolution for the entire OS on TV’s since we don’t control the OS.