Just over half of Amazon Fresh stores are equipped with Just Walk Out. The technology allows customers to skip checkout altogether by scanning a QR code when they enter the store. Though it seemed completely automated, Just Walk Out relied on more than 1,000 people in India watching and labeling videos to ensure accurate checkouts. The cashiers were simply moved off-site, and they watched you as you shopped.
I used to work at Amazon on this, and I know A LOT about how this technology works. If I remember correctly, JWO began to separate from Amazon Fresh back in late 2022 when they told us they would cancel JWO in the new Amazon Fresh upcoming stores and use their new dash carts instead. They told us that the focus would instead be selling JWO tech to third party vendors, such as stores in stadiums and airports.
I know (somewhat outdated) about how most of their tech works, as well as some of the internally-stated reasoning behind their business decisions. AMA lol
One thing I know for a fact that is probably the most troubling/dystopian, is that the technology absolutely exists and works pretty well to be able to track the movements of every individual person in a crowd, including by having multiple camera angles as a person moves in and out of view. Drones, satellites, and street cameras can absolutely be used to do this on a fairly large scale, and I have to assume it is being done. Amazon’s technology in particular isn’t exactly doing this and I don’t necessarily think they are providing it to 5 eyes or whatever, but I know for a fact it can and is being done at least for some BS convenience stores.
Probably schools tbh, there have been a lot of papers floated about using a camera and AI to tell if students are paying attention or cheating or whatever. There’s already exammonitor & others using webcams so it’s not a huge leap to make.
How do the cost savings stack up? Checkout staff are not paid very well and most stores have dumped most of the labour onto consumers with self checkout anyway
It’s super expensive, prohibitively so for now although they might actually bring down the cost. But they’re allegedly finding that sales are actually higher in their in-stadium stores compared with the alternative, so it might actually have a legitimate advantage there. Plus they mostly sell overpriced beer there, so it’s not totally crazy that it could be profitable.
so what did they do if you walked in, didn’t scan your qr code, and walked out? how did they think they were going to separate the two groups well enough to prevent theft? like the whole concept screams “figure out how to steal in full view of the entire store”. like one wig, a little costuming, and the cops are never going to find you from the video footage.
I spent a decent amount of time thinking of how best to shoplift from there, lol. My best idea is that with some decent slight of hand, you could easily fool the system into thinking you took 1 product instead of 2, or you returned an item when you actually didn’t. And then you can challenge your receipt to make them fix it. There are a lot of camera angles, but they are only 14fps, and it can still be pretty tough to tell what someone is actually doing even if they are acting totally normally and non-maliciously.
Another funny theory I had is that to beat the tracking system, you could lie down or something. It’s not something I ever tested in practice, but their tracking system relies heavily on the assumption that everyone is standing upright, so maybe if you rolled on the floor a little it would get confused. But when it gets confused it goes to one of those low paid workers in India or Costa Rica, so it might get corrected manually.
Horse costume for 2 people
yeah, the whole thing just screams “find an attack vector”. did they actually red team it?
I mean, they test it a lot. It seems like the general philosophy though was that the ways of fooling the system reliably are somewhat convoluted, so if someone is willing to do all that, just let them lol.
Amusingly, one of the particular vulnerabilities that they mentioned having problems with in the UK in particular was people just brazenly going into a store, taking down some cameras or other equipment (networking equipment, edge compute, etc.) and Just Walking Out.
It does seem like an easy way to do this is just get your buddy to shine a laser pointer at the camera while you grab the stuff.
Any given shelf is generally visible by like 6-8 cameras.
Oh, I see. That’s fucked. Over here stores have like…3 cameras total? Giver or take? Positioned in a way that multiple aisles are covered by a single camera. We don’t really have a lot of fearmongering about shoplifting though.
It’s not really about shoplifting, the cameras are a core component of the system, and if they could use fewer they would. Say what you will about the concept, but the cameras themselves aren’t really the insidious part, in my opinion.
The stores have gates that prevent entry until you scan the code. You could maybe hop the turnstyle, but there’s usually at least one employee near the front to “help people get in”. And to be fair, lots of people have questions about it so it’s not entirely bs, but I guess their job is also to dissuade this.
oh they’re walk out only? that’s bizarre.
I mean, to scan in is still really quick, it’s like going into a subway or something.
Wouldn’t this turn the store into a ‘stealing minigame’?
I mean yeah, you can try, but it’s actually kind of difficult. It’s way easier to steal from any conventional store, probably.
yeah, that makes sense. I guess it just feels like a dare.
A price tag on anything in any store is a dare imo. You just have to weigh the likelihood of succeeding if you accept the dare and what the stakes are if you fail.