Earlier this week Amazon announced Amazon Go, its store with automatic checkout. I immediately wondered how they are going to do that. In the past, the concept of an automatic checkout always involved RFID labels or something similar. But I never believed such a solution would happen anytime soon.
CONTENTS: Amazon Go | RFID based checkout | Cheap printed electronics | Disruptive technology got disrupted | Why is this important? | Updates
So I checked the information provided by Amazon, which isn’t that much, but it does give a clue: computer vision, sensor fusion, and deep learning. Via an article in Forbes I ended up on the Recode website, which provides a link to a patent application by Amazon. The title is ‘detecting item interaction and movement’, or translated in shopping terms: how to detect whether somebody took an item from a shelf and placed it in his shopping basket.
The system is image based. It will ‘see’ whether an item is moved from the shelf, with images taken at multiple moments. E.g. images which include the hand of the shopper, taking an item from the shelf. Next to that image analyses, the system will check whether an item has been removed from the shelf, and how many, via other ‘input devices’: a pressure sensor, a camera, an infrared sensor, a scale, a volume displacement sensor, a light curtain, a RFID reader, a barcode reader, a QR code reader.
So, in short: Amazon Go ‘sees’ if you have taken something from a shelf and confirms the removal of items from the shelf with another technology. A two step approach.
When entering the shop, you must identify yourself and your movements around the shop will be checked continuously. The moment you leave the shop, all the items in your shopping cart will be paid automatically.
RFID based checkout
The first time I heard about the idea of an automated checkout, around 2000, it relied on the use of RFID labels or a similar technology (it must have been in an article from MIT Auto ID Labs). The RFID labels from all items in your shopping cart would be read when crossing a checkout gate.
I never believed this RFID based checkout system would come true in groceries. This would imply that every single item (or package of items) would have a RFID label, even the cheapest item, which might only costs ten or twenty cents…
And another issue: what about fruit and vegetables, where you should put the amount you want on a scale and get a ‘personalized’ label for that amount? Since that is not a standard amount, the RFID label should be a writable version, not a read-only… Which comes at a cost.
Cheap printed electronics!
In those days, and even now, many people argument that RFID labels will become very cheap, thanks to printed electronics. Yes, they might become cheaper and cheaper, but they will never be free. Barcodes and QR codes are free and can be applied together with the rest of the design, during the printing process.
“But we will be able to print the RFID labels inline with the package, creating smart packages!”, I’ve heard this so many times over the past 15 years… Even if we could, would this make sense? You would need bigger presses for that, with more printing stations. And the ink for printed electronics is rather expensive, the losses in a conventional inking systems would become an issue.
And what about the yield of printed RFID labels? Even if 99,999% of the tags work, you are still wasting a lot of packages when printed inline. If we are ever going to print RFID labels, it will be on a dedicated device and they will only be applied to the package afterwards, after testing and approval.
Disruptive technology got disrupted…
Over the years, I’ve attended multiple conferences on printed electronics, the first about 15 years ago. And in many presentations, by many research groups, printing complete RFID labels was mentioned as the holy grail. With the automated checkout as the main use. “Just imagine what this would mean for the world of printed electronics! Imagine how many packages are bought every single day, worldwide!” Yes, indeed. That amount is huge. Even more than huge: gigantic! But now, with Amazon Go, it seems that we can create an automated checkout without the need for RFID labels on every item… The potentially disruptive technology got disrupted.
For the record: eventually there will be other uses for printed electronics, uses that might be disruptive. Eventually. But those research groups and companies that have bet solely on RFID labels need to rethink their business plan.
And ‘smart packaging’ is of course more than just a package that identifies itself via a RFID label or something similar. It could have sensors on it, e.g. to test the freshness. But in all cases of smart packaging there could be more disruptive technologies. Or there could be other reasons why the technology isn’t used. I already saw time/temperature indicators for food nearly two decades ago, but still we rarely find them in stores. Did you ever wonder why?
Why is this important?
Radical new technologies, like printed electronics and printed RFID labels, take time to develop and become reliable, economically feasible… It can take a long time before it reaches the market. And during that time another concept could make that ‘new’ technology obsolete, even before it was ready for the market. So you should not only closely watch your direct competitors and their research, you should also be aware of new concepts that could make your work obsolete: the huge advances in image recognition and artificial intelligence (deep learning) could make RFID labels in groceries, in warehouses completely outdated.
As a side note: do you know when the concept of RFID labels was first used in real life? Well, a long time ago: World War II. The allied airplanes had a transponder on board, which made it possible for the radar stations to distinguish friendly and enemy planes. Transponders and its successor RFID are not that new…
UPDATE 08/12/2016: There is one – important – aspect that I didn’t cover in this article: the impact on employment. Engadget put together a nice article on that: https://www.engadget.com/2016/12/07/tech-automation-retail-jobs/
UPDATE 20/01/2017: A few weeks ago, I went to the local Decathlon store (a sporting goods retailer) and they now have a mixed system. A part of the items have a RFID label, but others don’t. And in that case, the barcode is scanned. Fortunate for me: my items did have an RFID label. Below are a few pictures of them.
Today I returned to the Decathlon store and I tried to check what the treshold is between RFID labels and barcodes. And I found gloves at a price of 1,5 euro that had a RFID label. But that’s not the complete story yet: I also found (much) more expensive stuff that did not have a RFID label, that only had the original brand packaging (incl. barcode), without any label or other thing applied. Especially in the food and food supplements departement, I couldn’t find any RFID label. Some of these supplements are, however, quite expensive: over 30 euro… So I’m not sure how they decide when to use a RFID label. But still very important: they did use them on some items worth only 1,5 euro…
UPDATE 25/01/2017: Bloomberg recently published a very interesting article about the use of RFID in the retail industry. It also mentions Amazon Go and it confirms my analysis that it is based on different technologies, not on RFID. Thanks Tim for sharing the link!
UPDATE 28/03/2017: as with all new technology, not everything goes as planned. Amazon Go was supposed to open for the general public by now, but that has been delayed. One of the issues seems to be that it can only handle a limited number of customers… When more than 20 people are in the store and they are moving fast, the system seems to fail to track them.
UPDATE 20/09/2018: at this moment Amazon has opened 4 shops so far. And the ambition is to open 3.000 shops by 2021…
UPDATE 15/03/2021: Dutch retailer Jumbo is doing an interesting test, a shopping cart with built in cameras. These cameras will register what you put in your shopping cart and at the end, the bill will be made. Here is an overview, however only in Dutch. And also an update on Amazon Go: they didn’t reach the goal of 3.000 shops yet, they have 26 at this moment.
UPDATE 06/04/2024: This is an important update: Amazon stopped with Go and the ‘Just walk out’ technology! It seems that it was too expensive, it relied heavily on outsourcing to India to check video images… More on this in Gizmodo and in this follow-up article, with information from an ex-employee working on the technology. Amazon itself claims that stopping with the Just walk out technology was due to user acceptance: shoppers want to see their ticket when they pay (apparently now it is only available in the app afterwards?). The shops (or some of them) will now be equipped with self scanning.
Very interesting article but RFID tags have many advantages for retailers long before goods reach checkout. RFID allows retailers transparency over their stock, sales and shrinage. Where ever they are implemented, retailers see an immediate five percent increase in sales. They are not free, they are not cheap but they do see a return on investment within a few years.
Thanks for your comment Tim! On a case or pallet level there is certainly no discussion: there is a big benefit for RFID. But on an item level, I think it depends… And it depends on the sales price of the item. That’s why I never believed in the RFID-based automatic checkout in groceries. Some of the items are just to cheap to afford a RFID tag.
But it seems there is also a way in between: I recently visited a Decathlon store and they have a mixed system. Most items now have a RFID tag (and they indeed saw a big return due to accurate stock levels, etc.), but the really cheap items only have a barcode… I noticed it when I was waiting at the checkout: the lady in front of me had a dozen of items and most of them ‘beeped’ when the cashier put them in front of her. But some items (a few packages of candy) didn’t beep and the cashier had to scan the barcode. This is of course a valid system, but it is not a completely automatic checkout. I still have to return to the shop to check what the treshold between RFID and barcode is… When I do, I will update the article with that info.
I look forward to the update. I agree that RFID for groceries is a long way away. By the time the cost of printing complete RFID labels has become affordable enough for bananas, we will probably be able genetically modify them to grow their own barcodes.
Very nice article informative content thanks we liked it.
Thanks Herbert! 🙂