Facial detection used by Westfield malls for targeted advertising

Biometric billboards

The billboards, named SmartScreens, were designed by Scentre Group, the parent company of Westfield, and are equipped with cameras that can determine your age, gender and mood while you shop.

The billboards are supported by software provided by Quividi, a French company that measures over 1.5 billion people per month.

Quividi says its technology can accurately estimate individuals’ gender and age using their facial features. The technology can even predict how you’re feeling, such as whether you’re very happy, very unhappy, or somewhere in between.

Using this technology, advertisers can better demonstrate to clients how many people see an ad, how engaged those people are, and in some instances, whether the ad contributes to higher sales.

Targeting consumers

The technology also gives advertisers the ability to change an advertisement based on who is looking at it.

You can see how this can be applied in the image below, which shows how Westfield’s Quividi-powered SmartScreens tailored the advertisements of Samsung Galaxy S10 smartphones to customers in Australian Westfield stores in 2019.

If you shopped at Westfield, SmartScreens would show Samsung ads based on age and gender. SmartScreens showed:

• women 44 and under the slogan “With enough data for your insta-habit”
• women 45 and older the message that they’d have enough data to “shop till you drop”
• men of all ages the message “stream every 2018/19 Premier League match live”
• men 44 and under a young man with a phone
• men 45 and older an older man with a phone.

The results were great for the businesses involved – 73% of the audience saw a targeted advertisement, with targeted ads viewed for 29% longer than non-targeted ads. Older audiences were more receptive, with targeted advertisements leading to a 56% increase in engagement among older women and an 82% increase among older men.

A further study by Quividi has shown that the use of their screens, and the targeted advertising they provide, can lead to a 24% increase in sales.

Detection, not recognition

Quividi says its technology is “facial detection” as opposed to facial recognition technology (FRT). Unlike FRT, which can be used to recognise an individual, Quividi say its software can only determine if an anonymous individual is looking at a given point of interest, the time they stayed engaged and an estimation of their basic demographic characteristics.

In November 2022, we reported that Foodstuffs North Island (FSNI) had implemented FRT as a tool for crime prevention in 29 stores, using the technology to identify individuals that may represent a theft or security risk. This required holding facial recognition data of customers for up to five days and hosting a database of individuals that had committed criminal acts in store to spot them and prevent entry. FSNI announced they would disable the technology in December 2022.

The use of SmartScreens by malls is very different, allowing businesses to estimate which broad category a consumer fits into rather than identifying an individual. However, while FSNI have argued that FRT helped to make consumers safer in their stores, SmartScreens are used purely for advertising.

A slippery slope?

Gehan Gunasekara is an associate professor of commercial law at the University of Auckland, as well as a co-founder of the Privacy Foundation New Zealand and the convenor of its surveillance working group.

Gunasekara says, “In terms of potential harm, this is at the lower end, but as a privacy advocate, this is the thin edge of the wedge. People might start to accept this kind of thing, but then the next thing is that you might be offered health products on the basis of the way you walk, or physical characteristics that suggest you have some kind of disease.”

Do you know you’re being watched?

Consumer awareness is another issue, according to Gunasekara. “Another problem is transparency, where people are not aware that they are being secretly filmed, with advertising targeted on that basis.”

Westfield describe the use of SmartScreens in their privacy policy, but the explanation is full of jargon that consumers cannot be expected to interpret alone.

When did you last check the mall’s online privacy policy before shopping? Should you have to? And if you’ve been to one of these malls, did you know they are analysing your biometric data to target you with advertising?

We sent a member of staff to Westfield Riccarton to see whether there was any information about the use of SmartScreens at the centre. Upon entry, they saw a “Welcome to Westfield” sign at the store entrance which said, “For your safety: areas of this shopping centre may be under video surveillance”.

Yet safety isn't the reason the billboards are filming you. To find out what information is being gathered, you're referred either to the online privacy policy or a customer service desk inside the mall.

If you haven't read and understood the mall’s online privacy policy, and the mall is using SmartScreens, it’s likely that biometric data is being gathered without your knowledge.

Consumer CEO Jon Duffy says, “the presumption that a shopper will have gone online, read and understood a mall’s privacy policy and consented to it before entering a public space like a mall is ridiculous. It’s not okay to bury what you’re doing in some far-flung corner of the internet and call that disclosure. These companies need to do a better job of informing the public about their data collection and use.”

What does the Privacy Act say?

The Privacy Act 2020 sets the rules for our rights to privacy. Under the act, a business should take reasonable steps to make sure the customer is aware of what data is being collected. There are exceptions, notably, if an individual will not be identified from the data collected, or if the company has grounds to believe that not informing consumers about collection would not negatively impact them.

We asked Scentre Group, Westfield’s parent company, whether they believed they had an exception. They acknowledged our e-mail but did not respond to our questions.

A spokesperson for the Office of the Privacy Commissioner (OPC) said, “even if an exception does apply, OPC would recommend that simply on ethical, trust and confidence grounds it would be advisable for the agency to inform customers that they are undertaking targeted advertising based on real-time facial detection analysis of mall customers. This would allow customers the opportunity to make an informed choice about whether they wanted to be targeted in this way”.

An investigation into biometric billboards in Canada

In 2018, an investigation revealed that 12 malls in Canada were using “anonymous video analytics”.

The investigation does not name Quividi but describes a similar practice to that conducted by Scentre Group’s SmartScreens.

The investigation concluded that the malls, operated by Cadillac Fairview, collected and used consumers’ personal information through this technology, without ensuring knowledge, consent or notice.

Although Cadillac Fairview insisted they didn’t keep people’s biometric data, the investigation found their service provider had collected and stored the facial data of 5 million customers.

The preliminary report recommended that Cadillac Fairview could either stop using the technology or that customers should be informed about the privacy implications associated with the technology, with users given the opportunity to opt in or refuse to provide consent. Cadillac Fairview stopped using the technology.

International efforts to ban these technologies

Against a backdrop of global concern about the rapid growth of AI, there have been steps toward regulating the application of both FRT and the type of analysis conducted by Scentre Group’s SmartScreens.

The OPC is exploring developing a Code of Practice to regulate biometrics in New Zealand.

The EU has made the most significant strides in regulating AI. The European Union’s AI Act is a framework that aims to significantly bolster regulations on the development and use of AI. Due to its importance for global trade, where the EU goes, the rest of the world tends to follow, and there is potential for this legislation to inform New Zealand.

The proposed legislation would classify AI systems by risk, with several practices described as intrusive and discriminatory set to be banned.

Under this legislation, several practices adopted by New Zealand retailers would be banned.

This ban would include:

• real-time biometric identification in publicly accessible spaces (including supermarkets)
• biometric categorisation systems using sensitive characteristics including gender and race
• emotion recognition systems in law enforcement, border management, workplace and educational institutions.

This legislation is expected to be finalised and passed into law in the EU by the end of 2023, with a two-year implementation period, although provisions related to high-risk AI may take effect sooner.