Skip to content
Naked Security Naked Security

That Black Mirror episode with the social ratings? It’s happening IRL

Not picking up after your dog will cost you 10 points, for example, in China's Black Mirror-esque plan to socially score citizens.

What do you get when you cross the worst aspects of social media, people’s actual lives and giant, centralized databases?
The outcomes are already playing out. Certain cities in China have been piloting the country’s social credit score system – a system that’s due to be fully up and running by 2020, according to a plan posted on the Beijing municipal government’s website on Monday (the plan is dated 18 July).
One of the many repercussions of such a system is that people get blacklisted for not paying off their debts when a court thinks they’re capable of doing so, regardless of what the debtor says.
The ID photos, names and numbers of blacklisted people are displayed on billboards throughout the city, and they’re then barred from booking flights or high-speed trains (considered “luxury” travel) and blocked from staying in hotels. By the end of May, people with bad credit in China had been blocked from booking more than 11 million flights and 4 million high-speed train trips, according to the National Development and Reform Commission.

A permanent stigma?

The Indicator podcast on National Public Radio’s Planet Money has been covering China’s unfolding social credit system for the past few months. Last month, the podcast covered the story of Lao Duan, a 42-year-old coal broker, who told NPR that people’s attitude toward the blacklisted is one of suspicion. He can’t relax when socializing with anybody who isn’t also on the blacklist – which is the case with many of his former coal-dealing colleagues:

(Through a translator) The widespread attitude towards us is very resistant. People were saying, why are you cured being happy? Why do you still have time to be happy? Why do you not go out and make money to pay back your loan?

Unfortunately, as NPR reports, the stigma appears to be permanent. While in countries like the US, we can declare bankruptcy and then hopefully rebuild our credit, in China, the label of socially/financially unacceptable appears to be irreversible. Though it’s technically possible to get off the blacklist, it seems that nobody ever does.
A financial scarlet letter is only one aspect of China’s increasingly pervasive social credit system, which, according to Bloomberg, is designed to make it impossible for the “untrustworthy” to “move even a single step.”

Rewards and penalties

The pilot city of Rongcheng foreshadows how comprehensively China plans to judge each of its 1.3 billion people with its lifelong points program, which assigns each resident their own, personalized rating. The system is similar to the credit score system of the US, but it will also include ratings for behavior, including whether you squabble with your neighbors, clean up after your dog, or obey the traffic laws.
Xu Ranjan, a 32-year-old IT engineer, told NPR that it’s gotten a lot more peaceful in Rongcheng since it implemented the multifaceted penalties. It used to be that pedestrians risked their lives crossing the street. Now? The cars wait for you. If they don’t, their drivers will lose points. And that can be bad.
There’s a letter/grade system behind the points. Everybody starts at 1,000, though you can earn more by, say, giving to charity or donating blood. If you score between 960 to 1,000 or more points, you’re an “A.” Between 850 to 959 points, you’re a “B.” You’re a “C” if you’re between 849 and 600. That’s considered a “warning” level. Below that, you’re a “D,” and you’re labelled untrustworthy.
Driving while intoxicated? Automatic downgrade to a B. You’re also docked 50 points for spreading rumors online.
A high score gets you discounts at local businesses, a lower heating bill, free cable channels (such as a kung fu channel and a soap opera channel), and special invitations to community events.
A low score not only costs you those goodies; it also means you might not be able to get a promotion at work, regardless of your great performance and productivity.
NPR reports that the government is keeping track of all of this with the help of designated “watchers.”

These are people who keep track of goings-on in their neighborhoods. And they keep track of neighbors’ behavior and update people’s scores.

‘Nosedive’

If you haven’t already watched the “Nosedive” episode of the TV show Black Mirror,” perhaps you should, because it’s no longer fiction, at least in China. Somebody in the nation’s government must have interpreted it as a blueprint.
The system has some benefits. Xu Ranjan, who has a score of 1,000 and is living in Rongcheng’s pilot system now, says that it doesn’t feel creepy. He likes it. People are behaving, and they’re putting the needs of the community above their own selfishness, Ranjan said: a positive step in the evolution of society and people living together:

I feel people have become more friendly. I think in the history of human being developing towards, like, communities, if everybody can follow the rule, I’m sure it’s very good for, you know, every individual living in the city.

According to Beijing’s published plan, the capital city is on track to pool data from multiple databases and, by the end of 2020, will be rating some 22 million citizens based on their actions and reputations. To do so, they’ll rely on a range of services, with data flowing from agencies that deal with tourism, business regulators and transit authorities.
They’ve already got much of this already in place: just a few examples include an enthusiastic embrace of facial recognition, such as when Chinese police got facial recognition glasses to find suspects in the massive human migration that is China’s annual Lunar New Year. Chinese authorities love facial recognition technology so much that they use it in at least one public park to thwart toilet paper thieves.
As Bloomberg points out, the move to online banking has also made tracking individuals via financial data easier:

Apps such as Tencent’s WeChat and Ant Financial’s Alipay [have become] a central node for making payments, getting loans and organizing transport. Accounts are generally linked to mobile phone numbers, which in turn require government IDs.

With pieces of such a vast system of surveillance already in place, it’s hard to imagine anybody can escape, and that means that the state can punish anybody it wants, with ease.
Political dissidents? Ethnic minorities? Jaywalkers? Toilet paper hoarders?
Check, check and check: vast databases, linked together, enable states to find them, name them, shame them, fine them, or to persecute them. That’s why it’s called Big Data: the oceans of data being linked together are wide and all-inclusive. The repercussions can be deep.

17 Comments

Next up for China, High score people and companies “managing” (owning) low score citizens. I’ll bet 500 zeni that is normal by 2024 (5 years)

This is terrifying to me. So glad I don’t live in China.

Given the growing use of surveillance technologies by law enforcement on a global scale—and that certainly includes the US and the UK—I’m not relieved that I don’t live in China. China is further along than we are in the West, but we’re on the same path, and we’ve got the same infrastructure being put into place to enable the same overarching monitoring.

So you would rather be further down the path towards complete government control of your entire life than back where the UK/US are where we actually have a slight chance to stop it? Please explain that logic.

Pretty sure what Lisa meant is that relief for not being in China (at least in this context) would be a false sense of security, as the same steamroller is headed down our street as well.
(sorry late reply)

And then, when you fall below a given number, you are eliminated. This reduces the population and increases the quality of life for those that remain ….

Designated watchers with personal axes to grind against other citizens sounds like a scary scenario.
“NPR reports that the government is keeping track of all of this with the help of designated “watchers.” These are people who keep track of goings-on in their neighborhoods. And they keep track of neighbors’ behavior and update people’s scores.”

The author completely misses the point. This system is not about one’s creditworthiness as it is traditionally understood, but one’s standing in the eyes of the thought police. Obey or else. Dissidents are simply banned from participation in public services typically available to all citizens.

I’ve seen that Black Mirror episode three times & this technology & social judgement with labeling truly frightens me. It is creating a racism & segregation of a new type, which definitely includes the worst type of conformity. I agree with the above remarks that it’s a serious form of implementing “thought police” and also that the implementation of the ‘elimination’ of the undesirables is likely not far behind this bold move to control the behavior of the population.

I see a few issue here right off the bat.
What if someone just happened to be in the wrong place at the wrong time, or was just dealt a bad hand in life. This happens to a lot of people, and usually can’t be prevented. A bad hand could be as simple as someone robbing you, resulting in financial hardship, or even just having one of the dedicated “watchers” see something out of context and rate you inaccurately. By the sounds of this, that would result in undeserved demotion of your score. Add to that the fact that many people need to travel for work these days. If someone who does travel regularly for work suddenly just ends up in a situation where they have no control and get blacklisted, they can’t fly or stay at hotels, resulting in inability to perform at work, and ultimately, get fired. Now they are in a downward spiral that they most likely will not be able to escape.
I’m sure I could find more unmentioned issues with this system. Sure, automation and universal connectivity can be helpful, but this is a case where I can only foresee turmoil for innocent, but unlucky members of society. And there are a good amount of those people out there.

I’m afraid of this type of computer watchmanship. It puts me in mind of the novel 1984 by George Orwell. I found it frightening then and always will. This smacks heavily of Big Brother; nothing will be secret in our lives.

I might guess that Chinese with religious beliefs that conflict with the government’s dicta will not do well in the future.

Already happening. Do a search for – “Chinese religions re-education camps” in the last month.

Comments are closed.

Subscribe to get the latest updates in your inbox.
Which categories are you interested in?
You’re now subscribed!