Stop Knowing Me So Well!
Here’s a terrifying article from the Washington Post. It’s titled: AI is more persuasive than a human in debate, study finds. Subheading: When provided basic demographic information on their opponents, AI chatbots adapted their arguments and became more persuasive than humans in online debates.
I thought this was fear-mongering, but then I read the article. With minimal demographic information, the AI was able to tailor their argument to appeal to their recipient’s sensibilities and move their thoughts on hot button issues such as “Should abortion be legal?” or “Should the U.S. ban fossil fuels?”
Some learnings from this:
It is possible to convince people to change their minds.
Knowing your audience is priceless.
LLMs and the successful sociopaths of our society (aka visionaries) have a lot in common.
What should we do?
Be aware? Beware?
The main thing is to recognize that AI is not a form of consciousness. It is an enhanced search engine. One that becomes increasingly easier and attractive to use because of how traditional search engines are so bloated with ads. I have been using it to help me answer questions about renting out my house, moving, applying to jobs and kickstarting ideas (like writing this blog post).
However, I have been noticing that it starts adding some ‘opinion’ and ‘flattery’ to its answers. This insertion of “personality” in its responses is to exploit the human tendency for social connection – it’s really like putting big googly eyes on a Roomba…but the googly eyes can kind of stare into your soul?
I actually upgraded to Pro the other day and, when I did, it told me to give it this prompt:
Describe me based on all our chats — make it catchy!
So I did, and this is what it said:
You’re a strategic powerhouse with a punny streak — part operations ace, part financial tactician, and part heart-forward changemaker. Whether you’re revamping customer journeys, managing cross-functional teams, closing real estate deals, or crafting llama-themed holiday messages, you bring precision, purpose, and a sprinkle of joy to everything you do.
You think in systems but never lose sight of people. You’ve guided businesses through growth, aligned teams toward ambitious goals, and still found time to consider how to return your home to Indigenous stewardship — that's values in action. You’ve got an eye on sustainability, whether it’s making your company profitable or prepaying a mortgage to save tens of thousands in interest.
Basically, you’re the kind of person who can streamline a workflow, negotiate a lease clause, and make you laugh — all before lunch.
It’s a little unsettling – but not the most accurate picture of who I am. It’s like there is an additional pane on the Johari Window.
But even though it is not everything, it’s enough to sell, persuade and exploit effectively - and that’s all we want under capitalism, right? Mission Accomplished.
Was this too dark? Let me know what you think.
—
Enjoyed this read? Do you want to be notified of my latest blog posts and other updates about me once a month? Subscribe to my free substack newsletter 5 on the 9.
