By John Engel
Times of crisis, of disruption or constructive change, are not only predictable, but desirable. They mean growth. Taking a new step, uttering a new word, is what people fear most.
— Fyodor Dostoevsky
I’ve been thinking about datamining a lot these days, what it means and how it is changing in the context of elections, business, and real estate. But first, some context.
In the 1990s, I worked for a New Canaan firm that studied consumer behavior in the retail environment. What we learned was to divide the world not just into men, women, and children, but to divide consumers into cohorts and group them by how they behave, by their shopping habits.
Our client Rollerblade organized their customers into four groups: “Fad Followers,” “First Class,” “Specialty,” and “Value shoppers,” each motivated differently and with different price sensitivities. Some were motivated by a sale, others bought the latest new trend or color, still others preferred specific brands or the best of a category, and some looked for specific specialty attributes such as sport-specific gear.
In the 1990s, we followed consumers around the store, taking notes with a clipboard. I often wondered what category I fit in. Could I be a value shopper one minute and first class the next?
In the 2000s, this science of behavioral analysis made major leaps forward as we began studying consumer shopping behavior and decision-making on the internet. I stopped following customers around the Walmart and started following them online. We wrote algorithms. If you’re in zip code 06840 and a fan of Martha Stewart, you might be a candidate for a Suburban at Karl Chevrolet.
Now, add AI to the equation. Demographic, geographic, and psychographic analysis leads to dynamic pricing of everything from airline seats to movie tickets. Why not houses? Amazon would not exist without advanced and immediate behavioral cohort analysis. And now, artificial intelligence is accelerating our understanding of human decision-making.
What about real estate, particularly luxury real estate? Our industry has been slow to adopt technology, almost resistant to the idea, saying that the role of a great agent cannot be replaced with Zillow and the best algorithms. We’ll remind you that real estate is not a commodity like buying a car, and that intimate local knowledge sets great realtors apart. And we’ll point to current NAR statistics that show 97% of all houses sell through a realtor.
These were the same arguments made by Major League Baseball scouts in the days before Moneyball and statistical analysis took over professional sports. For those who did not read Moneyball, the point is simple: the intuition of the Realtor is being replaced by a better and mathematical understanding of consumer behavior. The tools are getting better every day, as are the means of data collection.
Showingtime is the appointment-setting software realtors use, and it was purchased by Zillow in 2021 for $500 million. Why? Datamining. Big Tech knows what homes you looked at, the amount and rate of your current mortgage, and how much equity is in your current home, and it knows where you are on the journey. Soon, it’ll replace my gut feeling about what you should offer on that house with a defendable and mathematically more-perfect recommendation.
Datamining will focus on two areas, maybe three: the inventory, the buyers, and the agents. Let’s break it down in our market.
First, the supply of homes is finite. There are only 7,000 homes in New Canaan, all in the Town Clerk’s database including when sold, the price, and building permits. AI is getting better at analyzing the publicly available photos of the interiors of our houses, enough to make a more precise evaluation of value. If this were easy or precise, Zillow would still be in the home-buying business.
Second, we study the buyers and their behavior — “Where are they coming from? Why are they moving? How long have they been looking?” — and we react, perhaps adjusting how we talk about the house and the market. It doesn’t take a great leap to imagine the photos changing and the listing description shifting depending upon who is looking, what they want, and where they are on the journey. The systems that currently show me “heat maps” of changing demand geographically are getting more sophisticated. Armed with a better understanding of buyers, the heat maps will model Fairfield County behaviorally.
Owners of the most expensive homes tend to think that their buyers are coming from far away. It’s statistically not true. Most come from not-so-far-away. Sellers ask if their house will be advertised in our New York, London, or Dubai offices. The truth is, we know which zip codes move to New Canaan, why, and what life events cause them to consider moving here.
Tracking the movements of ultra-high-net-worth individuals (UHNWI) and high-net-worth individuals (HNWI) used to be for private bankers and the major auction houses tracking art collectors. It is now incumbent on the major real estate firms (opening offices in every luxury market) to begin studying customer movements so they can get ahead with a recommendation or a referral at the most opportune time.
Third, there are the triggers. It’s not enough to know a buyer is wealthy with a Brooklyn zip code. When your child turns 10, you think about upgrading schools. We buy and sell real estate when certain major life events occur. Proctor & Gamble is so successful because they knew you were having a baby before you did, and they dripped offers on you to make you a customer for Pampers. So, too, we know that young families with Brooklyn zip codes are looking to move to New Canaan for the schools. Which of the major real estate firms in Brooklyn know their names and are making that transition easier for them? While the business still relies on the agent today, and many of those Brooklyn families are relying on the advice of a friend or family member to choose a realtor. With AI, it won’t be long before the agents in Brooklyn and New Canaan are communicating earlier and more efficiently about their shared prospective customer, and their agency (or Zillow) will be facilitating that.
Why is the real estate industry not further along in tracking consumer behavior? Agents are independent contractors. They move from agency to agency, and they take their Rolodex and relationships with them. The better agencies are providing increasingly sophisticated toolsets and technologies to get stickier with the most capable agents and their clients. Just as we rely on Showingtime to set the appointment, we’re encouraged to turn over our Rolodex to the agency to help manage the relationships: “We will help you stay in touch, at the right time, in the right way.” Google stories of people falling in love with AI chatbots, and you know this is coming. Will the best agencies be the ones who leverage datamining tools to understand the houses and buyers in these important luxury markets? What will be the role of the agent?
As the tools improve, agents in Brooklyn and Vero Beach connect with agents here, making it easier for me to sell to a Brooklyn family and help my seller buy in Florida. Pattern recognition improves: birds of a feather flock together. Be it New Yorkers, Canadians, or Fairfield County buyers, we tend to move to a predictable set of southern zip codes. AI of the future, knowing my kids, their schools and interests, the status of my mortgage, and how much home equity I have in my house, will know the towns I’m considering and have a ready-made solution for financing, staging, estate sales, moving companies, and insurance.
My neighbor recently moved to Florida to be in a very specific neighborhood, near a specific school, close to the squash facility and coaches, commutable to the office. If the business of real estate was less fractured, the relationships better established through better understanding, I might have made a referral.
What is the takeaway? In an election season where datamining took center stage, where we saw huge sums spent in understanding the behaviors of important zip codes, sliced down to the city block, why not real estate sales? If 2024 was the year lawsuits shook up the real estate industry, forcing agents and agencies to re-think where we add value in the process, 2025 is the year where agencies start getting serious about more than “marketing support.” Besides reach, the critical way agencies and agents add value is in providing a deeper understanding of each local market and customer behaviors on a granular level within that market. 2025 is the year where Moneyball meets Real Estate.
John Engel is a former Military Intelligence officer who shifted his focus from studying military behavior to consumer behavior in the early 90’s. He started two Internet companies, e-Media and Paper.com. Now a Realtor with Douglas Elliman, he is both afraid of being replaced by the machine and embracing the machine. John used spell-check on this article, but did not use ChatGPT.
