Wealthy investors find opportunities in stock market sell-offs

Wealthy investors and family offices shied away from stocks leading up to market swings this week, but many saw the drop in prices as an opportunity for tax savings and estate planning, according to wealth advisors.

Private banks and wealth managers say their clients have been reducing their stock holdings for over a year as part of a broader shift from public to private markets in light of recent concerns about an overheated tech sector.

According to a UBS family office survey, family offices have 35% of their portfolios in private equity — the largest of any asset class — compared with just 28% in equities. A Deloitte survey found that family office holdings of equities fell from 34% to 25% from 2021 to 2023, while their private equity jumped from 22% in 2021 to 30% in 2023.

When stocks tumbled Monday, with the S&P 500 and Nasdaq down 3%, wealthy investors neither panicked nor jumped in to buy, according to several advisors. They did have a lot of questions.

“The common question from clients was ‘What’s going on?’” said Sean Apgar, partner and co-head of portfolio and wealth advisory at BBR Partners, which advises ultra-wealthy clients. “It was more out of curiosity; there was no real motive for action.”

Apgar said the clients BBR advises — most worth hundreds of millions or billions — don’t react to short-term market events given their long investing horizons. Yet they did want to be educated about the market moves, the Japanese carry trade, the growing recession fears and rate cut odds. For his clients, their investment plan is still their investment plan. 

“The best thing clients can do right now is sit back and feel good about the investment plan we put in place with them long ago, with expected volatility and corrections along the way,” Apgar said.

The drop in prices last Friday and Monday also offered a chance for wealthy investors to take advantage of tax benefits and gift strategies.

William Sinclair, head of the financial institutions group and the U.S. family office practice at J.P. Morgan Private Bank, said a growing number of clients have so-called “separately managed accounts,” discreet accounts designed to hold a specific group of assets or stocks. With separate accounts, clients can more easily sell stocks that have declined in value and realize losses they can use to offset capital gains from their winning stocks, known as “tax-loss harvesting.”

With some Big Tech stocks down 15% or more over the past month, wealthy investors are selling at a loss, reaping the tax benefits and buying the stock back at a later date to retain their position.

“For taxable clients, the biggest inflows have been in tax loss harvesting strategies,” Sinclair said.

Others are using the price swings for estate planning. Under the current rules of the estate and gift tax, married couples can transfer up to $27.22 million to heirs and family members, while individuals can transfer up to $13.61 million. With the gift and estate exemption amount scheduled to expire at the end of next year, many wealthy investors are working to give away the maximum before the expiration.

Gifting stocks that have declined in value carries more benefits, since it allows investors to gift more stock under the exemption amount.

“Say you have a stock that was worth $100 and now it’s worth $80, you can transfer that lower value to the next generation, assuming the assets will eventually appreciate again,” Apgar said. “So you’re taking advantage of the depressed values. Tax advisors get generally excited about these environments because it opens up new opportunities.”

One group of clients that’s more sensitive to the recent bouts of volatility is made up of corporate founders and top executives. Since they often have a large portion of their wealth tied up in one company stock, advisors can help them structure complex hedges — such as variable prepaid forwards and exchange funds — to help dampen the blow of big stock declines. The stock decline of the past week highlighted the benefits of so-called “collaring” structures to many founders and CEOs. 

“People in these roles, in the C-suite, know that their job, as well as career, is going to center on the stock,” said Jennifer Povlitz, division director at UBS Wealth Management U.S., which advises many clients with concentrated stock positions. “So the financial planning part has to be a consideration.”

While the S&P 500 is still up roughly 10% this year, after gaining 24% in 2023, ultra-wealthy investors and family offices are continuing to shift more of their money into alternatives, especially private equity. Many see private companies as more stable and profitable over the long term compared to equities — especially after days like Monday. And they can have more impact on management with direct stakes in private companies. 

“Most family offices are so invested in alternatives, hedge funds, PE and real estate, that they aren’t moving their investments around anyway,” said Geoffrey von Kuhn, an advisor to several of the nation’s largest family offices.

Richard Weintraub, family office group head of the Americas at Citi Private Bank, said family offices have been moving their money to longer-term investments — which can grow over decades or generations — with less volatility. Along with private equity and venture, the big trend among family offices is direct deals to buy stakes or control of private companies.

“The larger family offices, so $10 billion plus, are deploying capital into operating companies they can hold in perpetuity and pass down generation to generation,” Weintraub said. “Like building the Buffett model.”

He added that the stock swoons of the past week “reinforced the idea of making that shift toward private investments.”

Michael Pelzar, head of investments at Bank of America Private Bank, said high-net-worth investors are still catching up to family offices when it comes to private markets and alternatives.

“In general, I think high-net-worth investors are under-allocated to alternatives,” Pelzar said. “We see this [volatility] as a catalyst to enable high-net-worth investors to continue to broaden their portfolio. I think that after this week there will be more open-mindedness when it comes to alternatives, whether in PE or real estate.”

Advisors say that when it comes to the overall investing environment, the biggest worries of high-net-worth investors are about geopolitical risks and fiscal spending. Jimmy Chang, CIO for Rockefeller Global Family Office, said the most common question clients are asking is not about stock market volatility but about the impact of government debt and deficits.

“They want to know the implications for tax planning and also for the economy and the market,” he said.