Another told him that wasn't entirely true. It was once "a vastly wealthy community," as evidenced by the great mansions on High Street.
The first man disagreed. It may have been wealthy long ago, but the recent influx of new wealth meant many of the old working-class neighborhoods were disappearing.
Yet another man chimed in, a relative newcomer to town, offended at the notion that non-natives were ruining the town.
"Many of us are not native Newburyporters who wouldn't want to live anywhere else," he said.
It's a debate that could have occurred yesterday on the central waterfront boardwalk or at the lunch counter in Angie's Restaurant.
But it didn't. It took place almost 32 years ago, in front of a committee that was charged with probing the impact of Newburyport's unique brand of urban renewal.
Newburyport's downtown revival of the 1960s, '70s and '80s left an indelible mark. It made the city a national model for preservation. It restored a dying downtown.
And it caused enormous social changes that have stretched over 30 years and counting.
"A lot of people want progress without change. There's no way you can have progress without change," then-Mayor Byron Matthews said at that meeting in front of the Growth Policy Committee in 1975.
Newburyport wasn't the first to shun the "federal bulldozer," the pejorative name for the urban renewal practice of tearing down the core of old cities and rebuilding them.
But by the mid-1970s, its story was so compelling, its turnaround so quick and its physical attractiveness so becoming, it became the poster child for success - even though it hadn't entirely succeeded.
At that time the revival of the downtown was still far from finished. Many buildings were vacant, and the waterfront was an ugly dirt lot. But the first phase - the Inn Street Mall - was alive and well, several storefronts along State Street were restored and reopening with unique artisan shops, and the streets and sidewalks had been attractively redone in a Colonial style.
Against a national backdrop in which America was struggling to revive its inner cities, Newburyport's successes were widely applauded. The New York Times, Wall Street Journal, Christian Science Monitor and many other newspapers trumpeted the spirit of renewal and restored charm. Typical of the headlines was one from the Lowell Sun: "Newburyport: A City that Came Back."