The Eagles led through Connor Wickham's first-half penalty, awarded after Glenn Whelan fouled Wilfried Zaha, but they looked to be on course for only a point after Stoke's Bojan Krkic converted from 12 yards himself after Damien Delaney was adjudged to have handled with 14 minutes to go.
However, when Glen Johnson blocked a shot into Lee's path 25 yards out, the substitute shaped a brilliant attempt through a forest of bodies and into Jack Butland's far corner to make it 2-1.
The win kept Palace in sixth, level on points with Tottenham and Manchester United at Christmas, and Pardew admitted he could not see the winner coming when South Korean Lee stepped forward late on.
"Amazing, wasn't it?" said Pardew.
"I was right behind it
Jordon (Mutch) has a shot that he completely miscues, Damo has a shot and it gets blocked and comes out and the best you can hope for really is Chungy to put it back into the mixer
He had other ideas.
"It was a really great technical goal because his control to hit that side-footed, with that power, against the wind, was amazing
It obviously brought the house down on our bench."
Pardew brought Lee to Selhurst Park from Bolton in the last window transfer window, but an injury and the form of Yannick Bolasie and Zaha has left the 27-year-old with few opportunities to impress
"He's a great player, he's unfortunate in this team," Pardew added
"I brought him to this football club, he would have wanted more opportunities
But, with Bolasie and Zaha filling those wide areas, he doesn't get a lot of opportunities
He certainly woke us all up today
"He can score goals - usually not from as far back as that, more of a taking people on and weaving his way through type of goals
I'm delighted for him
He's one of the most popular players in our dressing room
The lads are absolutely delighted for him."
Mark Hughes' Potters, who again deployed attacking trio Bojan, Xherdan Shaqiri and Marko Arnautovic together in the hope of recreating their showing against Manchester City at the Britannia Stadium, were frustrated by Palace keeper Wayne Hennessey on three occasions.
When he was eventually beaten by Bojan from 12 yards, Hughes did not expect a side that had substituted Wickham, Jason Puncheon and Zaha to find a winner.
"Towards the end of the game you're thinking Palace looked reasonably comfortable or happy with the point because they'd taken most of their more creative players out of the game," Hughes noted.
"But, lo and behold, they get one last corner and it drops to the lad on the edge of the box who will never hit a ball like that ever again, probably hasn't before either.
"That's what happens - it's sport, you have to dust yourself down and keep going
Confidence wise we're not affected because I thought we did okay today."
Source : PA
Source: PA