Dean Windass, on loan from Middlesbrough, had given United the lead in first-half injury time, his second goal in successive games, before Finnish international Aki Riihilahti hit back for Palace in the 69th minute.
Allison struck from close range after fellow sub Carl Asaba slipped in a low cross from the right with seconds to go.
Palace almost snatched the lead when former United midfielder Shaun Derry prized open the Sheffield defence with just two minutes gone, his precise cross finding Andy Johnson. The Palace striker beat his marker to power in a fierce header, Paddy Kenny diving to his left to push the ball round the post.
Johnson was on target again minutes later squeezing in an effort from 16 yards that was cut out by the diving Kenny.
Co-striker Dele Adebola followed up with a darting run which took him past Jon Harley, but centre-half Robert Page nipped in to steer the ball away for a corner with Adebola about to shoot.
United, struggling to find a way past Palace's well organised defence, tested Alex Kolinko, Michael Brown's long-range strike smothered on the line by the keeper before they snatched the lead in injury time.
Phil Jagielka threaded the ball to Steve Kabba on the right flank and as Kit Symons slipped, the former Palace striker unselfishly pulled it back to Windass who powered the ball into the roof of the net from 12 yards.
Kabba almost grabbed a second on 51 minutes starting with a strong run down the right wing and finishing with a stinging right-foot shot which Kolinko could only block, the Palace keeper reacting quickly to charge down another effort from Peter Ndlovu as the United midfielder raced on to the loose ball.
Kenny pulled off a stunning stop, springing to block a piledriver from Adebola, the Palace frontman's 20-yard strike crashing against the United keeper's right arm.
Trevor Francis's side we back on level terms in the 69th minute. Johnson carved out the goal crossing from the left as substitute Riihilahti steamed into to beat Kenny with a right foot shot 16 yards.
Adebola squandered a chance to score 15 minutes from time as he connected with a Danny Granville cross, but cracked the ball against the bar and Johnson shot high and wide minutes later, before Allison struck the winner.