Their League One opponents, who had staged a spirited revival, won the shootout by a 4-1 scoreline.
Dougie Freedman, who had put the Championship side ahead after half an hour, saw his kick brilliantly saved, while striker Shefki Kuqi hit the crossbar with his effort.
Freedman's superb free-kick after Tony Craig had been brought down had looked good enough to put Palace through.
They ran the game in the first half without taking their chances, but then Rovers came back at them strongly after Craig Disley equalised to take the tie into extra time.
Rovers found a fresh urgency with the arrival of two substitutes early in the second half.
One of those taken off, Andy Williams, was none too pleased. It was his home debut after a transfer from Hereford and his 21st birthday.
But the changes worked like a charm for the League One side who matched everything Palace could offer.
They were never short on confidence and, after Freedman had put Palace in front, had a similar chance of their own when frontrunner Richard Walker went down before a tackle which brought Jose Fonte a rather harsh yellow card.
For the second time in the half, Steve Elliott hammered a kick into the wall.
Palace had rested six of the side which stunned Southampton with a 4-1 win on Saturday and hat-trick man James Scowcroft was not even on the bench.
But they still made enough chances to have taken a comfortable lead only to waste all of them.