Palace made it five wins from six as the Eagles kept in touch with the play-offs. Danny Butterfield scored the all-important goal after they were reduced down to ten men.
On-loan Mark Hudson was shown red five minutes earlier after the referee decided he had comitted a professional foul on Mamady Sidibe.
Nicky Southall's spot-kick was brilliantly saved by Cedric Berthelin but the referee ordered the kick to be retaken and this time Southall proceeded to hit his penalty against the crossbar.
This inspired Palace and Butterfield brought the house down with a magnificent 15-yard volley after the Gills failed to clear Julian Gray's cross from the left.
Manager Andy Hessenthaler managed to sign Lars Hirschfeld on loan from Spurs to solve his goalkeeping crisis, and the Canadian international was outstanding to keep Iain Dowie's Eagles out in the first half. Twice he made exceptional saves to deny Andy Johnson.
Palace had an amazing escape on 17 minutes when Berthelin parried Agyemang's close-range effort and then prevented Danny Granville slicing the rebound into his own net.
After Hudson's dismissal, the 15th time the official had shown the red card in 26 games, Palace dug in and deserved to open the scoring. Gillingham however gave the Eagles a hard time, but Berthelin was unbeatable as he further underlined his claims for the number one jersey.
The Eagles could have wrapped up the points at the end as top marksman Johnson missed a couple of one on ones with Hirschfeld, but nonetheless the Eagles are flying and the play-offs are a real possibility.