The home side staged a late rally but the Dons coped with all they could muster and held on for the three points.
Falkirk started with confidence and Russell Latapy carved out a chance for Carl Finnigan but, with Jamie Langfield advancing to meet him, his lob was just too high.
Latapy tried his luck himself moments later but his low shot from 30 yards was easily gathered by Langfield.
Alan Gow and Finnigan combined well up front for Falkirk and Gow set up another chance for his strike partner after 12 minutes, but Finnigan looped his shot over.
But it was Aberdeen who made the breakthrough four minutes later. Barry Nicholson's corner found Andrew Considine at the back post and his header was bundled out by Dean Holden to the edge of the area, but Dempsey reacted first to slam a right-foot shot through a crowd of players into the corner of the net.
Falkirk had an appeal for a penalty when a Gow free-kick hit the Aberdeen wall but referee Iain Brines was not interested.
But Aberdeen squandered a great chance to double their lead three minutes from the break. Nicholson threaded the ball through to Darren Mackie with Falkirk caught short at the back, but he failed to make proper contact and the chance was lost.
It was almost more painful for Aberdeen as the Bairns broke clear but Gow's tame final effort was comfortably saved by Langfield.
Aberdeen doubled their lead after an hour when Foster scored with a left-foot shot after Mackie had cut the ball back.
But Falkirk pulled a goal back six minutes later when Finnigan played in Holden and he scored with an assured finish.
However, that was all Falkirk could manage despite storming forward in the closing stages.