Jacob Markstrom made 18 saves for the Devils (26-17-6), who have lost four in a row (0-2-2), including 3-1 here to the Philadelphia Flyers on Saturday. They are 2-6-3 in their past 11 games and have scored 20 goals over that stretch.
"Our best players have to be better ... it's that simple, end of statement," New Jersey coach Sheldon Keefe said.
Forsberg's best save in the first came when he stretched out his left pad on a backhand from the slot by Nico Hischier at 17:12 during a Devils power play.
The Devils outshot the Senators 16-2 in the first.
"I thought we were great defensively," Keefe said. "In the second, we lost some puck battles or pucks that we turned over inside our zone and they got some chances, but our structures were fine, which is why we're hanging around in these games and not getting blown out.
“I mean, defensively, we're fine. We just have to execute better."
Ostapchuk gave the Senators a 1-0 lead at 13:16 of the second period. The 21-year-old center stole the puck in the right face-off circle and spun around for a wrist shot through a screen that beat Markstrom between his hip and arm at the left post.
It was Ostapchuk’s 39th NHL game.
"It was a ton of relief (to score)," Ostapchuk said. "I've never gone that long without scoring, so hopefully the second one comes a little quicker. I didn't even see the goal; I saw [forward Matthew Highmore] celebrating behind the net, and then I kind of blacked out a little bit. It was a great feeling."
Tomas Tatar tied it 1-1 at 14:34, getting his own rebound and scoring on a backhand from the slot. It was his first goal in 20 games (Nov. 23).
"I think we had a great start to the game [but] as the game went on, we stopped playing," Tatar said. "When you're in a position like this, you just have to simply things and it's going to come eventually. We can't just be doing whatever out there. We have to simplify."
Artem Zub scored his second goal of the season at 2:50 of the third period, a wrist shot from point through a screen for the 2-1 final.
"I just saw an open spot, and [Shane Pinto] made a good pass to me and I tried to shoot past their guy (Nathan Bastian) and it went in," Zub said.