The teams combined for 74 minutes in penalties, including 64 in the second period.
Devils defenseman Brenden Dillon, who was playing in his 1,000th NHL game, was forced to the locker room after a fight with Dmitri Voronkov at 1:01 of the second period. Dillon returned to play 4:10 in the second but didn't play in the third period.
"After [Dillon left], emotions were high," Hischier said. "It was 2-2, the game was open, and some mistakes cost us."
New Jersey also was without defenseman Jonas Siegenthaler, who was given a game misconduct for not having his jersey fastened properly during a fight at 3:58 of the second period.
"I don't like how we handled those situations emotionally," New Jersey coach Sheldon Keefe said. "Obviously not long after the situation with [Dillon] they scored so I don't like that. We've got to do a better job of staying in it emotionally, as hard as it might be."
Hischier scored a power-play goal at 1:26 of the first period to give New Jersey a 1-0 lead when his centering pass deflected in off the stick of Columbus defenseman Ivan Provorov.
Palat extended the lead to 2-0 on a wrist shot from the slot at 3:03.
Denton Mateychuk pulled the Blue Jackets within 2-1 with a power-play goal on a redirection from the slot at 9:54.
The Devils appeared to take a 3-1 lead 17:08 into the first when Timo Meier scored a power-play goal off a scramble from the right post, but the Blue Jackets successfully challenged the play for goalie interference on Stefan Noesen in the crease.
Monahan tied it 2-2 at 1:28 of the second period during 4-on-4 play on a wrist shot from the left hashmarks.
Coyle converted his own rebound at the left post at 6:40 of the third period to give Columbus a 3-2 lead, before Monahan scored 34 seconds later on a snap shot outside the right post for a 4-2 lead at 7:14.