Jaroslav Halak stopped 33 shots, and Tom Pyatt also scored for Montreal, which trailed 2-1 through the first two periods while being outshot 25-9.
Maxime Talbot and Chris Kunitz scored 1:51 apart early in the first period to put Pittsburgh ahead. Pyatt had netted his first career playoff goal 2:34 in to put Montreal in front.
Lapierre drew the Canadiens even at 2 with a wraparound goal 2:20 into the third.
Game 5 is Saturday night in Pittsburgh.
Silenced for nearly two full periods, the crowd roared back to life after Lapierre swung around behind the net and whipped the puck off the back off Marc-Andre Fleury's right pad for the tying goal.
The fans had even more to cheer about at 3:40 when Gionta got credit for his fifth goal after his centering pass from the right side went in off Penguins defenseman Kris Letang's skate.
Fleury made 22 saves for the Penguins, who lost for the first time in five playoff road games.
Jordan Staal returned to the Penguins lineup six days after having foot tendon surgery. Staal, who missed Games 2 and 3, severed a tendon in his right foot on the skate of Canadiens defenseman P.K. Subban in Pittsburgh's 6-3 win in the series opener.
Pyatt scored 2:34 in when his sharp-angle shot beat Fleury. It was a goal Fleury would surely like to play over.
Talbot tied it 53 seconds later with his scored his second of the playoffs at 3:27, beating Halak on a breakaway for an unassisted effort.
Kunitz put Pittsburgh ahead with a power-play goal at 5:18 on a play that was reviewed because the puck went in off his left skate.
Montreal was outshot 15-6 in the first and 10-3 in the second. The Canadiens have been held to three shots in the second period five times in their last six playoff games.
Halak stuck out his right pad to stop Evgeni Malkin on a breakaway with about 4 minutes remaining. He also made a sensational lunging stop on Sidney Crosby during Pittsburgh's power play midway through the third.
Notes
Penguins RW Bill Guerin missed his second game because of an undisclosed injury. C Mike Rupp also didn't play. … Pyatt scored two goals in 40 games during the regular season. … Canadiens D Jaroslav Spacek missed his eighth game because of a virus.