Pettitte (9-6), who had not won since July 1, allowed one run and four hits in 6 2/3 innings, walking four and striking out six. He left with runners at first and second in the seventh but Phil Hughes came on and struck out Jose Bautista to end the inning.
Hughes gave up two singles to begin the eighth, then struck out two batters before being replaced by Mariano Rivera. Vernon Wells greeted the Yankees closer with a two-run double off the wall in left, cutting it to 4-3, but Rivera got out of it when Alex Rios grounded to short.
Hideki Matsui gave New York some insurance by homering to center on Halladay's first pitch of the ninth, his 16th.
It's the second time this season Halladay has allowed three homers to the Yankees. He also gave up three in a 6-5, 12-inning loss at New York on July 4.
Rivera allowed a pair of singles in the ninth but Aaron Hill flied out to end it, giving the closer his 31st save in 32 chances.
Halladay (11-5) lost back-to-back starts for the first time in 2009, allowing four runs and eight hits in his fifth complete game of the year. The righty, who walked none and struck out five, has won just once in seven starts since coming off the disabled list June 29 after missing two turns with a sore groin.
The subject of intense trade talk leading up to July 31, Halladay got a warm welcome from Toronto fans happy to still see him wearing a Blue Jays uniform beyond the non-waiver trade deadline. For the third straight home start, the righty got a standing ovation as he walked in from the bullpen before the game.
Halladay came in 8-0 with a 2.10 ERA in his past 10 home starts against the Yankees but was behind quickly in this one as New York scored twice in the first. Johnny Damon singled and scored the opening run on a two-out double by Alex Rodriguez. Matsui reached when Halladay couldn't handle a high throw from first baseman Kevin Millar, who was charged with an error. An alert Rodriguez raced around third and was safe at home when he kicked the ball out of catcher Rod Barajas' glove with a feet-first slide.
Toronto cut the deficit in half in the fourth. Hill led off with a single, extending his hitting streak to 13 games, advanced to third on walks to Millar and Wells and scored on Rios' sacrifice fly.
Damon and Mark Teixeira hit consecutive solo shots to right in the eighth, the 10th time this season the Yankees have hit back-to-back homers. For Damon, the homer was his 18th. Teixeira's blast was his 27th.
It's the second time in his career Halladay has allowed back-to-back homers. Tampa Bay's B.J. Upton and Carl Crawford did it to him on June 5, 2007.
Pettitte made his first start since earning a $100,000 bonus for spending 120 days on New York's 25-man roster; he reached the mark earlier this week. The lefty can earn $500,000 each for 150, 160 and 170 innings pitched, and $750,000 each for 180, 190, 200 and 210 innings. He'd get $200,000 for 130 days on the roster, $250,000 each for 140 and 150 days, and $400,000 each for 160, 170 and 180 days.
Notes: Yankees RHP Ian Kennedy, still recovering from an aneurysm, threw 25 pitches off a mini-mound Tuesday and hopes to play in minor league games before the season ends. Kennedy, who had surgery May 12 to remove an aneurysm from beneath his right biceps, will throw again Friday on a mound half the usual 10-inch height. … Former Blue Jays second baseman Roberto Alomar attended the game.