Hoffman, trying to protect a 3-2 lead, walked Albert Pujols with one out before serving up a 2-1 pitch that Holliday drove over the wall in center.
It was only the third blown save for Hoffman (1-2) this season, but it spoiled what would have been a remarkable victory for Carlos Villanueva. He was forced to fill in when starter Manny Parra had neck spasms after throwing 21 pitches retiring the Cardinals in the first inning.
John Smoltz had his first somewhat rocky start in a Cardinals uniform, lasting five innings and allowing three runs on six hits with seven strikeouts.