Coach Tom Coughlin announced the move late Thursday evening after Fewell spent the past few days weighing offers from the Giants and the Chicago Bears.
Fewell also interviewed for the vacant Bills coaching job but he has said for a while that he felt he was a long shot for that position.
Giants rookie defensive coordinator Bill Sheridan was fired the day after the regular season ended. The Giants gave up more than 40 points in each of their last two games and 427 points for the season, ranking third from the bottom in the NFL. Only the Detroit Lions and St. Louis Rams gave up more.
Fewell and Coughlin have known each other for years.
Fewell spent five seasons as Coughlin's secondary coach in Jacksonville. He spent the last four as the Bills' defensive coordinator and was made their interim head coach for the final seven games of the 2009 season after Dick Jauron was fired. The Bills went 3-4.
"I expect Perry to bring the same qualities that I expect from myself," Coughlin said. "I want him to be firm, fair, honest and demanding. My expectation is that he will solidify and unify our defense and be an outstanding teacher. I want energy, enthusiasm, toughness and to make the necessary corrections and game adjustments. Perry is a teacher and a leader and I thought he did an outstanding job of displaying great leadership as the interim head coach of Buffalo this season."
The Giants and the Bills both use a 4-3 defensive scheme.
Both teams struggled with injuries on defense this season. The Giants lost safety Kenny Phillips (knee), middle linebacker Antonio Pierce (neck) and tackle Jay Alford for the season and had cornerbacks Aaron Ross and Corey Webster, linebacker Michael Boley and defensive tackle Chris Canty sidelined for significant portions of the season.
The Bills had five starters go on injured reserve.
"I am ecstatic to come to work for the New York Giants," Fewell said. "It's a good football team with good defensive players, and it is a franchise known for defense."
The Bills struggled stopping the run (30th overall), but the defensive backs led the AFC with 28 interceptions, and were second in the NFL behind only Green Bay.
Rookie safety Jairus Byrd tied for the NFL lead with nine interceptions.
The Bills' defense allowed 14 touchdown passes in each of the last two seasons. The Giants gave up 31 in 2009.
The Giants had 24 takeaways and 13 interceptions in 2009.
In his four years as coordinator, the Bills' defense ranked 18th, 31st, 14th and 19th in the NFL, and that was with an offense that did not produce much or control the ball.
In 2009, Buffalo allowed 340.6 yards and 20.4 points a game. The Giants' gave up an average of 324.9 yards and 26.7 points.
The year before, the Bills were fourth in the NFL in red zone defense, allowing a touchdown on only 41.8 percent of possessions inside the 20-yard line.
Fewell coached Chicago defensive backs in 2005, when the Bears led the NFC with 24 interceptions and cornerback Nathan Vasher and safety Mike Brown were selected to the Pro Bowl. It's one of the reasons Lovie Smith wanted him back as coordinator.
"His defenses have consistently done a good job taking the ball away," Coughlin said.
Fewell was the secondary coach of the St. Louis Rams in 2003-04. He entered the NFL as the defensive backs coach for Coughlin in 1998 and stayed there through 2002. Jacksonville's pass defense ranked third in the NFL in 1999 and two years later the Jaguars gave up only 13 touchdown passes.
"We brought Perry into the league in Jacksonville," said Coughlin, "and I was impressed with his thoroughness, his ability to work with the players and his absolute intent on learning all he could about his profession. He was hired as the secondary coach and he has had great experiences in St. Louis and Chicago with Lovie Smith and that system and in Buffalo with Dick Jauron, who was my original defensive coordinator in Jacksonville."
Fewell was a college coach for 13 years before coming to the NFL working at North Carolina, Army, Kent State and Vanderbilt.
"I was a young coach, and he helped develop my philosophy of the game, the discipline, the know-how, the toughness that it takes and the attention to detail. Being prepared. That is what he instilled," Fewell said of his time with Coughlin.