On October 21, 2013, he began to play with the Cleveland Browns, where he recorded a season-high of 6 solo tackles during a 27-16 loss at the New England Patriots.
On October 16, 2016, Poyer collected 3 solo tackles before exiting the Browns 28–26 loss at the Tennessee Titans, due to an injury. He was diagnosed with a lacerated kidney and a possible concussion. On October 18, the Cleveland Browns placed Poyer on injured reserve.