To make the difficult decision of who was going through to the live shows and who was going home, Gary joined forces with Cheryl Cole at the stunning Northamptonshire estate, Boughton House.
Home of the Duke of Buccleuch, Boughton has been nicknamed 'The English Versailles' because of its grand style, vast designer gardens and rooms packed full of fine art.
The landscape surrounding the building is Grade 1 listed to it's the perfect spot for a summer day out. The estate isn't always open to the public, so if you want to visit you're going to have to book in advance.
The X Factor team aren't the first camera crew to take over Boughton House though. Earlier this year scenes for Les Miserable starring Hugh Jackman, Anne Hathaway and Amanda Seyfried were filmed there.
If you want to experience Boughton House and the surrounding gardens for yourself click here.