The former Good Morning Britain presenter had been a judge on the ITV competition series that year, which saw Paul Potts crowned winner, alongside Amanda Holden and Simon Cowell.
Writing in his Mail on Sunday column, Piers recalled how he met the Queen and the late Duke Of Edinburgh, who died last week at the age of 99, at the backstage VIP area.
‘Did you enjoy Paul’s performance?’ Piers said he asked her majesty.
He claimed that she replied: ‘Oh yes. [Looking at Paul] You were very good.’
But Prince Philip was less cordial, and looking at Piers as he asked: ‘You’re judges, is that right?’
Piers replied: ‘Yes, sir.’
He continued: ‘”So,” he [Prince Philip] scoffed, peering at the three of us with undisguised scorn before pointing to Paul, “you lot sponge off him then?”‘
Prince Philip was laid to rest on Saturday in a ceremony that took place in St George’s Chapel at Windsor Castle.
The BBC’s coverage of the funeral was watched by an average of almost seven million people, with the corporation devoting almost four hours to the event.
The late royal’s coffin was draped with his personal standard, featured his Admiral of the Fleet naval cap and sword next to a wreath of white flowers chosen by the Queen with a handwritten card from his wife of 73 years.
At 3pm on Saturday the nation came to a halt to observe a minute’s silence in memory of the duke who died a few months short of his 100th birthday.
The country was in national mourning from Philip’s death on April 9 until and including the day of the funeral.