South Africa, Brazil and Mexico come off the red list, which requires travellers to quarantine in an approved hotel at their cost for 10 full days.
Transport Secretary Grant Shapps said the changes begin on Monday and "mark the next step" in opening travel.
This latest move will be seen as a boost to the airline industry and families separated during the pandemic.
Panama, Colombia, Venezuela, Peru, Ecuador, Haiti and the Dominican Republic remain on the red list.
Pandemic travel rules in the UK have recently been simplified, with the amber list cut, and advice against holidays changed for 32 countries.
But consumer group Which? warned the changes only reflect requirements for arriving back in the UK.
"Travellers should be aware that they may still face restrictions on entry to many destinations, especially those under 18 who are not yet vaccinated," it said.
Arrivals from 37 more destinations will have their vaccination status certificates recognised, meaning they can avoid more expensive post-arrival testing requirements.
Vaccinated travellers from Brazil, Hong Kong, India, Pakistan, South Africa and Turkey will be treated the same as returning fully-vaccinated UK residents so long as they have not visited a red-list country in the 10 days before arriving in England.
All arrivals will still complete a passenger locator form.
The Scottish government said the changes were "agreed on a four-nation basis".
The Welsh government said that they increased opportunities for new infections and variants, but it was adopting them because it was not practical to have its own border policy.
For British expats Matt and Hannah Pirnie, who have lived in South Africa for a decade, the country's removal from the red list will mean it is easier to see family again.
"It's been a long pandemic for us. Not seeing family, not being allowed to go back, but more importantly grandparents not being able to come here and see their grandkids. It's been a long two years," Matt says.
"First of all when all the aeroplanes stopped initially - that was quite anxiety provoking - and then to be put on the red list for so long has just been quite hard to wrap your head around why," Hannah adds.
"Taking three children into a prison-like mentality was just a no-go, plus the cost. It's been quite hard really."
Announcing the latest changes, Mr Shapps said the government was "making it easier for families and loved ones to reunite".
He said that with fewer restrictions "and more people travelling, we can all continue to move safely forward together along our pathway to recovery".
In addition to the shorter red list, the government said passengers would soon be able to use a photograph of a lateral flow test as a minimum requirement to verify a negative result.
This change - affecting tests taken by eligible fully-vaccinated people from non-red list countries two days after arrival in England - would come into effect in "late October", the Department for Transport (DfT) said.
A UK government source said the government still aimed to replace the so-called day two "PCR test on arrival" with a cheaper lateral flow test by the half-term break, for many schools in England after 22 October.
But they said the government was still working on a date for when the new testing rule would be introduced.
Under current rules, travellers must use more expensive PCR tests for their post-arrival day two screening. People who are not fully vaccinated must provide a further PCR test on day eight.
The DfT said NHS lateral flow devices cannot be used for the purpose of international travel. "Both pre-departure tests and on arrival tests must be bought from private providers," it said.
Airlines and the travel industry praised a "much-improved system" but called on ministers to implement changes to testing as soon as possible and consider scrapping tests for passengers arriving from low-risk countries.
A spokesperson for London's Heathrow Airport said the announced changes would "kick start a global Britain".
"However, the missing piece to this is clarity on when cheaper lateral flow tests will be accepted, which is now critical in order to save the half-term getaway for many," they said.
A further 40,701 new coronavirus cases were reported in the UK on Thursday, alongside another 122 deaths within 28 days of a positive test.
The following destinations will be removed from the red list from 04:00 BST on Monday:
Afghanistan, Angola, Argentina, Bolivia, Botswana, Brazil, Burundi, Cape Verde, Chile, Congo (Democratic Republic), Costa Rica, Cuba. Eritrea, Eswatini, Ethiopia, French Guiana, Georgia, Guyana, Indonesia, Lesotho, Malawi, Mayotte, Mexico, Mongolia, Montenegro, Mozambique, Myanmar, Namibia, Nepal, Paraguay, Philippines, Réunion, Rwanda, Seychelles, Sierra Leone, Somalia, South Africa, Sudan, Suriname, Tanzania, Thailand, Trinidad and Tobago, Tunisia, Uganda, Uruguay, Zambia and Zimbabwe.