Grant Shapps said "every precaution" must be taken as passenger numbers were expected to increase when lockdown measures were eased further.
Coverings must be worn on buses, trams, trains, coaches, aircraft and ferries.
Very young children, disabled people and those with breathing difficulties would be exempt, he said.
It comes as the UK recorded the deaths of another 176 people who tested positive for coronavirus, taking the total number of deaths in the UK to 39,904.
Speaking at the daily coronavirus briefing, Mr Shapps said face coverings would be "a condition of travel" and not wearing one could "ultimately" lead to a fine.
But he added: "Why wouldn't people want to do the right thing?"
He said surgical masks must be kept for clinical settings and instead travellers should wear the kind of face covering that could be made at home. They should cover the mouth and nose and can be as simple as a scarf or bandanna tied snugly behind the head.
Mr Shapps also stressed that social distancing and regular hand washing were still the most important measures to prevent the disease spreading.
The rule change coincides with the planned reopening of non-essential retail and return of some secondary school pupils in England from 15 June, which Mr Shapps said would put "more pressure" on the public transport network.
The British Medical Association (BMA), however, has questioned why the government was waiting until mid-June to bring in the new measure, saying the risk of the virus would be "much less" if it was brought in straight away.
The doctors' union also said it would like to see compulsory face coverings in other areas of public life, where social distancing was not possible, and suggested the government should supply the public with face coverings.
Earlier, Scotland's First Minister Nicola Sturgeon said her government was considering whether to make it mandatory to wear face coverings in some situations.
Scotland currently recommends wearing coverings in shops and on public transport.
In Wales, face coverings have not yet been recommended for the general public. In Northern Ireland, people have been told to consider wearing them in places where they cannot observe social distancing.
The World Health Organization (WHO) currently says only two groups of people should wear protective masks - those who are sick and showing symptoms and those who are caring for people with suspected coronavirus.
The WHO does not usually recommend them for the public because they can be contaminated by other people's coughs and sneezes, or when putting them on, or taking them off.
The government said it expects the "vast majority" of people to comply with the changes but operators will be able to issue penalty fines for those who do not, in a similar way to people who travel without a ticket.
British Transport Police will help implement the new rules.
But Network Rail chairman Sir Peter Hendy said he was not expecting a "huge surge" in staff policing these new rules.
"I am expecting sensible passengers to do their duty and look after themselves and others," he said at the Downing Street briefing.
London Mayor Sadiq Khan said he was pleased "the government has finally seen sense" after he had lobbied for the change.
Unions had been pushing for compulsory face coverings after dozens of transport workers died after contracting Covid-19.
Welcoming the announcement, Mick Whelan, general secretary of the train drivers' union Aslef, said it was a "sensible step" which would help to ease concerns of travellers.
However, Mick Cash, general secretary of the RMT union, said it was "long overdue" and face coverings alone were "nowhere near enough" to protect passengers and workers.
Labour's shadow transport secretary Jim McMahon said this latest move was "just another example of the government being slow to act".
Meanwhile, Boris Johnson has told a global vaccine summit, hosted by the UK, that finding a coronavirus inoculation will require cooperation "on a scale beyond anything we have seen before".
His comments came as the global vaccine alliance raised nearly $9bn (£7bn) at a virtual summit in London, more than their target.
The money will be used to immunise millions of children against diseases such as polio, measles and diptheria.