Mr Johnson will step down as PM in September, but said he was "making sure there is extra cash" for his successor to provide additional support.
He also said people were starting to receive the money which the government announced earlier in the year.
The topic of rising bills has come to dominate the Tory leadership contest.
The two candidates to replace Mr Johnson have set out differing strategies on how they would respond to the cost of living crisis.
Liz Truss has argued that cutting taxes is the best way to help struggling households.
Rishi Sunak has proposed spending billions of pounds on further payments to pensioners and people on low incomes, contending tax cuts will not help them enough.
Elsewhere Labour has set out its own proposals including a plan to ensure pre-pay energy customers are charged the same as those paying by direct debit - a move they say would benefit four million people.
Speaking in Edinburgh, Labour leader Sir Keir Starmer said it was "outrageous and immoral" that the poorest people on pre-payment meters paid more for their energy bills.
Both Labour and the Liberal Democrats have also called for the government to raise money by backdating the windfall tax on energy companies.
Speaking at a hustings in Cheltenham, Ms Truss said she "absolutely" did not support windfall taxes, arguing it was "all about bashing business".
Mr Johnson has made it clear that he thinks further support will be needed from whoever succeeds him at the start of September.
But he isn't going to make that decision himself and several government figures have made it clear the big decisions can not be taken by the outgoing regime.
The lack of immediate action infuriates people like Martin Lewis, the money saving expert, and angers some opposition politicians too, who say the scale of the energy crisis is such that the government can't wait a month to decide what it's going to do.
But whatever options are suggested by the opposition or left on the table by the current government, it looks increasingly clear that concrete decisions are going to have to wait.
Not least because the two people who could be making them - Rishi Sunak and Liz Truss - sharply disagree.
Conservative members are weighing up which they prefer.
But while they decide, decisions over help with the cost of living look like they are going to have to wait.
Last week forecasts suggested that energy bills could rise to more than £4,000 per year next year for typical households.
Earlier this year the government announced a £15bn package including a £400 energy bill discount for every household.
Asked by reporters if this was enough, Mr Johnson said: "No", adding: "What we're doing... is trying to make sure that by October, by January, there is further support".
"I'm not going to pretend that things are easy for people right now, but there's more money coming anyway as a result of the decisions we've taken.
"I think over the medium term, the future's very bright."
He also argued that the price of energy would be brought down in the longer term by the government's investment in nuclear power and renewables.
Earlier Chancellor Nadhim Zahawi said new cost of living policies would be "ready to go" when either Mr Sunak or Ms Truss took office on 6 September.
Speaking on a visit to the Airbus factory in Broughton in north Wales, Mr Zahawi told broadcasters: "We're looking at all the options of what additional help we need to bring in later on in the winter months."
The BBC has been told one option could include expanding the windfall tax on energy companies.
The current windfall tax - also called the energy profits levy - applies only to oil and gas firms, but there has been speculation that it could be extended to electricity generators.
Boris Johnson: Energy bill support is not enough