The attack killed at least 21 people, including foreigners, and scores were injured.
Mr Prayuth said the perpetrators had not been identified, but that police were investigating one suspect seen on security footage at the shrine.
The bomb targeted the Erawan Hindu shrine, a major tourist attraction.
It was detonated at about 19:00 local time (12:00 GMT) on Monday when the shrine, and the busy Ratchaprasong junction where it is located, were crowded.
“This is the worst incident that has ever happened in Thailand,” Mr Prayuth said.
“There have been minor bombs or just noise, but this time they aim for innocent lives. They want to destroy our economy, our tourism.”
He said there was “a suspect who appeared on CCTV but it’s not clear” and the man was being investigated.
The BBC’s Jonathan Head in Bangkok said the shrine is surrounded by security cameras and members of the public are handing in their own coverage to the authorities.
He says the bomb was clearly placed to cause maximum casualties, and that Thailand is experiencing a profound sense of shock that this could happen in the heart of their capital.
National police chief Somyot Poompummuang earlier described the device as a “pipe bomb” and said it had been placed inside the shrine. Reports said 3kg of TNT had been used in the bomb.
“Whoever planted this bomb is cruel and aimed to kill. Planting a bomb there means they want to see a lot of people dead,” he said.
Marko Cunningham, a New Zealand paramedic working with a Bangkok ambulance service, said the scene was “like a meat market”.
“There were bodies everywhere. Some were shredded. It was horrific,” he told the Reuters news agency.
The shrine is dedicated to the Hindu god Brahma, but is also visited by thousands of Buddhists each day.
It sits between a five-star hotel and a popular shopping centre on the Ratchaprasong intersection, which has been the centre of political demonstrations in recent years.