A three-storey building collapsed in Nigeria's main commercial city of Lagos on Sunday during torrential rains, killing at least five people and seriously injuring scores more, rescue workers said.

The building, situated inside a railway compound near the densely-populated Ebutte Metta neighbourhood, collapsed in the early hours of Sunday morning, police spokesman Frank Mba said.

"The ground floor is used for warehouses. The first and second floors are residential," Mba said.

A rescue worker who asked not to be identified said the building was believed to house around 50 residents, at least 30 of whom had so far been pulled out alive. At least five people were believed to have been killed.

A spokeswoman for the Nigerian Red Cross said around 45 injured had been taken to hospital, mostly with fractures.

Substandard materials and disregard for building regulations mean such collapses are not uncommon in Africa's most populous nation, where infrastructure is old and poorly maintained.

At least 11 people were killed in March when a four-storey residential building collapsed in Lagos.

A similar building collapsed in 2006 killing 28 people, although the death toll was high because people were using a restaurant, bar and shops on the ground floor at the time.

Torrential rains and heavy thunderstorms in recent weeks have left some parts of Lagos, a sprawling city of 14 million people, submerged in more than a metre of water.

One rescuer said the building which collapsed on Sunday was believed to have been about 40 years old, and had been considered a more solid structure than some hastily-built newer residential and office blocks.