Underperforming Hibiscus

I have a hibiscus plant growing in a pot in the middle of the garden. The plant has been there for more than five years but this year it has failed spectacularly. Its dropping leaves, is very spindly and has barely flowered and when there have been flowers they are tiny. Should I prune it now and remove all the leaves that are left or just chuck it in the garden waste?

 

Possibly pot bound, please don't chuck it in the bin. I had one growing at the top of my garden, never had any flowers plus I never fed or watered it.....so last season I decided to water & feed it and it bloomed. This season I'm being kind to it waiting for the flowers to come.

 

I grow them in the ground.
Always flower every year.
A picture would be good.

Does sound like it's starved although they can take a bit of neglect.

You can prune them fairly hard in the winter but I don't think that is going to solve the problem here.