Lemon Drop Chilli

My Lemon Drop Chillies really aren't 'doing' for some reason. In the early stages, they threw on growth really rapidly, and became almost leggy (I wondered if they had spurted due to being fed chicken manure), thereafter they slowed and produced a few fruits and I thought they had settled.

The fruit produced hasn't ripened, a few have simply dropped off, and there is no sign of more blooms coming; to make matters worse, it would appear that something has been eating them as the leaves are all peppered with holes! I can see nothing insect wise lurking on the plants, however they are sitting on the sill of a window that is mostly open, so its not beyond the realms that something has flown in, had lunch and flown out again.

I am on the verge of ditching them, and growing some herbs instead, however didn't want to be hasty - are these chillies really fickle, or have I simply dropped the ball with them?

 

My lemon drops must have heard me - two of them are turning yellow! Still don't have much fruit though.