What is this and how do I get rid of it?!

I have just moved into a new house and there are woody stems (Picture attached) growing all over the garden, at both the front and back of the house. Many have been cut down by the previous owner but continue to grow back. They are even coming up through gaps in the paving slabs etc. Do any of you know what they are and how I would be best to get rid of them and stop them coming back??

 

Tree seedings ? Have you any mature sycamore or ash trees nearby?

 

The very dark tips suggest Ash to me too. Yes, dig them right up otherwise they will persist and will be really difficult to get rid of in a year's time.

 

Ok, thanks for the advice, I'll keep digging them up! The garden does back onto some mature wooodland which I guess could be the reason for them.

 

I hate these, they grow everywhere. I use to love watching the seed pods when i was a kid, we called them Helicopter seeds.
Every time I go in the garden I spot more of them growing, and yes they are very annoying to pull up when they have grown a little, they dig deep.