Part filling in my ponds

I have two ponds in my garden which i created years ago they are about two foot deep and i want to part fill them. if i use normal garden soil will it be to fertile and cause algae to grow? Any ideas?

 

I think normal soil will turn the water green. I d use aquatic compost if possible to lessen the nitrates in the water. Might be better to part fill with stones /stone and as little soil/compost as possible.