Rockery soil

I've put some new little plants in the rockery after clearing it up a fair bit. There's a lot of stones in the soil so I want to infill the gaps with new soil to make it look nicer. I was going to use normal potting compost partly due to it maybe benefiting the new plants settling & the dark colour of it offsets nicely I've googled this & google is it's usual noncommittal self... Am I ok covering the gaps, some of which are currently a decent size until the plants start spreading, or would I be better with topsoil?
Thanks

 

If the plants are rockery ones, they will like it very well drained, so (as I did with mine) mix in plenty of grit usually 50/50mix, with your favourite planting mix. The plants will appreciate the extra drainage.

 

I meant little stones

I did have a feeling as well, as soon as it rains it'll all go pear-shaped then

 

I've already had to take a fork to it all to unflatten the soil. Reggie hasn't helped with the trampling either

 

Thanks for the replies topsoil seems to be the way to go then, and give it all a good stir it's already quite gravelly, with the smaller stones in the soil