How do I identify this tree?

I found a tree growing in my garden - it has small white flowers in spring and then produces a small yellow plum-like fruit. I tried eating one and didn't keel over - in fact they were very nice. I at first thought it was a variety of plum but the branches have quite long spikes on them so am wondering what this can be. Any ideas please?