Aphids are pretty easy to control so don’t get rid of the tree. Spray thoroughly once a week for 3 weeks with Bonide Bon-Neem organic spray. Another option would be to use a systemic like Bonide Annual Tree & Shrub Insect Control…a liquid you’ll dilute with water and pour at the base of the tree, The tree takes it up and it kills any insect on or in the tree for the entire growing season.