Did you mound soil up around the potato plant as it grew? If not, then it probably put all its energy in the one potato that it could grow instead of into many potatoes had the soil been mounded up as it grew taller. Your yields of 3-6 potatoes per plant seem very low to me…are you growing in the ground or containers?