What happened to this garden’s fruit tree? It wasn’t the heat.

Q. I have a very nice, fairly young (more than 5, less than 10 years) orange tree. Last year it produced a huge quantity, and the fruits were ultimately quite small, although very tasty. This year there were just as many blossoms, but somewhere in that timeframe we had hot weather and about 90% of the blossoms fell off. Thus the amount of fruit remaining was very small. Throughout the course of this summer, we have had many very hot days and I have watered the tree much more frequently than I would normally.