It’s not memorizing that’s the problem, a friend said once. It’s having to perform from memory. This 2013 New York Times article discusses the controversy surrounding the de facto requirement of playing works from memory, raising the valid point that, up to the time when overachiever Franz Liszt decided to play concerts memorized, most concerts were performed with music.
In earlier eras there was composed music, which was always played from the score, and there was improvised music. Since performers were almost always composers as well, as Mr. Hough explained, for a pianist to play, say, a Chopin ballade from memory would have been considered the height of arrogance, as if the pianist were suggesting that he had composed the piece.