Taking into account that, according to Himmler,
'Archaeology is the science of Aryan supremacy in the beginnings of civilization'
(true) it is easy to see why Adolf did not think much of Himmler's efforts in digging ancient Germanic wooden huts in Pomerania, or in sending expeditions to measure funny crania in Tibet.
Having some talent in the field of 'les Belles Artes'
(at least as much as Churchill) Hitler fully appreciated that maintaining that 'Germanic genius was already showing in these wooden huts' while the Greeks were building the Parthenon was something a little off the mark, not to say plainly ridiculous.
So it was Himmler, not Hitler, who thought of the Parthenon as 'ruins'. In fact, this is the very first occasion in which the faithful servant dared to voice a difference of opinion with his master Adolf.
But newly unclassified sources mention that the real reason behind Himmler's hate of the Parthenon lies in the fact that on the one ocassion in 1936 in which he mustered courage to visit an SS brothel a Greek whore laughed at his 'little ruin'.