There was something seriously wrong with the Victorians. Their architecture has an inclination to ugliness that defies explanation by the shifting tides of tastes. So much of it is wilfully challenging, even visually hostile.
After millennia of experience, jobbing builders and, since 1834, professional architects acquired certain rules about proportion and detail that were generally agreed to work well, both practically and artistically. These so many Victorian buildings contumaciously defied. We look on them now with blank horror.
Of course, the thing about taste is that it does change. What is acceptable in one era is despised in the next, only to be revived later. The Bloomsbury aesthete Lytton Strachey said we can be absolutely certain that nobody will want to revive the Victorian era. Half a century later we had Laura Ashley.