Balliol College was founded in 1263 by John Balliol, one of King Henry III of England's Lords. It was originally a house for maintaining sixteen poor students. In the mid 19th century there were plans for the great architect Pugin to rebuild Balliol. The scheme was quashed but his drawings remain in the college.

The architect later chosen was Alfred Waterhouse whose design, with a central gate tower, owes much to Pugin’s original plans. This was his first Oxford commission although he later became one of the richest 19th century architects. The buildings of small blocks of Bath stone pointed with dark mortar, were erected between 1867-8. They are carefully varied in height and dominate the western end of Broad Street.