When the King's College building opened in 1760, students were to "lodge and diet in the College." There was a steward to keep the students' rooms clean and to make their beds. Students would have breakfast, dinner and supper at the College Hall, but they were not allowed to have meat with their suppers. After the use of the Park Place campus as a military hospital and then as barracks for the British soldiers during the Revolutionary War, the College was left with a building in need of repairs and gentle handling. The Statutes of 1785 included strict house rules to protect the building: a student could not deny the President of the College entrance to his room once a week to make sure the room was clean and in decent condition. By the early 1800s, the deteriorating facilities on the Park Place campus could no longer offer students quarters.

When Columbia College moved to its temporary, second campus on 49th Street and Madison Avenue, there were no residence halls. Students, for the most part, were left "to the comfort, the security, and the salutary influences of their home" according to Nathaniel Fish Moore, Columbia College President (1842-1849). However, with more competition for undergraduate attendance and the growth of the College, President Frederick A.P. Barnard argued for residences for the students. The college was a "day school" and the alumni wanted to foster more of the "college life" and "college spirit" they saw in the other residential colleges.

After the move to the Morningside campus, there was finally space available to build. Also, residence halls became more of a necessity as transportation to the area was not as convenient as the old campus: the subway did not reach 116th Street until seven years after the campus opened. In 1898 there was a proposal to build four residence halls on the Green along 120th Street. However, this never happened as there was no funding for this project. Based on the successful model of the Teachers College's private dormitories, there was a short-lived experiment of Knowlton Hall on Broadway and 124th Street. The building was privately owned but there was a Columbia faculty member living in the building to supervise the students, in exchange for free room and board. Unfortunately, there was not enough interest by the students and the building was sold to new owners.

President Nicholas Murray Butler finally found the financial support needed to create a dormitory system in Marcellus Hartley Dodge, CC Class of 1903. Dodge and his aunt, Helen Hartley Jenkins, funded the construction of Columbia's first residence hall. Along with donations from other alums celebrating Columbia's 150th Anniversary and college money, the first two residence halls opened in 1905: Hartley Hall and Livingston Hall (named after the Class of 1765 alum and one time Chancellor of the State of New York Robert Livingston, but renamed Wallach Hall in 1981). Ten-floors high and with rooms for 600 men, the dorms were open to all Columbia students (not just undergraduates) and proved popular. In 1906, students living on campus could take "table board" at the Commons in University Hall and, as residents, they could have assigned seats throughout the year but this option was not popular. Students would have to wait another 20 years before having a dorm with a dining hall on campus.

In 1913, another ten-story dorm, Furnald Hall, opened to meet the growing demand for campus housing. The hall was named after Royal Blackler Furnald, Class of 1901, and allowed another 300 men to live on the perimeter of South Field. During World War I lower enrollment figures impacted the use of residence halls. Furnald Hall, which had been offered over the summer session to women students, was turned into a women's residence hall for five years (1917 to 1922).

One of the last McKim, Meade & White commissions, John Jay Hall, opened in October 1926. Elevator service was not complete at the time and students had to walk up the 15-story "skyscraper" then the highest building on campus. John Jay Hall, named after the Class of 1764 alum and first Chief Justice of the United States, combined student housing with student life. It was the home to the first dining hall and included meeting rooms and offices for student groups and publications such as the Jester and the Columbia Daily Spectator. Undergraduate students were finally able to be housed together in Hartley Hall and the lower floors of John Jay. The four halls combined allowed for over 1350 men to live on South Field. Columbia did not build another men's residence hall until New Hall (renamed as Carman Hall in 1965) in 1959.