Classes were first held at King's College (now Columbia University) in 1754 inside the vestry room of the Trinity Church schoolhouse on lower Broadway. This room housed classes until 1760 when the school moved to a building on Park Place in downtown Manhattan, near the present site of City Hall. Founded by royal charter of King George II of England, King's College was the only institution of collegiate rank in New York at the time. Classes were suspended during the American Revolution in 1776 and the building was used as a barrack and hospital for both British and American troops. When instruction resumed eight years later, King's College changed its name to Columbia, in keeping with the contemporary political climate.

Classes continued in the Park Place campus until 1857, when, to accommodate its continuing expansion, the campus moved to 49th Street and Madison Avenue, occupying a tract of land previously owned by The New York Deaf and Dumb Institution. Surrounded by vacant lots and underdeveloped land, this campus was virtually rural. This location was favored to the alternative: relocating to the remote Botanic Gardens, three miles outside of New York. The forty years at this Madison Avenue campus saw the foundation of the School of Mines and School of Political Science and the inclusion of the College of Physicians and Surgeons as part of Columbia. In 1897, Columbia left the Madison Avenue campus; the following year the Berkeley School bought the land and destroyed most of the buildings.

The University made a third move in 1897, occupying four blocks in the area now known as Morningside Heights, between 116th and 120th Streets and Amsterdam Avenue and Broadway (then known as The Boulevard). This land belonged to the Bloomingdale Insane Asylum, which was owned by the New York City Hospital. Under the leadership of President Seth Low, the architecture firm of McKim, Mead & White was commissioned to design an urban academic village on this site and the asylum's land was forever transformed with the domed Low Memorial Library overlooking a stately collection of Renaissance influenced buildings. The initial phase of construction between 1895 and 1897 saw the erection of Low Library, Schermerhorn Hall, Fayerweather Hall, and University Hall (later demolished).

The expansion of the Morningside Heights campus continued steadily throughout the twentieth century with St. Paul's Chapel, the School of Mines (now Lewisohn Hall) and Hamilton Hall, all constructed between 1903 and 1907. Kent Hall, Philosophy Hall and Avery Hall were constructed between 1909 and 1911; the early 1920s saw the completion of Dodge Hall, John Jay Hall and Pupin Hall. Expansion continued throughout the ensuing decades, with the School of International Affairs completed in 1970 and a large East Campus Housing project developed between 1977 and 1981. The most recent major addition to Morningside Heights' campus is Alfred Lerner Hall, a modern glass structure that replaced Ferris Booth Hall as the student center in 2000.

The University began expansion plans in 2004 in the Manhattanville area, also named "West Harlem" bordered by 129th to 133rd Streets and between Broadway and 12th Avenue. The proposed construction is expected to provide a total of 6.8 million square feet of space above- and below-grade for teaching, academic research, and civic and commercial activity, as well as below-grade parking and facilities support. This is a multi-phased project with completion expected in 2030.