During the late 19th and early 20th centuries the East Side Manhattan area of Kip's Bay, extending from 34th to 59th Streets between Third Avenue and the East River, was a densely populated working class neighborhood. Immigrants from Poland, Germany, Italy and eastern Europe lived in the area's crowded tenements and brownstones, and worked for such east side businesses as slaughterhouses, breweries and warehouses. In their leisure hours they established benevolent societies and fraternal organizations, and attended local churches and synagogues. But even as working class culture flourished, the dense population exacerbated a host of problems. Poverty, hunger, disease, crime, decrepit housing and unsanitary streets were pervasive here as elsewhere in New York and in rapidly growing cities across the country. Such conditions dimmed the hopes of many immigrants. They also alarmed many wealthy and middle-class Americans who perceived them as threats to moral order, political stability and cultural progress. Early attempts to ameliorate conditions in a changing urban society included the creation of charity organizations, industrial training schools, and church missions.

In London, a similar increase in social problems led reformers in 1884 to establish the first settlement house, Toynbee Hall. The settlement model, originally distinguished by a commitment on the part of its college-educated volunteers to "settle" in working class communities in order to confront their problems first-hand and to contribute to the moral uplift of their neighbors, was quickly imported to the United States. In 1886 Stanton Coit, a devotee of Felix Adler's Ethical Culture movement and early observer of the experiment at Toynbee Hall, founded The Neighborhood Guild (later renamed University Settlement) on Manhattan's Lower East Side. Over the next several decades settlement houses were established in cities across the country, staffed largely by recent college graduates, many of them young women eager to take an active role in public life. American settlements sponsored such programs as kindergartens, day care, social clubs, health clinics, visiting nurses, summer camps, arts education and vocational training. They served as observation posts for sociologists, journalists, and other researchers of urban conditions. Many settlements provided forums for public debate of political issues, and galvanized popular opinion in support of progressive social legislation.

In March of 1915 twenty women inspired by the settlement model, including Madelaine T. Astor and Charlotte Grosvenor Wyeth, rented a basement at 405 East 50th Street where they operated a day care and lunch program for neighborhood children. A year later Grosvenor Neighborhood House was legally incorporated, and the activities of the institution quickly grew to include the traditional range of settlement house programs such as clubs and classes, adult social and educational groups, a circulating library, athletics, and country camping trips. These activities were led by a staff of volunteers, residents and "Head Workers" that included during the early years Mrs. Chalmers Charles, Mrs. Emma A. Stafford, and Maude I. Purnell.

To enlarge the work of the settlement, the Grosvenor Board of Directors in 1922 purchased two brownstones at 321 East 49th Street. The new Grosvenor Neighborhood House included space for art classes, vocational training, a summer play school sponsored by the Child Study Association, and medical examinations. In addition, for several years during the 1920s Grosvenor hosted the Turtle Bay Music School, which offered instrumental and vocal lessons to community residents for a low fee. During the 1930s depression, a strong emphasis was placed on "relief work" or the distribution of food, clothing and coal to impoverished families. This was continued into the war years, when Grosvenor also sponsored Red Cross and Civilian Defense activities, and a nursery school. After the war Grosvenor further expanded through the establishment of a senior citizen program. By 1951 annual attendance for all settlement programs was over 80,000.

But during the next several years shifting demographic and real estate patterns severely reduced. Grosvenor's membership rolls. The construction of the United Nations complex, the expansion of medical and educational institutions, and the construction of luxury housing displaced many low-income residents who were the settlement's traditional constituency. In 1957 the Grosvenor Board of Directors decided to move the settlement to a community where felt its services would be more essential. The New York City Planning Commission had recently announced plans for a "West Side Urban Renewal Project" (WSURP) entailing the demolition and rehabilitation of tenement buildings and the construction of public housing projects in an area bounded by 87th and 97th Streets between Amsterdam Avenue and Central Park West. With the support of city and housing officials and the encouragement of other settlement house leaders Grosvenor Neighborhood House moved to the WSURP area. Under the leadership of Executive Director Jerome Janowitz social service and recreational programs were provided from a storefront on West 87th Street and in the gymnasium of a local private school. In an effort to check juvenile delinquency and gang violence, special emphasis was placed on group work with teenagers. The settlement also encouraged community involvement in WSURP through participation in such community organizations as the Park West Neighborhood Association.