During the late 1800s Manhattan's East Harlem experienced a dramatic increase in population and economic activity as transportation lines were extended into the area and new housing was constructed. Successive waves of Irish, German Jewish and Italian immigrants moved into tenement buildings and formed a vital community establishing small businesses benevolent societies and fraternal organizations. But the dense concentration of population also gave rise to a host of urban problems poor housing inadequate health care lack of educational opportunities and crime.

Such an increase in the social problems attending urban growth had led reformers and philanthropists in England to establish Toynbee Hall the first settlement house in 1884. Originally distinguished by the commitment of educated upper and middle-class volunteers to settle in working class communities in order to understand their problems first-hand the settlement model was imported to the United States in 1886 when Neighborhood Guild was established on New York's lower east side. American settlement houses were in the vanguard of efforts to provide social services in their neighborhoods through such programs as kindergartens day care hot lunches health clinics visiting nurses camps playgrounds and arts education. In addition the settlements joined Progressive-era reform movements for improved housing public health and sanitation.

In 1898 Anna C. Ruddy a Canadian who had done missionary work in East Harlem for the previous eight years decided to establish a settlement house in the neighborhood. Ruddy opened The Home Garden in a tenement on East 115th Street but soon moved to 322 Pleasant Avenue where the settlement was legally incorporated in 1901. Its early focus was on programs for children including clubs a day nursery and classes in sewing cooking and carpentry. There was also a strong religious emphasis through Bible classes and a Sunday school. During the 1910s under the leadership of Headworker Bertha Muriel Gage the settlement broadened its focus to the adults of East Harlem through mother's clubs debating groups and citizenship classes. Children's activities were expanded to include Fresh Air Camp a Boy Scout troop and health care by hosting a district nurse sponsored by Henry Street Settlement. To facilitate these activities the settlement moved to larger quarters at 405 East 116th Street and eventually to 311-316 East 116th. In 1916 the settlement's name was changed to Haarlem House.

During the 1920s and 30s East Harlem's demography began to shift as Puerto Ricans and African-Americans moved into the neighborhood. Many other ethnic groups remained as well reflected by a 1926 news clipping which reports that people of 27 different national origins participated in Haarlem House activities. Italian-Americans were well represented among these and Director of Men's Work Edward Corsi coordinated many citizenship programs oriented to them. Corsi also served briefly as Executive Director of the settlement before going on to a political career that included positions as U. S. Commissioner of Immigration and New York State Industrial Commissioner. In 1955 he returned to the settlement as Executive Director then joined its Board of Directors.

In 1956 Haarlem House was re-named LaGuardia Memorial House in honor of former East Harlem congressman and New York Mayor Fiorello LaGuardia. During this period the settlement operated a music school and worked to improve relations between Puerto Rican and Italian-American youth in the area. In 1967 LaGuardia Memorial House transferred title of its East 116th Street property to the New York City Housing Authority which built new housing for senior citizens on the site as well as a new community center facility which has since been operated by the settlement. In 1973 Peter Pascale became Executive Director of LaGuardia Memorial House. Pascale had grown up in East Harlem using the settlement and first joined its staff as Athletic Director in 1931. Both he and his successor Jim Soler maintained the agency's historic commitment to the youth of East Harlem through such programs as day care after school recreation summer day camp and athletics.