The necessity for an Armoury in Summerside arose after the Drill Shed on Spring Street burned to the ground in the Great Fire of 1906. In 1908 the Dominion government purchased land for a new building to be located south of the Post Office, now part of City Hall, on Summer Street. The contract was awarded in October 1910 to M. F. Schurman & Company but excavation for the 38 x 34 foot cellar did not begin until June 1911. The two-storey structure was completed by the following summer and the formal transfer from the Department of Public Works to the 82nd Regiment took place June 22, 1912.

Members of the No. 3 Company of the 82nd Regiment (Abegweit Light Infantry) held their first annual drill in the Summerside Armoury in May 1913. The following year B Squadron of the PEI Light Horse was organized. Many of the local militia members enlisted in the Canadian Expeditionary Force at the beginning of the First World War. In the early months of 1916 new recruits used the grounds, known as Armoury Square, for regular drilling. The men constructed trenches complete with dugouts and loopholes and used dummies to practice bayonet fighting.

In the early 1920s the Armoury Square was the convenient location for Chautauqua, the adult education movement that spread across North America in that decade. Its purpose was to bring entertainment and culture to communities through presentations by prominent speakers, musicians, preachers and entertainers of the day.

After the war, veterans established a Great War Veterans Association (GWVA) clubroom in the Armoury and continued to use the facility after obtaining a branch charter for the Royal Canadian Legion in 1931. During the 1930s C Company of the PEI Highlanders, which had replaced the 82nd Regiment in 1922, kept uniforms and equipment in the building and held drills in the local curling rink.

The Armoury was used on a regular basis for army recruitment during World War II. It was also the headquarters of B Squadron of the 17th (Reserve) Armoured Regiment. By 1946 the federal government admitted that the facility was “old and out of date.” However, the structure continued to be used by the local squadron of the PEI Regiment until June 1992 when the forty members of B Squadron began to operate out of a building in Slemon Park, the former Canadian Forces Base Summerside.

Public Works Canada, acting on behalf of the Department of National Defence, sold the structure to the City of Summerside in 1996. The municipal recreation department had offices in the building until the new City Hall was opened in 2003. The Armoury is still owned by the City and is currently housing the International Fox Museum and Gallery 33.

In 2011, to celebrate the 100th anniversary of the building, a series of vignettes were produced to record the significance of the Armoury to the community of Summerside. Also in that year a didactic panel reviewing the history of the century-old heritage structure was installed on the lawn.