Brown Hall is a co-ed freshman dormitory located at the corner of Prince George Street and North Boundary Street. The current structure was built in 1930 to replace the original Brown Hall, which is now known as the Prince George House.

Brown Hall was named in honor of Mrs. Jane Brown and Cornelia Brown, members of a prominent Methodist family in Virginia. The new Brown Hall was built in 1930 with funds left by Mrs. Edward Brown of Lynchburg under the supervision of trustees for the Board of Home Missions of the Women's Missionary Society of the Virginia Methodist Conference. The building originally served as housing for Methodist female students at William & Mary but eventually opened up to the entire female student body. The designer was Charles M. Robinson, the university architect, and the contractor was J.W. Davis.

Brown Hall was purchased by William & Mary in 1939. At various points throughout its history, Brown Hall also has been an all-male dormitory, housing for Army Specialized Training cadets, and rented space for service men and families living in Williamsburg. In recent years, archeological digs around Brown Hall have unearthed evidence of a building that may have been the Bray School, a school for African Americans in Williamsburg in the 1700s.