A substantial three-story Greek Revival hotel, there was a two-storied front porch with Doric columns supporting an overhanging roof. Here in 1840, it is recorded, was said the first Roman Catholic mass in either Dorchester or Milton for the large number of Catholic factory and granite industry workers, and was on occasion continued until 1863 when St. Gregory the Great was consecrated on Dorchester Avenue. By 1871, Thayer's Hotel was operated by William B. Brown, an employee of the Thayer Family, who was quite an enterprising innkeeper. It was said in the "Saturday Evening Express," that his neighbors had accused him of snaring pigeons, which led to this ditty:

" There was a hotel man named Brown, 
Whose house was the talk of the town, 
He found be was able 
To set a good table
So long as the pigeons flew down!"

Well, Thayer's Hotel did not share in the same comfortable carriage trade that patronized the Milton Hill House, but it still provided meals, libations and a comfortable stop for travelers.

Seen in the accompanying photograph, kindly loaned by Anne Thompson, is the Tally Ho rented by John Talbot from Henry Crane & Son's "Lower Mills Stables" on "Derby Day" in 1909 stopped in front of the Hotel Milton on Adams Street while being photographed by Frizzell, the village photographer. Mr. Talbot was proprietor of Talbot's Grocery Store at 1157 Washington Street in the Lower Mills, which had traded since 1815 in as much alcohol as it did in groceries, and a real estate agent with offices at 85 Water Street in Boston. An avid horseman who later lived at 21 Canton Avenue (now De Wolfe Realty Company), he often raced his trotting horses in the Dorchester Gentlemen's Driving Club and at the Readville Race Track in Hyde Park, which was laid out on the former Civil War Camp Meigs, named for Brigadier General Montgomery Cunningham Meigs (1816-1892), "soldier, engineer, architect, scientist, patriot." This Tally Ho, amply filled with Mr. Talbot and eight of his friends, was pulled by a "four- in-hand," which was often unmanageable by inexperienced drivers. On Thanksgiving Day in 1908, Dorothy Forbes of Milton Hill recounted that Mr. Talbot's four-in-hand was "going down Milton Hill [where] one horse, much against the instinct of the other three, decided to run away. While still in the minority the Walter Baker factory whistle blew just as they went by, which started the other three horses at full speed. The fact that many of us on horseback, or in horse-drawn vehicles, chased the coach as best we could, kept this fair going the rounds of the town until the four-in-hand, both horses and those a-top the coach, were in a white lather."

Walter Baker & Company, then owned by the Forbes Syndicate and headed by J. Murray Forbes (1845-1937), purchased the somewhat decrepit Hotel Milton in 1906 from A. Lizzie Mann, which had Cohen Brothers' Tailor Shop, Littlefield's Wallpapers & Steam Matting, W.F. Goward Dry Goods Store, Anton Gramer Furniture Store and let rooms above, after which it was demolished. In 1919, the company commissioned Milton architect George F. Shepard, Jr. (1865-1955) a partner of Frederick Baldwin Stearns in the Boston architectural firm of Shepard & Stearns, to erect their new brick and limestone office building, to which their business and mill offices were moved upon its completion in 1920.