The interior of Huron County was still a wilderness when settlements along the shoreline of Lake Huron and Saginaw Bay were growing rapidly.

Vast stands of inland forests were harvested and the logs were floated to the sawmills at the mouths of the county's rivers to be cut into lumber. Small towns began to emerge in cleared areas where the dense forests once stood. Pigeon was one of the last of these towns to be founded.

In 1883 the Pontiac, Oxford and Port Austin railroad line was extended to Caseville and a depot was built at Berne. Around 1886 the Saginaw, Tuscola, and Huron Railroad built tracks through the tamarack swamp and crossed the north-south line one mile south of Berne; commerce swiftly moved to take advantage of the intersecting rail lines. Originally called Berne Junction, the new community quickly established its own identity and early settlers selected the name of Pigeon, which was taken from the name of the nearby Pigeon River. The river was named for the huge flocks of passenger pigeons that fed from the marshy swamps along its banks. Observers said the flocks were so thick that, when in flight, they darkened the sky. By 1914 the pigeons were extinct.

After incorporation as the Village of Pigeon in October, 1902, eight citizens were elected in December of 1902 as the first village council: A.G. Kaumeyer, J.W. Liepprandt, Louis Staubus, Joseph Schluchter, Albert Hartley, H.H. Gould, Harry Hershbert, and E.W.E. Bundscho. Charles Sting was treasurer and Warren Challis the assessor. There were 107 registered voters in the village and 102 of them cast ballots on election day.

The railroad depot that now serves as the museum was constructed in 1908 and served two railroad lines: The Chesapeake and Ohio (C & O) and the Grand Trunk. The C & O was purchased by Huron and Eastern which currently operates the railroad. The Grand Trunk line has been abandoned.