Building shows in 1951 aerials and in that one a box car appears to be on a siding that runs next to the building furthest east. The west end of that was added on later filling most of the gap between it and this building.
Topographic maps from 1952, 1960, 1970, 74 and 83 show a siding extending up to the road and next to the front of this building.
At some point between 1971 and 1994 the current bridge replaced a grade crossing for NY-259.
Were I to guess it probably was a freight station.