I think it's safe to say that there's still something resembling an embankment or bridge back there, based on this shadow which I've circled (from NY GIS). They have not yet performed LIDAR scanning in that area, otherwise I'm fairly certain the GIS Hillshade view would show some sort of embankment there. The older DEM data is not fine enough to show this and historically trees seem to interfere with basic satellite imagery height mapping, whereas LIDAR, in every instance I've looked to so far, is not as affected by trees and I've managed to find some very obscure ROWs with a high degree of accuracy since I've started using the LIDAR maps.
To expand on TrainDetainer's maps a bit... I think it's safe to say the R&E ROW hugged the north side of that alleyway anyway, given other clues in the area, especially when compared to what little is shown on Sanborn maps, so I think your "slight curve north" happened a little to the west of where TD shows it.
In particular, I believe this area was a bit of a softer curve to the south than TD shows. The house on the corner of Elm and Main is in Sanborn, as are the two houses to the north of the ROW. Distinctly missing from the Sanborn map is that 3rd house which falls squarely in the middle of the red lines I've drawn (attempting to roughly indicate both tracks)
And finally, just to expand a little more on the eastern "exit" of Pittsford, clues would suggest that the R&E ran to the SOUTH of the powerhouse. After looking a bit more at Google Earth after making this image, I would almost guess it turns south in the area between the powerhouse and the shed at the extreme right of the image, but still stayed just to the north of said shed.