Found this site stating that there was a plan in the 1870's to extend to Ridgefield: http://www.imediaet.com/ncrr/fs/home.htm
A company called the Stamford & Danbury Railroad is mentioned as a line item in the 1922 edition of the General Statutes of the State of Connecticut; I suppose if one were interested the original charter must be on file up in Hartford somewhere. The proposed line is also shown on a 1935 map of "Abortive Railroads in Connecticut" that's reproduced on page 304 of my copy of "Connecticut Railroads: An Illustrated History."
Seriously doubt though that even a shovelful of earth ever got moved for this project. I've certainly never seen any trace of an old ROW thereabouts; unlike with the New York, Housatonic & Northern and the Ridgefield & New York, both of which left cuts and fills that are visible today, particularly when the leaves are off the trees.