Was Browns yard used to interchange with the Raritan River Railroad? I know that South Amboy was the huge interchange point for dropping off the coal cars for transfer to the barges as well as for the power plant there. Is it that the PRR kept other freight operations out of South Amboy to keep things clear for the many coal trains dropping off?
If so, that would have rooted everything over at Browns Yard when they discontinued the use at South Amboy. The freight down the C&A was not that significant that they needed all that space in South Amboy plus there was the whole property tax issue for the large South Amboy Yard itself. Why keep all that property and get taxed for it if you don't have enough other freight to make it worthwhile?
South Amboy used to also have an engine facility where they exchanged the electrics coming down from the north for the diesel run down south. When did they first string the electric to Matawan? That would have ended the engine transfer in South Amboy.
Basically, Browns Yard was better sized to handle the amount of freight heading south from there and cheaper too (as far as taxes go).
Just my supposition.