You are quite correct saying that the Rogers plant was the site of other kinds of manufactureing. Originally, silk weaving looms were built there.
In fact they were called Jackard looms. I'm not sure I spelled that right.
Paterson, before becomeing famous for locomotive production, had already built a name for itself in the silk weaving and dyeing industry.
The Rogers plant is still there on the corner of Market and Spruce Streets. Where the "General" was built. Slightly south on Spruce Street, and on the other side of the street from Rogers, stood the Danforth Co., where the "Texas" was built.
Also located in the same general area was the Grant Co., New Jersey
Machine and Foundry, and Paterson Machine Works. All involved in the manufacture of railroad locomotives.
There were no good connections in that area of Paterson to any of the railroads. Most, if not all locomotives built in Paterson, had to be transported through the streets of downtown Paterson to the Erie Railroad, which was at ground level in those days. But, six foot gauge.
The reason the major machine shops were locoated there was because of the use of water power from the Great Passaic River Falls.
Sorry for the long story. Just figured this might be of some interest.
BTW, I grew up in Paterson, NJ, as did my father and my Grandfather.
One half city block from the River Street station on the Erie main.