At one time, New York Central used to originate a mail train (with E units) from the Post Office on the west side of Manhattan. A friend of mine rode it (I think early in the Penn Central era), but it disappeared shortly thereafter when the Post Office took mail off trains.
Other than that, there was local freight service only. Photos I have seen show a single hood unit (usually a GP9 or something similar) peddling a few cars. Freight traffic gradually dwindled to nothing as condos and apartments replaced the warehouses and factories in the Meatpacking District and farther south. St. John's freight terminal in Greenwich Village was demolished in 1968 or shortly thereafter, and several blocks of the lower end of the "High Line" as well.
When I was working in New York in the early 1980s, I and a friend walked part of the West Side Line north of 42nd Street, where it is briefly at grade and then in an open cut for a few blocks. It was clearly out of service even then (1983).
As a child I recall driving into New York City from Westchester with my family, and looking at the West Side Line from the West Side Highway. This would have been the early 1960s, and the long ties that had formerly supported third rail were still clearly visible (although the third rail had been gone for a decade or so at that point). I later saw the "R" moters that had been bought by NYC to work the West Side Line (among other places) working on local freight on the Chicago, South Shore, and South Bend. I believe the third rail went only to 34th Street, however. The lower end of the line was always worked with steam or diesel.
Randy Resor, aka "NellieBly" passed away on November 1, 2013. We honor his memory and his devotion to railroading at railroad.net.