From the 1940s to the 1960s, the prevailing thinking was that rapid transit extensions out to Route 128 would replace all but perhaps a few commuter rail corridors. Until midcentury, that tended to be the area of highest commuter ridership, so the expectation was that the extensions would handle most ridership and the terminal park-and-rides would handle the rest. The Oak Grove and Quincy Center extensions were designed during this time - it's why they reduced their mainlines to a single freight track. However, postwar suburban sprawl enabled by the highways plus rapid transit extensions being more difficult/expensive than expected meant this wasn't really workable.
There was a brief period around the early 1970s where commuter rail ridership was bottoming out, Amtrak's survival was uncertain, and the urban core was rather hollowed out - but when urban-core highways had been proven destructive and inefficient. The terminals in particular were a huge cost for the commuter rail system. The most obvious rapid transit extensions (Southwest Corridor, Alewife, Braintree) went forward with redirected highway funds, but no one really knew what to do with the suburbs. That's how you got proposals like
the 1972 study to extend rapid transit over existing commuter rail tracks - trying to serve the outer suburbs as cheaply as possible.
From the mid 1970s on, there was an oil crisis, the urban core has recovered, and long suburban car commutes have sucked. There's been a general consensus that commuter rail on the eight major mainlines (Fitchburg, Lowell, Western, Eastern, B&A, Midland, Shore Line, Old Colony) is needed. With few exceptions, service levels and ridership have consistently increased since then (often by a factor of 3 or more), and only a handful of marginal stations have closed.
The current regional rail/electrification movement is a natural extension of this progression. There's a need for greater regional mobility, both suburban and urban, and an increasing recognition that even electric and self-driving cars don't fix the fundamental geometry problems of roads. Electrification makes for quieter, cleaner, faster service that can handle infill stations, and it enables higher-frequency service that has a realistic ability to make transit viable across a wider area than rapid transit can reach.
I don't see deconversion of the Braintree Branch to regional rail as useful, though. It's a corridor that benefits from having rapid transit - high frequency, one-seat ride to the Kendall job center, and easy transfers downtown. While regional rail will be a major improvement and worthwhile investment for the suburbs, it won't be able to match the service quality that a fully enhanced Red Line can achieve on that corridor. (Among other things, there's no one-seat ride to Kendall, and downtown transfers will be less convenient because the NSRL will be deep.) Having an additional 3+ local stops added to all Old Colony trains will also make them slower than they could be with just stops at Braintree, QC, and JFK.
The single-track pinches on the Old Colony mainline are difficult but not infeasible to fix, and they can be done incrementally rather than requiring a single megaproject. First step is to cut-and-cover a second track at Quincy Center to enable meets there. JFK-Savin Hill can be fixed by consolidating the Red Line tracks, Wollaston by narrowing the overbuilt Newport Avenue (or cut-and-cover for the second CR track), etc.
