Well, first take a look here:
http://safetydata.fra.dot.gov/OfficeofS ... nyr1a.aspx
The Railroad Accident/Incident Reporting System (RAIRS) is available on FRA's Web site, and anybody can run reports. The link above will show the most recent ten years. Reportable "accidents/incidents" (see site for definitions) declined 22% over the period. So the trend is in the right direction, and has been for most of the 31 years I've been in the industry. It's a lot safer than it was in 1977, and it was actually pretty safe then.
So that's one answer.
As far as worker fatalities are concerned, railroading is about in the middle of any list of industries. Fishing and lumbering are at the top of the list, office workers at the bottom. So I answer the question, "Are railroads safe?" with, "Safer than what?"
Rail/highway crossing accidents have been on a long decline, but "trespasser" fatalities have been rising in the past few years. Nobody's sure what to do about it, because the definition of trespasser is someone who has no reason to be there. Will fences keep them out? Probably not.
A friend of mine once worked as a trainmaster at Oak Point Yard in the Bronx. He had a contractor build a thousand foot long fence at the edge of the yard to prevent looters from breaking into and stealing from freight cars. The contractor finished the job on a Friday. On Monday morning someone had cut down the fence posts and removed the entire fence without a trace. Welcome to New York!
Roads are mostly unfenced, and we deal with it by ensuring that our children don't play in traffic. I do believe that where rail transit and railroads share a right of way (but not track) fences should separate the two systems, simply to prevent maintenance workers or transit riders from inadvertently wandering where they shouldn't be. But the fence won't stop derailed freight cars from fouling the transit ROW. You need intrusion detection for that.
So it's complicated. But I feel confident in saying a couple of things:
1) railroad are the safest way to move land freight
2) railroads in North America are substantially safer (by many indicators) than railroads just about anywhere else.