I'm a little out of my teens now (turning 22 next month), but I've had an on-again, off-again interest in trains through most of my life.
I don't have any memory of it, but I was born in Edison, NJ, and the house we lived in (for about a year and a half) had a backyard that was near the NEC tracks. I'm told I was fascinated by the trains that would go by (and how I wish I were still there, just for the view).
At some point, my dad and I got a little bit of HO scale model rail stuff. I don't remember ever having seen it set up, but from time to time, I would remember that we had it and want to set it up (but withot anywhere to do it; I actually just found the stuff up in our attic, and I intend to bring it with me whenever I move out on my own).
Another thing that probably helped was that I have an uncle who, for the longest time, didn't have a car, and would always take the train from New York whenever he came to visit, and I would usually ride along with my dad to pick him up from the train station at Trenton (which has a nice view of the tracks from the station building).
Reading through the other posts here, my experience seems a little different, in that my family was never really a "railroad family;" other than the short trips to Trenton to pick up my uncle, and a rare family trip into Manhattan on the train, the only other encounter with trains I can remember involved my neighbor (who used to watch over me, my brothers, and his niece, who would spend a lot of time next door to us). He used to take us out for walks, and one day we happened upon some active tracks (that I assume was the Conrail Bordentown Secondary, since it's the only thing I can find on a map that would make sense), and one day, as we were walking along the tracks, a freight passed by.
The only other distinct rail memory I have (up until recently) was when my dad and I were taking the train into New York to see a tennis championship. It was the day after the Amtrak derailment at Portal Draw, and it was announced on our train that we would have to get off at Newark and take the PATH to the city. As it turned out, our train was the first allowed between Newark and New York, so I remember seeing the cleanup work out the side window.
Then I went off to college in Hoboken, and recently (within the last year or so), I've taken a much stronger interest in trains (though I've ridden them a lot, between school and home, and between school and work for a summer). I regret that I missed the last "Try Transit" festival in Hoboken, but I've been out lately taking pictures and notes of what I see around the Hoboken terminal/yard areas.
Overheard in NY Penn: "All aboard! Get on the train if you're coming with us!"