A few good tips have already been mentioned, allow me to fine-tune a few of them for you.
First off, unless the RS scanner you just bought was used RS radios are not automatically garbage just because they are from RS. I'm a scanner buff (as well as a licensed ham radio operator) and own (and owned) numerous RS scanners over the years. They are great radios and I've never had a problem with any of them. Uniden/Bearcat radios are just as good, I own/owned numerous Bearcats as well.
In regard to the squelch setting...the proper squelch setting is when you reduce the squelch back to the point that you hear static. Now advance the squelch forward just to the point where the static stops. Your radio should now be set for optimum reception.
Make sure your antenna is correctly connected to the radio. Most scanners today use a BNC connector. Line up both lugs on the radio's antenna connector with the notches in the antenna's connector, seat the antenna fully on the radio, and then rotate the antenna a quarter turn until it is fully seated on the connector. Your antenna should now be properly connected.
The railroad band is VHF-Hi and is in the FM mode. Most consumer-grade scanners automatically select the correct reception mode as the frequency is entered. VHF is line-of-sight but that shouldn't be that much of a factor unless you are trying to listen to something 50+ miles away. I live in a valley and my handhelds receive transmissions from 30-40 miles away just as well as my base radios do that are coupled to the roof-mount antenna on top of my house.
The best way I've found to test if a scanner is working correctly is to program in (or press the 'WX' button if so equipped) your local National Weather Service frequency in. NWS transmits in a continuous loop, 24/7. If you're not receiving NWS you're either to far out from an NWS station or you have a problem with your radio or your antenna.
Hope these tips help.....good luck!