Not a problem. The area surrounding the station is a slightly-decaying center city area, but is reasonably safe for visitors. As for me, I generally carry wherever I go, and remain aware of who and what is around me. But, apart from the trackside viewing area and overpass, plus the nearby museum across the tracks, that's about it for "downtown" Altoona. It ain't what it used to be.
Along the Station Square side, with its sidewalk viewing area, is a great place to watch trains as they roll through.
Me? I'd recommend the trackside park in Cresson, with its viewing platform and loudspeaker repeating the hotbox/dragging equipment broadcasts as the trains approach, plus you can watch the eastward signals change at the interlocking plant. Down where the tracks go over Old Route 22 is an ice cream stand made from a former New York Central Railroad steel-underframed wood caboose (yes, I know, it's Pennsylvania Railroad territory).
Ex-NYNH&H SS Opr