When traveling in a southerly direction, you can be said to be going down, right?
That's certainly the way it looks if you stare at a map.
But in Spokane, heading south typically means going up, at least in an elevation sense.
So how do you get around this directional conundrum?
One answer is to simply avoid the whole "I'm going up to..."/"I'm going down to..." construction.
"Over to" works pretty well in almost any situation.
Or you can always just point and grunt.