Mortar vs Rifle are dead even on damage versus a 4+ save. Rifle is better against 2+/3+ saves, and mortar is better against 5+/6+.
Rifle has better range. But if you're shooting something within 3", then the mortar wins. If you're garrisoned, the mortar loses the +1 to hit, so the Rifle wins.
So I guess it depends on what you're planning to do. If you're going to be on foot the whole game, a case can be made for the mortar, but I think it's still a toss-up. If you're going to be garrisoned the whole game, the rifle is better, but maybe still take one mortar just in case you get out.
If you're planning to start garrisoned and then get out, you want at least one mortar.
IMO, it's close enough that you probably can't go wrong. But the rifle looks like it's better in more situations due to the extra range, so I think I lean towards it.