Moss grows on a stone wall because the surface stays damp and shaded long enough for spores to take hold, which usually means the wall gets limited direct sun, sits near trees or gutters that drip on it, or has a rough, porous surface that holds moisture longer than a smooth one would. It's mostly a sign of that section of wall's microclimate rather than a defect, but heavy, longstanding moss can trap moisture against the stone and mortar underneath.
Moss needs three things to establish itself: consistent moisture, shade, and a surface it can anchor to. Natural stone, especially textured stone like fieldstone or rubble stone, gives moss a lot more to grip than a smooth surface would, and any wall facing north, sitting under a tree canopy, or catching runoff from a gutter is going to stay damp longer between rains than a wall in full sun and open air. Two walls built the same day out of the same stone can look completely different a few years later just based on which way they face.
Moss on its own isn't destroying the stone, but it does hold water against the surface for longer stretches, and in a climate that goes through hard freeze-thaw cycles all winter, that extra trapped moisture can contribute to joint erosion and mortar breakdown over repeated seasons, particularly in mortared stonework rather than dry-laid. A dry-laid wall with no mortar to protect is generally more forgiving of moss than a mortared wall depending on it to keep water out.
When we're checking a mossy wall, we're really checking what's underneath it: the condition of the mortar joints, whether water is pooling at the base, and whether the shade and drainage situation is likely to keep feeding heavy moss growth. If the mortar's sound, it's mostly a cleaning job. If it's soft or crumbling under the moss, that gets addressed, and in some cases trimming back nearby vegetation or redirecting a downspout does more long-term good than any cleaning would.
More on this from Chelo's Brick & Stone.
Chelo's Brick & Stone serves Hillsville, Galax, Wytheville, Fancy Gap, and Independence, VA. Call for a straight answer and a free on-site estimate.