Not sure if you're thinking the books or the movies, but in either case, at one point or another, the sword gets repaired (reforged, no longer broken). Tolkien didn't emphasize the repairing a whole lot in the books, which may be where your confusion comes from. Aragorn (as Strider) does say something about the sword not being especially useful while it's broken.
As others have pointed out, even a broken sword can still cut, even if it doesn't have nearly as long of a reach.
The whole broken sword thing comes out of Arthurian and Norse legend. Arthur's original sword(pulled from the stone) breaks in a fight against Arthur's future ally King Pellinore, and he afterwards receives a new sword from the Lady of the Lake. In Norse myth, the hero Sigurd reforges his father's sword, which was broken by the god Othin.
In both cases, of course, as in Lord of the Rings, the sword grants remarkable combat abilities to a very specific bearer (Arthur, Sigurd, and Strider/Aragorn).