It sounds like the previous owner may have been trying to depress the diff lock while the tractor is moving. You must be stopped with the clutch in before pressing the pedal. If you have a hydro tranny, make sure it's in neutral. You also have to keep your foot on the pedal to keep it engaged. When you release the diff pedal, it disengages.
There's a couple things it can be.
First is linkage review and adjustment. It's a 2-person job. One to depress the pedal, the other to follow the linkage.
There's the actual diff lock mechanism. If the previous owner was engaging the diff lock when the tractor was moving, it may have rounded off one half of that mechanism. Anticipating drivers goofing up and engaging it inappropriately, one half of that gear is soft steel and it absorbs the wear. The other half is hard and is the actual gear to engage. That soft steel part is kind of a throw-away and as I understand is cheap and easy to replace. Since you have the schematic or if you have a dealer around, you should be able to locate that part and get an idea of how to replace.