Yeah, and that's what it was, too... a little over half an hour. Argh!
Now I have to post the fast version, as I'm absolutely exhausted and exasperated for reasons totally not pertaining to the board.
OK, here's the deal: the angle cut that pulls the cam forward against the thrust plate works only when the cam is powering the oil pump--at cruise or acceleration. Under
deceleration, that same gearset works to pull the cam
backwards against the rear core plug, which in your case apparently has a funny accent since you say it's Welsh.

The situation is worse under compression braking, since the crank is really dragging on the camshaft but the load (per RPM) on the oil pump does not change, so that backwards pull is more significant. It would certainly push the core plug out, though.
If you don't see where I'm getting this, I can tell you how to demonstrate it rather easily and quickly with the engine torn that far down.
Long and short of it: Your cam and lifters are junk, just as is your timing set. Other areas I'd check:
- Crank snout where the timing gear resides: If the timing gear's not a snug fit (I mean any wobble at all), the crank is junk. There is not a timing set on the market that corrects for lateral movement of the cam, so that rearward tug could have damaged the crank. This means a new crank, since yours is iron.
- #3 main bearing, on the thrust surfaces: See above... if the cam's been out for a walk, it's dragging the crank with it. Look for abnormal wear on the forward face.
- Lifter bores: Again, if the cam was moving back and forth, the odds are good that the lifters were being dragged around against their will. If they're ovalled your only choice is to either spend a pile having them bushed, or find another block. A lifter in an ovalled bore will not spin, and will wipe the cam in short order. Check to see if the lifters will rock in their bores.
- Pushrod cups/rocker arm cups: Perfectly shiny and hemispherical. If they're not, that would indicate a wandering lifter or lifters. Side loads on the rockers would also be evidenced by uneven valve-tip wear or uneven marking of the rocker shafts. This would explain why the valve seals and guides are junk... but the heads are not likely the problem and new guide liners and seals should make 'em A-OK. However, if they're 587 or later castings, they're not worth the investment.
That's the short version. The long version was so much better, but even this much is beyond the energy I have at the moment... I'm more wiped than your timing set at this point.
I will answer further questions happily.