The practical maximum for hand spanning is about 200lb and this was probably true for tough, hardened medieval warriors as well. (Of course there will be some tough buggers who could span more, but on average this would probably be about the top end.) I know a bowyer - and a very good bowyer at that who has made a yew prod of about 500lb. He tillered it using a car jack, but to my knowledge he hasn't made it into a crossbow yet. I would have thought that if you tried to increase the power of a wooden prod too much, you would just increase the mass and lose too much draw length and you would not get anything much back for it in terms of power and velocity. There would also be the maximum speed that the material could recoil which could not be exceeded by increasing draw weight beyond a certain level.
I would be interested to hear about experiences anyone might have on this. What is the most powerful crossbow known to have been made from wood? Is there any evidence of mechanically spanned wooden crossbows from the past?