The short answer is, effect of temperature reduces the yield strength of metal.
Here I have explained the effect of temperature on yield strength using simple diagrams.
In the above image you can see the initial structure of an undeformed metal piece.
Now this image shows the structure after the metal piece has been compressed along its height.
Because of the applied force, the grains get elongated in the direction normal to the direction of applied force.
Due to this elongation, the strain hardening of the material takes place. And so its yield strength and hardness increases while its ductility decreases.
Also the strain hardening takes place because of increase in dislocation density due to cold deformation.
Now when the temperature increases, the movement of dislocations becomes easier and they readjust due to the stresses locked in a lattice.
The dislocations which are having opposite signs may beat each other.
This process is called the recovery process in which the residual stresses are reduced.
Due to the effect of temperature and force, the atoms reach a certain higher energy level and new crystals are formed. This process is known as recrystallization. (See above image).
The temperature at which the old grain structure is destroyed and new crystals are formed by mechanical working within specified time is called recrystallization temperature.
Now again if the metal is heated to certain higher temperature than this recrystallization temperature, the formation of new grains takes place at the boundary of old grains.
If temperature is maintained for some time during this process, then the new grains will grow to cover the entire structure, and this process is called primary recrystallization.
So the effect of temperature (i.e recrystallization temperature) removes the strain hardening effect and hence it reduces the yield strength of metal, but it increases the ductility of metal.
However, the process does not end here.
Some grains start growing over the other grains till the complete structure is covered by the bigger grains. This process is called secondary recrystallization or grain growth.
I hope you have understood how temperature affect yield strength of metals.