In textmode, \underset with an operator typesets using subscript

The expression

$\underset{x}{\min}$

should typeset with the x underneath, but it is moved to the subscript position due to the movablelimits setting on the operator. The \underset and \overset macros should remove the movablelimits from the base?