Orthogonal imbalance correction of microwave echo signals is essential for achieving high-precision measurements in continuous wave radar systems. However, in close-range measurement scenarios, the echo signals suffer from amplitude-phase imbalance, DC offset, and coupled unknown amplitude modulation. This unknown amplitude modulation distorts the geometric characteristics of the I/Q trajectory, reducing the effectiveness of conventional methods and significantly degrading displacement measurement accuracy. To address this issue, this work proposes an elliptical envelope recovery correction (EERC) based multiparameter simultaneous estimation method for I/Q imbalance calibration, enabling the imbalance parameters to be estimated under unknown amplitude modulation. First, the I/Q signals are divided into multiple groups according to their periodic characteristics. Ellipse parameters are independently estimated for each group, and the initial imbalance parameters are obtained by averaging the results. Subsequently, an amplitude recovery term is introduced to compensate for the influence of unknown amplitude modulation. Furthermore, a joint objective function is constructed from geometric and time-domain perspectives to minimize the circularity and linear offset errors of the amplitude-recovered trajectory, thereby enabling the simultaneous estimation of multiple imbalance parameters. Simulation and experimental results demonstrate that the proposed EERC method effectively overcomes the limitations of existing I/Q imbalance correction techniques under unknown amplitude modulation. In a 20 mm displacement measurement experiment, the proposed method achieved an average displacement error of 2.5 μm and a nonlinearity below 0.05%, reducing the average measurement error by 38.91% compared with existing methods. In two nonlinear displacement measurement experiments, average displacement errors of 1.9 and 2.6 μm are achieved, with repeatability errors below 1.1 μm. These results verify the accuracy and robustness of the proposed method for I/Q imbalance correction and displacement measurement.