Although IBDI, a coding tool that increases internal bit depth to improve compression performance for high quality video, is able to significantly improve the coding efficiency, the internal memory increment problem occurs because of the necessity of storing reference frames. Therefore, memory compression algorithm is proposed to solve the internal memory increment problem while maintaining the coding performance of IBDI. The memory compression methods have successively reduced the reference frame memory while preserving the coding efficiency by dividing a reference frame into the fixed size processing units and using additional information of each unit. However, additional information of each processing unit also has to be stored in internal frame memory; the amount of additional information could be a limitation of the effectiveness of memory compression scheme. Therefore, to relax this limitation of previous method, we propose a selective merging-based reference frame memory compression algorithm, dramatically reducing the amount of additional information. Simulation results show that the proposed algorithm provides much smaller overhead than that of the previous algorithm while keeping the coding efficiency of IBDI.