The Constrained-Random Verification (CRV) approach is used in generating numerous stimulus which defined by constraints setting to interact with the design under test (DUT). As the designs get more and more complex, the stimulus space become larger. The CRV approach will be harder to generate the specific stimulus to hit the difficult corner cases. A well-known design verification corner case nowadays is to verify functional operations is working or not when the stack is full of specific FIFO. Through the whole article, we take some FIFO full behavior in MMU (memory management unit) as a corner case example to demonstrate how we apply machine learning to approach corner case in CRV. In MMU, the address of input stimuli is one of the dominating factor to effect the whole design behaviors. So, we design a set of constraints to control the input address alteration. The problem is how to determine a sequence of constraints to trigger FIFO-PUSH behavior as possible as it could to fill up the stack. In this paper, we propose a two-stage framework which applied supervised learning model Transformer and Reinforcement Learning methodology. As the results we demonstrated, we can significantly increase the hit rate of the corner case in design verification without human expert guidance, the most improvement of the hit rate even achieve 380 times better than the traditional CRV.