Objective assessment of motor function is essential in neurological rehabilitation to determine the effectiveness of treatments, being one of the most commonly used clinical tests the Fugl-Meyer Assessment (FMA). This multi-item and performance-based test evaluate different functional aspects including balance and motor functioning. This paper presents a virtual version of the FMA that combine the classical FMA mechanics with a play-centric approach to accomplish a more autonomous test for assessing motor function, making it easier to administer. Our system aims to provide automatically the FMA score for six items focused on the upper extremity (UE) and one item of the balance section. For that purpose, a depth camera was used to track the user’s movements and reproduce them in a virtual environment. Interactions within the virtual scenario are designed in order to quantify the performance level using two approaches: virtualization and gamification. The first approach was implemented for evaluating the UE functioning while the second approach was used in the postural stability assessment. Thus, not only the traditional FMA score is given, but a further descriptive indicator can be obtained, providing additional information about the patient’s performance. The proposed system aims to be a high-resolution, autonomous, and objective tool for motor function assessment in neurorehabilitation.