This paper presents the implementation of a modular framework that allows the integration of different peripheral components towards facilitating the control of an assistive robotic arm. For that purpose, several components, such as proximity sensors, an RGB camera and a joystick, were integrated into the same control paradigm of the robotic arm. Firstly, the proximity sensors are arranged on the robot casing to sense the nearby workspace. This configuration is helpful to detect obstacles in robot trajectories and ensure the user’s safety. Secondly, an RGB camera was fixed in the robot end-effector to capture visible information in order to assist the object’s grasping via a visual servoing (VS) strategy. Finally, a multimodal approach was considered to increase the input options for moving the robot by off-the-shelf joysticks. All of the above components were integrated into the control strategy through a software-based modular architecture. The proposed framework aims to provide an intuitive, reliable, and modular solution to facilitate the control of a robotic manipulator in task performing, and extendable to people with disabilities.