As part of our Digital Systems Processing course in the third semester of my graduate study, we developed a Master-Slave Manipulator Mechanism, MSM that can read the gestures in real-time scale. The system comprises of the Raspberry Pi 4 and the IR camera module that directly gets mounted on to the raspberry Pi’s camera connector. The Operating system runs the python script that transcode the gestures into commands for the MSM. The Code was developed in python that utilizes open CV2 based libraries that run on Linux based Operating System. The approach used for the project is close to HSV (Hue, Saturation, Value) segmentation. 

The end system thus realized, was able to actuate a mechanical arm that mirrored the gestures made by the user in front of the IR camera. The user/Master points his/her hand towards the camera, which in turn will process the data. The post-processed data controls the actuator assembly, a robot prosthetic limb having 2 degrees of freedom. The data generated by the real-time gesture recognition algorithm, controls the motor-driver controller that included its direction to turn the hand right and left. The hand also supported a claw that opened and closed to grip any object that fits its size.

Use of Infra-red camera instead of the standard camera helped improve the performance in foreground and background separation with the added benefit of being able to use it in a comparatively dimmer and multi-chromatic environment. Also, led were added for an added visual feedback mechanism. The data is stored as an integer that is then fed to the system as an input for the state machine. The signals thus generated are then sent to the motor controller and actuation functions as an input. These, in turn regulate the amount of power and duration for which the power needs to be sent to each motor. Two independent power systems are used here to tackle the initial power drop of voltage while starting the actuation assembly and causing the Brown-out reset in the circuit. To optimize the control functionality of the GPIO peripherals, NAND GATE was interfaced as an inverted pair logic supply, thereby reducing the number of pins required to control the motor. A snubber circuit, employing 2 reverse biased Schottky Diodes was developed and implemented as freewheeler circuit.