Automatic Paintball TurretSep 2017
A paintball turret was developed for the MECH 4806 course at Carleton University in the winter 2014 semester. The system is intended to autonomously track, target, and engage an object
Once a target lock has been acquired, a paintball gun fires at the target with good accuracy and precision.
A preliminary design of the system is overviewed along with a description of the component and materials selection. The results from the live firing showed successful tracking and targeting of objects and enemies with reasonably accurate and precise paintball shots.
A paintball gun consists of 4 main components. A hopper stores paintballs, which are used as ammunition. The air tank provides pressure which is converted into kinetic energy to accelerate the paintballs out the barrel. The barrel launches the paintballs on a predictable trajectory. The body holds the components together and provides a trigger used to fire paintballs. Instead of a person holding the marker a turret was constructed, this allows for the turret to be position mechanically. An overview of the system and design goals is provided in Section 2 with a description of the system requirements in Section 3. In Section 4 the preliminary design criteria and constraints are discussed, along with the material and electronic component selection, mechanical build, control systems, and software choices. Finally, Section 5 presents the results from live tests with recommendations for improvements and conclusions made in Section 6.
The goal of the system is to develop a paintball sentry gun that can acquire, track and engage moving targets autonomously. Using a video image coordinates of the target’s position are determined and sent to turret motors which then orient the paintball gun to aim at the target and fire paintballs. The video image is acquired by a stationary webcam and transmitted to a laptop where the image is processed to acquire the desired target’s position. The position of the object relative to the paintball marker is then found and a desired motor position is then sent to an Arduino for further manipulation. The Arduino then computes the change between the current and desired position to calculate the time the motor is given to move to the final position. The Arduino relays the final position and time allowed to the motors’ onboard controller that handles the trajectory planning and internal PID controller. The servo motors frequently check position and correct following their PID controller. The Arduino validates if the marker is in position and ready to fire paintballs.
The following is a detailed description of the various systems that are part of the paintball sentry system. Section 3.1 describes the physical system including the hardware and supports of the sentry. Section 3.2 speaks on the image processing is discussed this deals with how the intended target is found and its position is extracted. Section 3.3 describes the motor control which deals with the positioning and trajectory planning for the motors. Section 3.4 specifically discusses how the onboard PID motor controllers were used to optimize motor control. And finally, Section 3.5 discusses the logic, circuitry and safety mechanisms behind the firing control system.
In order for the system to orient itself to aim at targets within it’s range, two rotational degrees of freedom were required. The first degree of freedom, a rotation about the vertical axis, will be referred to as pan. This aims the barrel of the paintball gun left or right. The second degree of freedom, a rotation about the horizontal axis, will be referred to as tilt. This aims the barrel of the gun up or down. To achieve these degrees of freedom, two motors were required to power the system. To avoid gear alignment and the need for optical encoders, direct drive servo motors were desirable.
For a smooth pan rotation, it was desirable to remove any axial loads acting on the pan motor. For this reason, the weight of the assembly was supported on a large lazy susan bearing oriented for pan rotation. This lazy susan was rated well over the required weight and the square shape made for very easy mounting. The motor was mounted underneath in a non-load- bearing method.
For a smooth tilt rotation, two lubricated bearings were selected to bear any radial loads due to weight, inertia, and recoil that would otherwise transfer directly to the motor. These bearings were sealed to prevent the possible intrusion of paint or dirt and flanged to make for easier mounting. An affordable half inch rod was selected to support all loads. To mount the motor to the shaft, horns were employed to simplify construction and minimize cost of the prototype.
The housing was made out of ½” plywood. This selection made manufacturing simple, eliminated the need for a machine shop, and kept the housing relatively light. Before mounting the horns to the housing, the housing was suspended and with the use of a plumb the center of gravity was found to make the movement as smooth as possible. The entire assembly was mounted on a tripod constructed of ¾” schedule 40 steel piping. This allowed for flexibility in design with readily available materials. The weight of the tripod also provided for a low and stable centre of gravity. If additional mass was required the piping could be filled with sand.
3.2 Image Acquisition and Target Tracking Target tracking was done by taking visual information from a webcam, processing this visual information and sending a message to the Arduino for the pan and tilt of the servos. The following paragraphs will break down the system that connects these elements. Using the python programming language and opencv, a set of various computer vision functions. The program is set up into 2 parts: initialization; and a loop that runs continuously during operation of the turret.
The initialization section starts by importing libraries needed including opencv and a serial library used to talk with the Arduino microcontroller. The full list of imported libraries can be seen in the python in the appendices. To finish up a video capture object, video writer object and a serial communication object were created. These objects get released when program is done running.
In the initialization, simple preferences switches were added for testing. One preference includes connection to the arduino; testing can be done without an arduino. Another preference shows on the screen what image is going through the webcam; this also gives more information for debugging purposes. A target lead switch was added which will be discussed later. The final switch determines whether the webcam footage is to be recorded. The loop runs after initialization while the system is operating.
The process of the loop is to take an image, find the middle of the largest element of a colour range specified and perform coordinate transformations to find the position of the target relative to the gun and return the correct pan and tilt to aim at the target.
The first step is to retrieve a frame from the camera. The camera being used is the Logitech C270 HD Webcam, with a frame displaying 640 by 480 pixels. This gives a total pixel count of 640*480 = 307200 pixels. Each pixel contains 3 numbers to define its state. The frame uses RGB(red, green and blue), each colour is a number between 0 and 255 which fits in one byte. The frame is written to a file and blurred slightly to reduce noise and make contours of greater uniformity.
When finding the colour range it is easier to define in another pixel state by using HSV (Hue, Saturation and Value). The image is then converted to HSV and run through a filter that takes only numbers in between two HSV values which are defined by the operator. The result of this process is a 640 by 480 image but instead of colours, pixels are defined as boolean.
The contours of the boolean image are outlined using a function in opencv called the findContours(). This creates a list of contours which is used in the next step to find the contours with the largest image. Within the contours, area is found using another built-in opencv function called contoursArea(). The largest contour is sent through a moments() function and the center of the contour is found.
Finding the pixel which we want to target is now possible, but we need a pan and tilt relative to the gun. To get from this pixel state involves finding the spherical coordinates with the camera as the origin, transforming it into a cartesian coordinate system, shifting the origin to the gun and then converting this back into a spherical coordinate system relative to the gun. From this we can get the pan and the tilt of the gun.
Figure 3: Aspect ratio illustration
The first step was to transform the pixels to angles. The current frame mentioned above is 640 by 480 px with an aspect ratio, illustrated in Figure 3, of 4:3. On the webcam’s spec sheet gives a Field of View(FOV) of 60 deg. Knowing these two, the equivalent FOV for the x and the y direction can be found using pythagorean theorem assuming that pixel distance is directly proportional to angular position of the image. From this it was assumed that the centre of the screen screen was the zero theta and zero phi with positive theta up and positive phi right. Theta covers angles in the horizontal direction and phi in the vertical. This is demonstrated in Figure 4.
Figure 4: The spherical coordinate system
With the theta and phi, but from a single point, r, the magnitude is unknown. To find this value either an estimated constant is added or stadiametric ranging can be used. Stadiametric ranging is where the height of the target is known, and assuming that the angle between the top and bottom is close enough to a right angle can find the distance away using simple trigonometry.
The origin needs to be at the turret to find the pan and the tilt needed. The spherical coordinates are transformed into cartesian coordinates then with addition of the difference in position between the camera and the gun. The gun can then be place at the origin on the cartesian coordinates. This is simply transformed back into spherical coordinates and the pan and tilt are given by theta and phi.
The last thing to do before sending the pan and the tilt is get something the motor will understand. So a conversion script was created that took the theta and phi and converted it to the appropriate pan and tilt
Built in the coordinate transformation function is a target prediction that can be used to estimate where the target will be in a defined period of time. It is using simple linear extrapolation of the position based the targets current velocity, illustrated in Figure 5.
Figure 5: Linear extrapolation of a velocity vector
The velocity is estimated by comparing the current and previous frame and observing how far the centres of the object have moved. Using this distance and the time elapsed, the velocity of the object can be found. To predict the next position the current position is taken and added to the velocity times a defined period of ‘leading’ time which represents how far ahead we would like to predict. Longer periods of leading time can be used to track fast moving or long-distance targets. However, accuracy will likely decrease with longer leading times.
To help with debugging a screen output was used to display the current frame. On the frame the contours were drawn and the point which is targeted. Additional information was also used including loop time, size of largest contour and pan and tilt sent to the motors.
The motor control system receives its input from either the video processing system or the joystick. An Arduino microcontroller receives serial position information over a USB link from PC processing the video. This takes the form of the ASCII character p for pan or t for tilt, followed by a step number between 0 and 1024 to signify the position between 0 and 320 degrees, finalized by a newline character ‘\n’. An example is p200/n, telling the pan motor to reach 62.5 degrees. The microcontroller ensures the position information is valid then calculates the time the motor has to move to the new position. This was calibrated as 2 milli seconds per change in step. Continuing the example, if pan started at position 300, it would have 200 milliseconds to move to position 200. The Herkulex DRS-0101 motors contain microcontrollers which plot a trapezoidal path between their current and final position. This reduces their peak power draw by ramping up, leveling off, then ramping down near the final position. The internal PID control is further explained in section 3.5. The DRS-0101 communicates over serial communication channel. The original Arduino library written by Alessandro Giacomel from Italy. Support was added for Arduino Due through additional control statements for the different build sections. The Arduino code written for this project will now build for Due, Uno, and all previous Arduinos which support either hardware UART Serial or SoftwareSerial.h.
Motor PID Controller Tuning
The HerculeX DRS-0101 comes with a built in PID controller to improve motor control performance. Tuning this controller allowed for better response time, a reduction in overshoot and oscillation, and a minimized steady state error. A PID controller receives an error signal, in this case the difference between the current motor position and the desired motor position, and uses this error to send an ideal output to the motor.
The equation defining the output of a PID controller can be defined as Where e(t) is the error signal, K p is the proportional constant, K i is the integral constant, K d is the derivative constant, and u(t) is the output signal to the motor. The proportional constant, K p , affects the output to the motor based on the current error signal. Increasing this constant will increase the response time of the system but possibly result in overshoot and oscillations.
The integral constant, K i , affects the output based on the current error signal and the duration of error. Meaning that it affects the output based on past errors. Increasing this constant helps to eliminate offset in the steady state error but can lead to a lag in the response. The derivative constant, K d , controls the output based on the slope of the error. In other words, it affects the output to the motor based on future predicted errors. Increasing this constant results in a faster response and improves settling time but is sensitive to high frequencies in the system, for this reason, a low pass filter is required. In this case, the inertia of the system acts as a low pass filter.
As mentioned, the HerculeX motors used in the system have built in PID controllers. The Servo Manager Kit software allows for the adjustment of position K p , position K i , and position K d constants of the motors. Originally, it was planned to employ the Ziegler-Nichols rules for tuning the PID controllers. These are experimental methods for determining PID constants by observing the step response of the system with both integral and derivative constants set to zero . Before applying the experiment however, the step responses of each motor with the current PID settings were observed and plotted as shown in Figures 6 through 7.
Figure 6: Step response of tilt motor when tilting up (top) and down (bottom)
Figure 7: Step response of system for pan directions right (top) and left (bottom)
Note that on all vertical axes, the position is in the step units of the motor. After observing the system’s response, it seemed satisfactory for our purposes. The system was capable of settling within one second in all cases with very minimal overshoot. For this reason, additional tuning was not done.
A key design choice made early in the project was selecting a paintball gun with an electronic trigger. This allowed the project to electronically trip the trigger instead of building an additional mechanical system to depress the trigger. Below in Figure 8 is a circuit diagram of the microcontroller and circuit within the Smart Parts Ion PaintBall Gun. Highlighted with the arrow is the mechanical micro switch trigger mechanism. When the switch is depressed pin 15 of the ATMega48v senses connection to ground from the NC connection through the switch. The spring loaded microswitch returns to open when not depressed. The microcontroller will only fire on the falling edge, leaving the trigger depressed will not allow the gun to fire until the microcontroller senses pin high again.
Figure 8: Paintball Gun Circuit Diagram
The Arduino Uno used in the final design operates at 5v, while the paintball gun at 3.3v. To compensate for this difference, a voltage divider (as seen in Figure 9) was designed. This was an effective design as power draw by the paintball gun was nonexistent and the Arduino was wall powered, thus losses through the resistors were deemed acceptable.
Figure 9: Arduino Electronic Circuit
Recommendations and Lessons Learned
During field testing, the compressed air line and paintballs were added to the paintball sentry. As this additional mass was not accounted for during initial testing and motor tuning, the tilt servo motor was overloaded causing it to stall and eventually caused internal gear damage. Future designs should account for masses added to the gun during use. Additionally, alternate housing designs should be considered to reduce weight.
Future projects should consider using the HerkuleX motor line as their serial communication and internal PID control made for excellent prototyping.
A functional paintball sentry turret was successfully constructed. It successfully tracked targets of a designated color, transformed coordinates of the targets into motor positions, engaged the motors to reorient to aim at the target, and fired paintballs precisely and with reasonable accuracy. With the success a few lessons were learned, the largest being the importance of sizing an adequate motor and balancing the load. A video demonstration of the system can be found on Youtube