Case Study: Robot for foosball table using XIMEA high speed camera

Foosball: Human versus Robot using high speed XIMEA camera

 
Professors and students at the Swiss Federal Institute of Technology in Lausanne are developing a robotic player installed into a foosball table.
The plan is to make it more accurate, faster, and more strategic than any potential human player.
 
Therefore, the system should be able to find the ball's position on the table, evaluate a strategy of how to score and execute it with the help of precision motors.
The latest performance improvements to the vision and strategy aspects were made possible by the implementation of a high-speed camera from XIMEA.
 
The camera, placed under the table, runs at 500 frames per second (fps) and creates a solid basis for the detection and control of the ball in real time.
At its current stage of development, the foosball table robot is able to beat any regular player it encounters. 

Project details

The project is called Automatic Control Baby-foot and it is one of several at EPFL - École polytechnique fédérale de Lausanne.
 
It provides an opportunity for engineer students interested in robotics and automation to improve their skill set by completing a multidisciplinary project.
Through covering many subjects, this project requires knowledge of mechanics, analogue and digital electronics, programming, and other qualifications.
 
For example, the complete automatisation of the rod rotating arm needs to be designed to implement complex computations during the game.
That affects not only defensive strategies, but also attacking and early evaluation of the counterparty’s moves.
 
The overall goal is to provide the automatic laboratory with a tangible, effective, functioning demonstration system of both software and hardware assemblies.

People of all ages are fascinated by the concept every time it is on display, thanks to the ingenious implementation that brings an old game to the 21st century.
 
At first glance, the foosball table located in the middle of the Automatic Control Laboratory looks perfectly normal, but looks can be deceiving.
One of the levers has a mechanical arm capable of propelling the ball into the opposing goal at a speed of six meters per second.
 
“This is already enough to beat the average player,” said researcher Christophe Salzmann, who heads the project. Yet that is only the beginning. 

Baby-foot project evolution

As with many other projects in similar fields, positive results require time, effort and most importantly, persistence.
Eight years of time and effort persistently spent to achieve the set goal now yielded an impressive outcome.
 
Everything started in 2012 when various students passionate about this subject took a semester to focus entirely on the Baby-foot, putting theory into practice.
These students began with the creation of the mechanical part of the foosball table and the automation of movements that match human performance.
 
In 2013, successive students continued the improvement based on the cameras, so that the basis for ball positioning via laser could be developed.
Moreover, the mechanical design was optimized by rethinking the shape of the components and finally the control system.
 
Starting in 2015, the attention shifted to the tactics of the game: fast positioning, correct information delivery and measurement of the opponents’ performance.
Concentration on these features initiated the first steps towards autonomous strategic decision making.

2016 and 2017 were the years of vision calibration and strategy systems with an accent on ball control and position tracking.
These have been systematically upgraded further in 2019 together with capture systems development, measurement and testing.

Ball capture – Vision and Strategy

The skills needed to win a foosball table game include the ability to capture and manage the ball during the play.
For humans, time and experience are a must to master this, whereas a robot relies on the mechanical movement control of the arm and ball location.
 
Made from scratch by students, the robotic arms are managed by two algorithms that run on one computer.
First controls the physical motion of the arm and the second tracks where the ball is at any given time. 
 
The primary objective of the current project is to modernize the automatically controlled arms to intercept the ball and retain static possession.
To achieve that, computation of ball movement speed with the necessary dampening is needed to move arms in advance thus catching the ball.

In the current implementation of the project, image acquisition has played an important role in capturing ball position.
Students replaced the bottom of the foosball table with a transparent material and set up XIMEA high-speed cameras on the ground to film the game board.
 
The accuracy of the ball vision system depends on the data acquired by a xiQ camera from XIMEA which offers 648×488 resolution at more than 500 fps.
The camera is used at high framerate of 500fps to guarantee the precision necessary for fast paced object tracking.
 
By running the algorithms with the camera connected to the USB3 port, it is possible to reach a tracking rate ranging between 400 and 500 [Hz].
This means, there is a record of a new ball position every 2.5 [ms] in usual conditions.
 
Knowing that a professional player can move the ball up to 15 [m/s], this would mean that one would get a measurement every 37mm of movement.
Nevertheless, when trying to shoot as fast as possible the students only reached speeds of about 3 [m/s], which means they got measurements every 7.5 [mm].
 
This is reasonable provided that the mechanical players on the table are distanced by 150mm between each row.
Thus, by applying the equation it was possible to theoretically compute the speed of the ball and predict its position before it reaches the next row of players. 

Who wins

Humans versus robots is nothing new in the 21st century, nevertheless, this topic finds new forms and remains one of the most discussed in different fields.
That is also what grasped the minds of the students of EPFL, their strive to develop an automated foosball table transformed the Laboratory into a training camp.
 
The robotic foosball table has been in use for several years, but recent upgrades have given the system more speed, accuracy and a power boost.
So much so that that human players with an average skill level are regularly being beaten by the robotic table soccer player.
 
The developers reckon that the winning advantage hinges on the millimeter precision and high-speed acceleration.
Despite the leaps forward in vision control mechanism and strategies, the table will want more refinements in software and hardware parts.
 
In particular, better speed estimation for the ball, predictive placement, advanced attacking and passing strategies and ultimately better overview in the edges.
These are the areas where Christophe Salzmann and his research students will have to win to make the robot win his matches.