Competitions as Scientific Method

Martina Seidl (Johannes Kepler University Linz)
 
Abstract: Automated reasoning as successfully applied for software and hardware verification is a complex task. It is supported by advanced reasoning tools like solvers and theorem provers that implement sophisticated techniques for solving provably hard problems. To objectively evaluate the state of the art, research communities organize competitions that distinguish the fast and best tools. Besides providing insights into recent tool developments, such competitions also identify interesting research problems. Thus the outcome of competitions motivates researchers to push the boundaries of their technologies, further improving the state of the art. In this talk, we take a closer look at the role of software  competitions as a scientific method. Based on the examples of some successful competitions, we explain their general setup and how they contribute to the  scientific progress of a research community.