Quotation Polyvyanyy, Artem, Solti, Andreas, Weidlich, Matthias, Di Ciccio, Claudio, Mendling, Jan. 2020. Monotone Precision and Recall Measures for Comparing Executions and Specifications of Dynamic Systems. ACM Transactions on Software Engineering and Methodology. 29 (3)




The behavioural comparison of systems is an important concern of software engineering research. For example, the areas of specification discovery and specification mining are concerned with measuring the consistency between a collection of execution traces and a program specification. This problem is also tackled in process mining with the help of measures that describe the quality of a process specification automatically discovered from execution logs. Though various measures have been proposed, it was recently demonstrated that they neither fulfil essential properties, such as monotonicity, nor can they handle infinite behaviour. In this article, we address this research problem by introducing a new framework for the definition of behavioural quotients. We prove that corresponding quotients guarantee desired properties that existing measures have failed to support. We demonstrate the application of the quotients for capturing precision and recall measures between a collection of recorded executions and a system specification. We use a prototypical implementation of these measures to contrast their monotonic assessment with measures that have been defined in prior research.


Press 'enter' for creating the tag

Publication's profile

Status of publication Published
Affiliation WU
Type of publication Journal article
Journal ACM Transactions on Software Engineering and Methodology
Citation Index SCI
WU-Journal-Rating new INF-A
Language English
Title Monotone Precision and Recall Measures for Comparing Executions and Specifications of Dynamic Systems
Volume 29
Number 3
Year 2020
Reviewed? Y
URL https://doi.org/10.1145/3387909
DOI https://doi.org/10.1145/3387909
Open Access N


Di Ciccio, Claudio (Former researcher)
Mendling, Jan (Details)
Polyvyanyy, Artem (University of Melbourne, Australia)
Solti, Andreas (-, Germany)
Weidlich, Matthias (HU Berlin, Germany)
Institute for Data, Process and Knowledge Management (AE Sabou) (Details)
Information Systems and Operations Management DP (Details)
Competence Center for Sustainability Transformation and Responsibility WE (Details)
Research Institute for Cryptoeconomics FI (Details)
Research areas (Ă–STAT Classification 'Statistik Austria')
5367 Management information systems (Details)
Google Scholar: Search