{rfName}
A

Indexed in

License and use

Citations

5

Altmetrics

Grant support

This work has been supported by the Government of Spain through project BugBirth (RTI2018-101963-B-100), the Regional Government of Madrid (CM) through project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER and the European Commission through European Project H2020 822717: MICADO.

Analysis of institutional authors

Soto-Sanchez, OAuthorMaes-Bermejo, MCorresponding AuthorGallego, MAuthorGortazar, FAuthor

Share

September 27, 2022
Publications
>
Article
Hybrid Gold

A dataset of regressions in web applications detected by end-to-end tests

Publicated to:Software Quality Journal. 30 (2): 425-454 - 2022-06-01 30(2), DOI: 10.1007/s11219-021-09566-x

Authors: Soto-Sanchez, Oscar; Maes-Bermejo, Michel; Gallego, Micael; Gortazar, Francisco

Affiliations

Univ Rey Juan Carlos, Mostoles 28933, Spain - Author

Abstract

End-to-end tests present many challenges in the industry. The long-running times of these tests make it unsuitable to apply research work on test case prioritization or test case selection, for instance, on them, as most works on these two problems are based on datasets of unit tests. These ones are fast to run, and time is not usually a considered criterion. This is because there is no dataset of end-to-end tests, due to the infrastructure needs for running this kind of tests, the complexity of the setup and the lack of proper characterization of the faults and their fixes. Therefore, running end-to-end tests for any research work is hard and time-consuming, and the availability of a dataset containing regression bugs, documentation and logs for these tests might foster the usage of end-to-end tests in research works. This paper presents a) a dataset for this kind of tests, including six well-documented manually injected regression bugs and their corresponding fixes in three web applications built using Java and the Spring framework; b) tools for easing the execution of these tests no matter the infrastructure; and c) a comparative study with two well-known datasets of unit tests. The comparative study shows that there are important differences between end-to-end and unit tests, such as their execution time and the amount of resources they consume, which are much higher in the end-to-end tests. End-to-end testing deserves some attention from researchers. Our dataset is a first effort toward easing the usage of end-to-end tests in research works.

Keywords

Application buildsComparatives studiesDatasetEnd-to-end testsProgram debuggingRegression analysisRunning timeSoftware testingStatistical testsTest case prioritizationTest case selectionTest-case prioritizationTestingUnit testsWeb applicationWeb applications

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal Software Quality Journal due to its progression and the good impact it has achieved in recent years, according to the agency Scopus (SJR), it has become a reference in its field. In the year of publication of the work, 2022, it was in position , thus managing to position itself as a Q1 (Primer Cuartil), in the category Media Technology.

From a relative perspective, and based on the normalized impact indicator calculated from the Field Citation Ratio (FCR) of the Dimensions source, it yields a value of: 1.38, which indicates that, compared to works in the same discipline and in the same year of publication, it ranks as a work cited above average. (source consulted: Dimensions Aug 2025)

Specifically, and according to different indexing agencies, this work has accumulated citations as of 2025-08-07, the following number of citations:

  • WoS: 2
  • Scopus: 2

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-08-07:

  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 18 (PlumX).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

  • The work has been submitted to a journal whose editorial policy allows open Open Access publication.

Leadership analysis of institutional authors

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (Soto Sánchez, Óscar) and Last Author (Gortázar Bellas, Francisco de Asis).

the author responsible for correspondence tasks has been Maes Bermejo, Michel.