Skip to main content

Advances, Systems and Applications

A microservice regression testing selection approach based on belief propagation


Regression testing is required to assure the quality of each iteration of microservice systems. Test case selection is one of main techniques to optimize regression testing. Existing techniques mainly involve artifacts acquisition, processing and maintenance, thus hard to apply in microservice regression testing since it is difficult to obtain and process required artifacts from multiple development teams, which is normal in cases of microservice systems. This paper proposes a novel approach, namely MRTS-BP, which takes API gateway logs instead of artifacts as inputs. By mining service dependencies from API gateway logs, MRTS-BP analyzes service change impacts based on a propagation calculation, and selects test cases affected by changes based on impact degree values. To evaluate the effectiveness of MRTS-BP, empirical studies based on four real deployed systems are presented. Retest-all strategy and a regression testing selection approach based on control flow graphs called RTS-CFG are compared with MRTS-BP. The results show that, MRTS-BP can significantly reduce both the number of test cases and overall time cost while maintaining the fault detection capability of selected test suite, and that MRTS-BP can save more time cost than RTS-CFG with the similar safety and precision.


Microservice architecture is an effective architecture pattern popularly used in developing current cloud applications, for which services are built independently and integrated at run time by using container technique such as docker [1,2,3]. Microservice architecture well supports frequent business expansion and smooth upgrading by facilitating independent development and deployment of services [4]. Whenever service modification happens, regression testing is required to detect any potential faults introduced in modifications [5]. Common strategy employed in regression testing is to rerun previously used test cases (referred to as original test suite), namely retest-all strategy. The cost of retest-all strategy, however, might be not acceptable in the case of microservice system with a large amount of services deployed and rapid iterations. For example, WeChat, a social microservice system, has tens of thousands of services deployed and takes several months normally to conduct regression testing with retest-all strategy [6]. To reduce testing cost, many techniques such as test case prioritization, test suite minimization and testing selection are proposed and applied [7].

Regression testing selection (RTS) reduces testing cost by selecting a subset, selected test suite, from original test suite to intentionally cover modules which are changed or affected by other changed modules introduced in last iteration [5]. Many researches at first identify targeted modules by change impact analysis and then produce selected test suite [8,9,10,11]. Artifacts such as requirement specification, design model and code file are required usually to conduct change impact analysis effectively. For microservice testing, covering scenarios of service invocations is one of main test objectives [4, 8, 11], and thus a service invocation chain is usually set as a test case, also called a test path. With such test paths, artifact based RTS extracts service dependencies from artifacts to analyze which invocation chains may be affected by service modifications such that corresponding test paths can be identified and selected. In addition, safety and precision are also considered while evaluating RTS approaches [5]. A RTS approach is safe if it will contain all test cases that can reveal faults in regression testing, and a RTS approach is precise if it will not contain any unnecessary test cases. To assure the safety, RTS approaches normally expand selected test suite, but that would decrease the precision. Usually, we want to improve precision (i.e. to reduce testing cost) while maintain the safety of test selection at an accepted level.

However, challenges might arise when artifact-based RTS approaches applied in microservice regression testing: (1) Artifacts acquisition. When the microservice system under testing is developed by multiple teams, it faces several issues while achieving artifacts. Extra communication is acquired for gathering which results into elevating total cost. Additionally, expense is increased through obedience with numerous security strategies [8]; (2) Artifacts processing. Diverse development approaches (e.g. different modeling methods and coding frameworks) applied [12,13,14] increase the difficulty to agree on the integrity, comprehensibility, and consistency of artifacts, which will seriously hinder the performance of artifacts processing; (3) Artifacts maintaining. With growths of system scale in uninterrupted environments, maintenance of artifact versions will cost extra. Especially when multiple versions of a service need to run together, the integration of artifacts might bring confuses. Hence, above three main challenges indicate that artifact-based RTS approaches are not suitable for microservice regression testing.

Furthermore, API gateway layer of microservice system logs every API invocation at runtime for quality of service investigation, including requester, responder, time, status code, etc. [4]. Given a large amount of API gateway logs collected, frequent collaboration patterns among services can be mined to indicate service dependencies. Based on service dependencies, change impact analysis can be conducted based on belief propagation [15], which can conquer above challenges, this motivates the approach presented in this paper, microservice regression testing selection technique based on belief propagation (MRTS-BP).

Initially, MRTS-BP generates service dependency matrix (SDM) by mining service dependencies at business level from API gateway layer logs. Secondly, a directed graph of dependencies among services is established from SDM, and is inputted to a change impact propagation algorithm to measure change impacts quantitatively. Thirdly, existent satisfaction, complete satisfaction and k-existent satisfaction are adopted for different cases to generate selected test suite. Moreover, to evaluate the effectiveness of MRTS-BP, we conduct empirical studies on four real deployed systems to measure reduction rate of testing cost, recall, precision and F-measure. Retest-all strategy and a typical artifact-based RTS called RTS-CFG [16] are compared with MRTS-BP. The results show that, MRTS-BP can significantly reduce both the number of test cases and overall time cost while maintaining fault detection capability of selected test suite, and that MRTS-BP can save more time cost than RTS-CFG with similar safety and precision.

Main contributions of this paper are: (a) data utilizes to select test cases are extracted from API gateway logs; (b) change impacts are quantitatively calculated through mathematical approach; (c) three selection strategies are proposed to meet practical testing scenarios.

The rest of this paper is structured as follows: Related work section presents related work on microservice testing, regression testing selection and belief propagation. Methodology section presents MRTS-BP in detail, Empirical study and Results and discussion sections present empirical analysis, and Conclusions and future work section discusses conclusions and future work.

Related work

Microservice testing

Microservice testing comprises unit testing, service testing, and end-to-end testing [4]. However, unit testing is utilized to identify culpabilities in functions or classes. Additionally, this process is sustained by two experimental tools such as xUnit [17] and mockito [18]. In order to bypass user interference and rapid assessment, the service-testing is preferred. End-to-end testing focuses on behaviors of entire system. Due to challenges by service autonomy, dynamic binding and access restrictions [8, 11], service testing and end-to-end testing are very different from traditional software, thus leading to many researches and practices [4]. Whereas, test cases in this paper cover both services and end-to-end behaviors of the microservice system under testing.

Furthermore, both procedures at same interval involve multiple services and their invocations. For instance, consumer-driven based evaluation includes consumer, target, and stubbed-services [4]. Therefore, test cases of these procedures are abstracted as test paths [19,20,21,22], defined as follows:

Definition 2.1 test path

Let <si,sj > represent an invocation between two services, a test path is a sequence of <si,sj > and each of elements in the sequence can be a single invocation or a sequence composed of multiple invocations. A test path can be formally defined as a recursive regular expression tp = <<si,sj > (,<si,sj>)* > | < (tp,)+ > .

In our work, since logs of microservice systems collected present interfaces exposed by services, the granularity of “service” in Definition 2.1 is a service interface.

Regression testing selection

Formal definition of regression testing selection problem is follows as [5]:

Definition 2.2 Regression Testing Selection Problem, RTS issue.

  • Given: The program, P, the modified version of P, P′ and a test suite, T.

  • Problem: Find a subset of T, Ts, with which to test P′.

Most of existing RTS approaches mainly concentrate on formal presentation of change impact scopes from P to P′ and searching which test cases cover these scopes [5]. Due to close relationship between RTS approaches and system architectures, with continuous development of architecture paradigms, various RTS approaches for different architecture patterns are proposed, which can be divided into two categories: independent program oriented RTS and web service oriented RTS.

Early application systems have relatively small set of functionalities, which are mainly in forms of independent programs. RTS researches concentrate on change impact analysis with code files, such as data flow analysis approach, graph traversal approach, firewall approach, etc. Data flow analysis approach extracts information detailing locations of definitions and uses, which is needed by an inter-procedural data flow tester to guide the selection and execution of test cases [23]. Though applied in regression testing for spreadsheet programs [24], such approach is difficult to conducted for codes that does not cover data flows. Graph traversal approach relies on graph models such as control dependency graph [25], program dependency graph [26], system dependency graph [26], control flow graph [27, 28]. This approach usually includes two phases: analysis and selection. In analysis phase, different granularities of graph models are established from P to P′ to identify change impact scopes; in selection phase, relationships between such scopes and test cases are established, and then, test cases that relates to change scopes are selected. Integration scopes affected by changes are defined as “firewalls” in firewall approach, which is proposed for module integration testing [29, 30]. Based on firewalls figured out from code files, test cases covering firewalls are selected.

With the wide deployment of web applications, a large number of RTS researches concentrate on the issue of web services regression testing selection. Since web service testing concentrates on service compositions [8, 11], change impact analysis are conducted based on specifications and behavior models of web services. Treating web services testing as black box testing, web service description language (WSDL) based specifications for functions from an end-users point of view, are required as inputs to figure out change scopes to select test cases [31, 32]. In analogy to graph traversal approach, a two-stage RTS based on control flow graphs is proposed for web service regression testing selection [16] (referred to as RTS-CFG in the following). Such approach includes initialization stage and key stage: in initialization stage, artifacts of system under testing are collected and control flow graphs are established to represent service invocation logic; in critical stage, test cases are selected according to dangerous edges of control flow graphs. Considering different granularities of services, various RTS approaches are proposed. For service endpoints, a RTS approach is presented based on path analysis [21]. This approach compares the invocation path changes between service endpoints before and after iteration, and then selects test cases covering such changes. For service interfaces, a RTS approach based on service interface contract analysis is proposed [20]. In this approach, conflicts caused by contract changes are figured out to select test cases covering such conflicts. For services as a whole, a RTS approach is presented based on business process modeling [22]. Such approach requires structured business logic specifications as inputs and follows the graph traversal method. For large scale systems, a RTS approach needs to inject additional codes into services to collect relationship data between JAVA codes and test cases [14], but its implementation depends on Google’s infrastructure, which makes it not universal.

Compared with program oriented RTS, web service oriented RTS approach tends to select test cases based on change analysis of artifacts such as specifications and models. When such RTS approaches are applied in microservice regressing testing, artifacts collection, consistency checking, information extraction and modeling are necessary, but require substantial efforts to implement since microservice systems are usually developed by multiple teams and using different techniques. Whereas, MRTS-BP is proposed to replace artifacts processing with extracting service dependencies from API gateway logs based on frequent pattern mining [33].

Frequent pattern mining

General process of frequent pattern mining is: given a frequent threshold, when the frequency of an item set in transaction set exceeds the threshold, the item set is considered as a frequent item set, which can be used to generate association rules [33]. The frequency of an item set in the transaction set is called “support”, which is computed as the ratio of the number of transactions containing the item set to the size of the transaction set. Frequent item sets with k elements are called k-frequent item sets. Non-empty subsets of a frequent item set must be frequent item sets. Therefore, the support of a frequent item set must be less than or equal to that of its non-empty subsets.

There are many types of frequent pattern mining algorithms, such as candidate set based algorithms, tree based algorithms and recursive suffix based algorithms [33], which are customized according to meet practical requirements of mining problems. In our approach, the mining problem is to extract service dependencies as a basis for change impact analysis.

Belief propagation

Belief propagation algorithm (BP) is a repetitive process for estimated interpretation based on graph structure. There are numerous applications of BP which include: forward propagation algorithm, the Viterbi algorithm, decoding algorithms of low density parity check (LDPC) and turbo codes. Such methodologies are utilized for different scenarios [15]. Generally, BP algorithm is follows as.

  1. (1)

    Initialization: setting initial value of each node.

  2. (2)

    Propagation: update all message values and node confidence values.

  3. (3)

    Determining whether node confidence values are convergent. Incase convergent, inference results obtained according to confidence values. Otherwise, it will jump back to step (2) and propagate iteratively.

In recent years, studies on BP algorithm comprise application and optimization. According to its application, scholars predominantly focus on communication coding and signal processing. In order to reduce complexity of sparse code multiple access, dynamic edge assortment procedure based on BP algorithm is introduced. Through iterative calculation, range boundaries of nodes are detected [34]. However, nonlinear equalization method utilized neural network where BP algorithm is applied to remove signal noises [35]. Additionally, for massive multiple-input multiple-output channel detection, BP is used purely based on deep neural network [36]. In optimization aspect, investigators primarily focus on implementation and convergence condition. Furthermore, LDPC along computational process assists in parallelization and merging memory access [37]. Beside this, convergence problem of BP algorithm and numerical polynomial-homotopy-continuation method revealed influence of structures. Therefore, parameters of graph models solved through fixed points [38].

Literature study reveals BP algorithm is not yet applied in microservice regression testing selection. Proposed work acquires to analyze impacts based on service dependencies from API gateway logs. Additionally, when service dependencies are transformed to a directed graph, impact analysis can be translated into impact propagation from some nodes to others, which can be addressed by BP-like methods.


MRTS-BP resolves issue as given in Definition 2.2 for microservice systems. This problem is tackled through three steps: service dependency mining, change impact analysis and test case selection as displayed in Fig. 1. Furthermore, Inputs primarily comprise API gateway logs and original test path set. While, the output is selected test path set to be re-tested.

Fig. 1
figure 1

Flowchart of MRTS-BP

Service dependency mining

Microservice systems provide user accessible functions through service cooperating, leading to data exchanges and service invocations, called service dependencies [4]. API gateway logs record requests among services. One can see business flow and data flow triggered by users. Thus, our approach mines service dependencies from API gateway logs to generates service dependency matrix (SDM). This step mainly contains two activities: data preprocessing and service dependency matrix generation. The former establishes user request chains from logs and generates a transaction set, while the latter generate a SDM from the transaction set.

Data preprocessing

To facilitate log mining, raw data should be preprocessed to remove irrelevant items and to form into structured data [39]. API gateway logs may contain requester address, service name, service address assigned by load balancer, status code and so on, though its concrete structure varies from system to system. The first step in data preprocessing is to remove irrelevant items such as self-checking records from API gateway logs. Only required data fields such as requester address, service name and service address will be retained to form a structured data set.

Next, with cleaned data, our approach takes user requests as starting point to search for associated service invocations and then formulates into service invocation chains to represent a user session, called as user session extraction. Extraction process is implemented based on another key component of microservice systems called “service chain monitoring”. Service chain monitoring mainly collects, analyzes and displays service invocations while microservice systems running, which supports for fault diagnosis and performance optimization, e.g. Open Zipkin of Twitter, CAT of, and Naver Pinpoint etc. Service invocations associated with the same user request share the same tracing ID. Therefore, a list of service invocations representing a user session can be formed by determining the consistency of tracing ID.

Then, a transaction set for mining is generated from service invocation chains. Considering that service invocations can directly represent dependencies between services, our approach takes an invocation as an item, and takes a service invocation chain as a transaction. Formally, let S = {sj|0 < j ≤ n} represent the service set of a microservice system and n denote the number of services, then:

Definition 3.1 Global Item Set.

A global item set Ia = {<sj,sk > |sj,skSj ≠ k}, where item ijk = <sj,sk > represents sj invocates sk.

Definition 3.2 Transaction Set.

A transaction set D = {T|T  Ia}, where T represents a transaction.

According to Definitions 3.1 and 3.2, transaction set generation can be implemented by traversing the service invocation chains once, which is as follows: (1) initialize a transaction set as an empty set; (2) traverse each invocation of each service invocation chain, and denote each invocation as an item. After removing duplicate items, an item set is generated as a transaction and appended to the transaction set; (3) after traversing all service invocation chains, output the transaction set.

Service dependency matrix generation

Service dependencies are mainly derived from two type of sources:

  1. (1)

    Request flows

    Request flows are represented as invocation chains of services, which can be decomposed into invocations between services. If an invocation occurs frequently, it can be inferred that there may be a dependency between corresponding two services, which is defined as “request dependency” in our approach. Since an invocation is defined as an item in Definition 3.1, request dependencies are represented as 1-frequent item sets.

  2. (2)

    Data flows

    On the one hand, data flows may directly occur with a single invocation between services, which can be also considered as the category of request dependency. On the other hand, data flows may occur indirectly through multiple invocations, two basic cases of which are shown in Fig. 2. In Fig. 2, invocations between service 1 and service 3 do not exist. In the left case, service 2 is invocated through Req12 and Req32 respectively. When Req12 changes some persistent data in service 2 and Req32 needs to query such data, service 3 indirectly exchanges the data with service 1. A typical example is data subscription with data decoupling patterns proposed in [4], which includes customer management service (service 1), subscription management service (service 2) and report service (service 3). Customer management service sends new customer data incrementally to subscription management service, and report service queries subscription data from subscription management service, including customer data. In this example, report service does not directly interacts with customer management service, but the former relies on the latter indirectly through the customer data. When customer management service changes (for example, the structure of customer table is changed), such changes may also affect report presentation of report service, which is needed to verify in regression testing. Similarly, in the right case of Fig. 2, service 2 invocates service 1 and service 3 through Req21 and request Req23 respectively. When parameters of Req23 include some data returned by Req21, service 3 may indirectly rely on service 1 through such data. Service dependency generated by indirect data exchange is defined as “data dependency” in our approach. From the perspective of frequent patterns, data dependencies are expressed as 2-frequent item sets in the transaction set, and requestors or responders of two items are the same.

Fig. 2
figure 2

Two basic cases of indirect data exchange

Through the analysis above, it is concluded that the request flows may lead to request dependencies, while data flows may lead to both request dependencies and data dependencies. In order to measure the possibility of a service dependency quantitatively, confidence value is defined as follow:

Definition 3.3 Confidence of service dependencies.

Given frequent threshold c, let F1, F2 respectively represent the set of 1-frequent item sets and the set of 2-frequent item sets in transaction set D, count(I) represent the number of transactions containing item set I in D. Then the confidence value of service si depending on sj is given by equations in Formula 1:

$$conf\left({s}_i,{s}_j\right)=\left\{\begin{array}{ll}\frac{count\left(\left\{<{s}_i,{s}_j>\right\}\right)}{\mid D\mid },& \left\{<{s}_i,{s}_j>\right\}\in {F}_1\\ {}\frac{{\left( count\left(<{s}_i,{s}_k>,<{s}_j,{s}_k>\right)\right)}^2}{\mid D\mid \times count\left(<{s}_i,{s}_k>\right)},& \left\{<{s}_i,{s}_j>\right\}\notin {F}_1\wedge \left\{<{s}_i,{s}_k>,<{s}_j,{s}_k>\right\}\in {F}_2\\ {}\frac{{\left( count\left(<{s}_k,{s}_i>,<{s}_k,{s}_j>\right)\right)}^2}{\mid D\mid \times count\left(<{s}_k,{s}_i>\right)},& \left\{<{s}_i,{s}_j>\right\}\notin {F}_1\wedge \left\{<{s}_k,{s}_i>,<{s}_k,{s}_j>\right\}\in {F}_2\\ {}0,& otherwise\end{array}\right.$$

Formula 1 divides confidence degree of service dependency into four situations: the first equation computes the confidence of service dependency between two services in the same invocation, corresponding to request dependency, which is measured by the support of 1-frequent item set; the second and the third equations compute the confidence of service dependency between two services in different invocations that belong to a same data flows, corresponding to data dependency, which are measured by the product of the support of 2-frequent item sets and the confidence of association rules from corresponding tuples [33] (the second equation corresponds to data dependency in left case of Fig. 2, while the third equation corresponds to right one). Except for above situations, it is considered that there is no dependency between services, and the confidence is defined as 0.

For example, in Fig. 2, supposing that Req12 appears 500 times in logs and the log size is 2000, given c = 0.2, then <s1,s2 > F1 and conf(s1,s2) is 0.25 according to the first equation of Formula 1. Supposing that Req13 does not appear anywhere and Req12, Req32 appear together in the same transaction 400 times in logs, them {<s1,s2>, <s3,s2>}F2 and service 1 may depend on service 3 via data flow with global probability conf(s1,s3) = 0.25 × 0.8 = 0.2 according to the second equation of Formula 1. Meanwhile, service 3 may also depend on service 1 via data flow based on the confidence of Req32.

It is noted that the support of 2-frequent item sets is less than or equal to the support of 1-frequent item sets, and this method does not consider service dependencies whose confidence is less than c. Then, the definition of service dependency matrix is as follows:

Definition 3.4 Service Dependency Matrix, SDM

Given a service set S = {si|0 < i ≤ n}, SDM is a n-order square matrix, and its element aij in row i and column j is defined as follows:

$${a}_{ij}=\left\{\begin{array}{ll} conf\left({s}_i,{s}_j\right),& i\ne j\wedge conf\left({s}_i,{s}_j\right)\ge c\\ {}0,& otherwise\end{array}\right.$$

Based on Definition 3.3 and Definition 3.4, an algorithm for generating SDM from D can be proposed as Algorithm 1. Firstly, the algorithm constructs 1-frequent item sets, and generates candidate sets by Cartesian product, and then removes infrequent item sets by c to obtain 2-frequent item sets (line 1 to 4). Secondly, SDM is initialized as an n-order zero square matrix (line 5). Based on the first equation of Formulas 1 and 2, the element of SDM corresponding to 1-frequent item is set as the support value (line 6 to 8). Thirdly, by traversing 2-frequent item sets, when tails of two invocations are the same, corresponding element of SDM is updated as the maximal value of its current value and the result computed with the second equation of Formula 1 (line 10 to 18); when heads of two invocations are the same, the third equation of Formula 1 is adopted (line 19 to 28).

figure a

Algorithm 1 Service dependence matrix generation algorithm

For example, a SDM of 9 services generated from logs is shown in Table 1. As Table 1 shows, elements on the diagonal of the SDM are all 0, and elements above 0 indicate dependencies where the confidence level exceeds given threshold. According to Algorithm 1, non-zero elements of SDM are related to F1, F2 and c, and therefore to the scale of logs and frequency threshold. Theoretically, the scale of logs is larger, more dependencies among services are covered, and there will be more SDM non-zero elements. Given logs, frequency threshold is smaller, more SDM non-zero elements will be generated. More non-zero elements indicate that SDM is more complete, which can affect the safety of test case selection, and it will be discussed in Empirical study section further.

Table 1 An example of SDM

Change impact analysis

Change impact analysis also including two activities: directed graph generation and impact propagation computing. The former builds a directed graph model from SDM, while the latter measures impacts by an impact propagation algorithm.

Directed graph generation

Directed graph is used to represent impact propagation network of services. Nodes of the graph represent services and directed edges represent propagation paths among services. Since each element of SDM represents the confidence of corresponding service dependency, the weight of each directed edge can be initialized. Therefore, based on Definition 3.4, a directed graph can be defined as follows:

Definition 3.5 Directed Graph, DG

A directed graph for impact propagation is a tuple DG = (N,E), where node set N=S, edge set E = {eij| ajiSDMaji>0}, eij represents a directed edge from si to sj, and w(eij) = aji represents the weight of eij.

Based on this definition, an algorithm of directed graphs generation from SDM is shown in Algorithm 2. Node set of DG (line 1) is established based on service set. And then, all elements of SDM are traversed (line 2 to 12) while directed edges are established between nodes with confidences greater than 0. The direction of a directed edge is opposite to the direction of corresponding service dependency (lines 5 to 8). For example, a DG corresponding to Table 1 is shown in Fig. 3. It can be seen that directed edges on DG are inverted with respect to non-zero elements in SDM.

figure b

Algorithm 2 Directed graph generation algorithm

Fig. 3
figure 3

An example of DG

Impact propagation calculation

Given a service set S and its modified version S′, it is easy to obtain a list of changed services with service registries [4], which is denoted as ∆S. Since services affect each other through service dependencies during changing, based on DG, change impact analysis can be translated into a quantitative assessment of impact propagation from some nodes in ∆S to others, which can be addressed by BP-like methods. The difference is that messages in belief propagation are used to calculate the probability of edge distribution of nodes, while messages in impact propagation are used to calculate the probability of nodes being affected by changes. Referring to the framework of BP algorithm, an algorithm of node updating and message propagation is proposed, and its convergence of iterative calculation process is also analyzed.

Node updating

Since there is no limit to the sum of change impacts on all service nodes, standard BP algorithm can not be applied. Our approach defines “impact degree” to measure change impacts of service nodes. Impact degree of nodes in ∆S is defined as 1, as the upper limit value, and impact degree of nodes not affected by changes is defined as 0, as the lower limit value. When a node acts as the end node of a directed edge, its impact degree may be updated with the message passing from the directed edge. The updated value of this node should be the maximum value of all messages sent to it and its current impact degree. The formal definition is as follows:

Definition 3.6 Impact Degree

Given a directed graph DG of Definition 3.5, let Ni represent the neighborhood of si, mji represent the message from sj to si, t represent iteration rounds of message propagation, then impact degree pt(si)[0,1] of si is recursively calculated as follows:

$${p}_0\left({s}_i\right)=\left\{\begin{array}{c}1,{s}_i\in \Delta S\\ {}0,{s}_i\notin \Delta S\end{array}\right.$$
$${p}_{t+1}\left({s}_i\right)=\max \left({p}_t\left({s}_i\right),\underset{j\in {N}_i}{\max}\left({m}_{ji}\right)\right)$$

Apparently, impact degree of a node after message propagation is not less than that before message propagation, that is, pt + 1(si) ≥ pt(si).

Message propagation

In a directed graph, through directed edge eij, impact degree of si can be propagated to sj based on weight w(eij), so the message is defined as follows:

Definition 3.7 Message

Given a directed graph DG of Definition 3.5, when eijDG.E, message mij propagated from si to sj is:

$${m}_{ij}={p}_t\left({s}_i\right)\times w\left({e}_{ij}\right)$$

Apparently, since w(eij)<1, message value is always less than current impact degree of the sender node, that is, mij<pt(si).

Convergence analysis

Definitions 3.6 and 3.7 show iterative computing process of change impact propagation. When there are no loops in DG, that is, a directed acyclic graph, propagation rounds of each node do not exceed the number of edges contained in the longest path of DG, so calculation process must be convergent. When there are some loops in DG, updating rounds are uncertain. In this case, it can be divided into two situations to analyze, changed services in the loops and not in the loops, respectively discussed as follows:

  1. (i)

    When there is a node s∆S in a loop, then p0(s) = 1. Because pt + 1(s) ≥ pt(s), and pt(s) ≤ 1, so pt(s) = p0(s) = 1, that is, impact degree of s will not be updated by message propagation. The directed edge with s as the end node does not work in computing process and can be considered as interrupted. At this time, the loop can be directly disconnected and transformed into a directed acyclic graph, so computing process converges, which can be shown schematically in Fig. 4.

Fig. 4
figure 4

A node of a loop is changed services

  1. (ii)

    For any node s of the loop, s ∆S, then p0(s) = 0. According to Definition 3.7, messages propagated on the loop is always 0 until the loop receives external input messages. When there are multiple external input messages, because the output of function Max only depends on values of parameters, but has nothing to do with the number of parameters, so we can suppose multiple external input messages reach the loop at the same time by aligning iteration rounds. Then computing process of the loop can be divided into three phases:

    • Computing all input messages. At this phase, because message propagations in the loop are not considered, it is transformed into a directed acyclic graph, and computing process converges in this phase;

    • Computing message propagations in the loop. After external input messages enter the loop at the same time, impact degrees of nodes are assigned with Formula 4. Through numerical comparison, node sm with maximal impact degree can be obtained, which is denoted as pt(sm). Since messages propagated in the loop satisfy mij<pt(si), maximal value of messages in the loop is less than pt(sm). Therefore, no matter how many rounds messages propagates in the loop, pt(sm) does not changes. That is, the loop can be disconnected from the directed edge with sm as the end node, and the loop can be removed, which means calculation process converges.

    • Computing all output messages. According to the calculation results of phase 2, output message of each node in the loop is calculated directly with Formula 5.

The schematic diagram of calculation process in case (ii) is shown in the Fig. 5.

Fig. 5
figure 5

No nodes of the loop are changed services

To summarize, computing process of impact degrees determined by Definition 3.6 and Definition 3.7 is convergent, that is, impact degree of each node can converge to a stable value in finite iteration rounds. The results of impact propagation computing can be put in a dictionary structure called change impact table (CIT) to access conveniently in our approach. The pseudo codes for generating a CIT from a DG is shown in Algorithm 3. Firstly, impact degrees of all nodes in the DG (line 1 to 3) are initialized according to changed service list, and the CIT (line 4) is also initialized correspondingly. Secondly, iterative computing process (line 5 to 14) is started. Message values (line 6 to 8) are computed with current impact degrees of nodes in each iteration, while impact degrees (line 9 to 13) are updated with the message values. The exit condition of iteration process is that impact degrees of all nodes in the CIT no longer change (line 14). For the DG in Fig. 3, supposing that p0(s6) = 1, p0(s7) = 1 and p0(s8) = 1, 4 iterations of impact propagation will be performed based on Algorithm 3. And the output CIT = {s1:0.2, s2:0.1, s3:0.022, s4:0.45, s5:0.32, s6:1, s7:1, s8:1, s9:0.0057}.

figure c

Algorithm 3 Impact propagation algorithm

Test case selection

This step selects a subset Ts from test path set T based on a CIT. For any test path tpT, a service set Stp can be derived from the invocations which make up tp with service registries. By querying a CIT, we can obtain impact degree of any service sStp, denoted as CIT(s). Then, to select test paths which are affected by changes, a threshold p can be set to determine whether CIT(s) of service s contained in the test path exceeds p. To avoid ignoring test paths affected by changes, p is set the minimum non-zero value of elements in a CIT. To adapt to different cases such as high safety, tight schedule etc., three selection strategies are proposed as follows according to the number of services that meet the conditions:

  1. (1)

    Existent satisfaction strategy. Given a filtering threshold p, if a test path tpT contains services satisfies CIT(s) ≥ p, then the test path is appended to Ts. That is,

$${T}_s=\left\{ tp|\exists s\in {S}_{tp}, CIT(s)\ge p\right\}$$
  1. (2)

    Complete satisfaction strategy. Given a filtering threshold p, for a test path tpT, if every service in Stp satisfies CIT(s) ≥ p, then the test path is appended to Ts. That is,

$${T}_s=\left\{ tp|\forall s\in {S}_{tp}, CIT(s)\ge p\right\}$$
  1. (3)

    The k-existent satisfaction strategy. Given a filtering threshold p, for a test path tpT, if the number of services in Stp, which satisfies CIT(s) ≥ p, exceeds k, then the test path is appended to Ts. That is,

$${T}_s=\left\{ tp|\underset{k}{\exists }s\in {S}_{tp}, CIT(s)\ge p\right\}$$

From above strategies, existent satisfaction strategy is the most relaxed strategy, and the scale of selected test path set is largest. Especially when p is the minimum non-zero value in CIT(s), it means that as long as a test path contains services affected by changes, such test path is selected. The complete satisfaction strategy is the most strict strategy, and the scale of selected test path set is smallest. The k-existent satisfaction strategy is between the two, which can be used to adjust the scale of selected test path set as needed.

Corresponding test case selection algorithms can be proposed. The pseudo codes of existent satisfaction strategy are shown in Algorithm 4, in which original test path set is traversed once (line 2 to 9). By querying a CIT, whether the current test path is selected into the TS (lines 3 to 8) can be determined. The pseudo codes of the other two strategies are similar and will not be described further.

figure d

Algorithm 4 Test case selection algorithm based on the existent satisfaction strategy

Empirical study

In order to evaluate MRTS-BP and analyze the influence of process parameters on testing selection, we implemented whole process based on Python 3.5, and collected testing data of four microservice systems for experimental analysis. Our empirical study investigates four research questions as follow:

  • RQ1 Whether MRTS-BP is safe or not, and how the value of the frequent threshold c affects its safety.

    An RTS technique is safe if it will contain all test cases revealing faults in regression testing [5]. Safety determines the availability of RTS techniques. In MRTS-BP, the frequent threshold c is directly related to the number of mined frequent patterns, and then affects network structure of directed graph, which has a great influence on the results of change impact propagation. Therefore, the number of test cases selected by MRTS-BP is related to the value of c. It is necessary to analyze the relationship between c and the safety of MRTS-BP, and find out the range of c that can ensure the safety.

  • RQ2 On the premise of guaranteeing safety, whether MRTS-BP can save testing cost of microservice regression testing.

    Compared with retest-all strategy, RTS techniques save testing cost by reducing the number of test cases, but extra time consumption is caused for selecting. When time cost saved are greater than extra time consumption, RTS techniques can save time cost of regression testing overall. Through the experiments, the reduction of the number of test cases and testing time cost are both counted to determine whether MRTS-BP can save testing cost of microservice regression testing.

  • RQ3 Compare MRTS-BP with a typical artifacts based RTS approach.

    Theoretically, MRTS-BP does not rely on artifacts such as specifications, design models and code files. It is completely decoupled from techniques for constructing of the systems under testing, that is, the scalability of MRTS-BP is obviously better than artifacts based RTS approaches. In order to make a more comprehensive comparison, an RTS approach based on control flow analysis (RTS-CFG) [16] is chosen to compare with MRTS-BP to reveal the practicability of the two in microservice regression testing.

  • RQ4 How to choose selection strategies of MRTS-BP to optimize time cost.

    Existent satisfaction strategy, complete satisfaction strategy and k-existent satisfaction strategy are proposed in our approach to meet different testing requirements. To analyze the influence of selection strategies on the efficiency of MRTS-BP, experiments are needed to clarify how the selection strategies affect the number of test cases selected, the safety and the precision of MRTS-BP, which will be helpful to select appropriate strategies in practice.

Case introduction

The following four microservice systems are adopted in our empirical study:

  1. (1)

    m-Ticket: a multi-end ticket system based on SpringBlade (an open source microservice framework available at, provides ticket services in various fields such as transportation, accommodation, tourist attractions and movies, supporting service management, monitoring and tracing.

  2. (2)

    z-Shop: a mobile oriented mall system based on Zheng (an open source microservice framework available at, provides one-stop management services for goods, stores, content promotion, orders, logistics, etc.

  3. (3)

    Need: a knowledge system based on Spring Cloud, provides data collection, auxiliary analysis, information extraction, knowledge graph construction, intelligent query and other knowledge graph management services.

  4. (4)

    JOA: an office automation system based on Spring Cloud, provides comprehensive information display, document circulation, process approval, plan management, organization personnel management, contract management, fund management and material management services for the organization with multiple departments and secret levels.

Table 2 shows the numbers of logs, services, versions, test cases and faults of all cases above. The number of faults is collected from corresponding testing reports where all faults found in testing are reported. Based on these data, a posteriori method is adopted to setup experiments, that is, execution results of test suite are known before, and main activities are to select test cases and to perform statistical analysis. As shown in Table 2, m-Ticket, z-Shop and Need are relatively small systems, but JOA have much more services respectively. To compare with artifacts based approaches, we also collect design documents, logs and testing data. Since multiple teams developed JOA and they did not agree to grant the access permission, we failed to collect corresponding artifacts for JOA. Artifacts based RTS approaches can not be applied in JOA.

Table 2 Properties of each case

Evaluation metrics

According to common RTS evaluation metrics [11,12,13,14], considering problems in experiments, 3 metrics are proposed as follows:

  1. (1)

    Testing time cost saving rate (ET) is used to measure the extent to which RTS approaches reduce regression testing time cost. Let TO represent total execution time cost of original test suite, TR represent execution time cost of selected test suite based on RTS approaches, and TS represent the time cost of selection process, then ET is given by equation in (9):

$$ET=\frac{TO- TR- TS}{TO}\times 100\%$$
  1. (2)

    Percentage reduction of the number of test cases (EN) is used to measure the ability of RTS techniques save testing cost only in terms of reducing the number of test cases [12, 13]. Let NO represent the number of original test cases, NR represent the number of selected test suite, then ET is given by equation in (10):

$$EN=\frac{NO- NR}{NO}\times 100\%$$
  1. (3)

    Recall (R) indicates the percentage of selected test cases relative to all failed test cases [11,12,13,14], which is used to measure the safety of RTS techniques. Let NOF represent the number of test cases revealing faults in original test suite, NSF represent the number of test cases revealing faults in selected test suite based on RTS techniques, then R is given by equation in (11):

$$R=\frac{NSF}{NOF}\times 100\%$$
  1. (4)

    Precision (P) indicates the accuracy with which test cases were selected to be rerun [11,12,13,14]. Since NR represents the number of selected test suite, then P is given by equation in (12):

$$P=\frac{NSF}{NR}\times 100\%$$
  1. (5)

    F-measure (F) is a combination of both P and R [11,12,13,14], which indicates the combination of safety and accuracy of RTS approaches, and F is given by equation in Formula 13. It can be seen that the lager F is, the better the combination of safety and accuracy is.

$$F=\frac{2\times P\times R}{P+R}$$

Experiments setup

According to research problems above, four experiments are setup as follows:

  • Experiment 1: determine the safety of MRTS-BP and its relationship with frequent threshold c

    For each case, from version v (v ≥ 2), following steps are carried out.

    1. (1)

      Taking logs of version v-1 as input, global item set and transaction set are generated respectively. After frequencies of all items obtained, the minimum value and maximum value are taken as the lower bound and upper bound of frequent threshold c respectively. Then divide the range of c into ten equal parts, and take the lower bound of each equal part as a value of c, which are denoted as ci (i = 1,2…10). The ten values of c are used respectively to generate SDMs through transaction set mining.

    2. (2)

      Based on the ten SDMs generated in step 1, ten CITs are generated respectively. For each CIT, the minimum non-zero element in CIT is taken as selection threshold p, and test cases are selected with existent satisfaction strategy.

    3. (3)

      NOF is counted from testing report of corresponding version. For each test suite selected, NSF and recall R are also counted.

  • Experiment 2: compare the ability to save testing cost of MRTS-BP and RTS-CFG

    For each case, from version v (v ≥ 2), following steps are carried out.

    1. (1)

      Find out selected test suite in Experiment 1 with smallest number of test cases and R = 100% (set NA when there is no R = 100%). Collect time cost to select such test suite, including the whole process of MRTS-BP, which is TS of MRTS-BP. Then, TO and TS are computed from testing report of corresponding version, and then ET is computed.

    2. (2)

      Apply RTS-CFG on each version with available artifacts and tools [16] to select a test suite with R = 100% (set NA when there is no available artifacts or no R = 100%). Collect time cost to select such test suite, including the whole process of RTS-CFG, which is TS of RTS-CFG. Similarly, count TO, TS and compute ET.

    3. (3)

      For MRTS-BP and RTS-CFG, count the number of test cases selected and compute EN respectively.

    4. (4)

      Compare ET, EN of MRTS-BP and RTS-CFG.

  • Experiment 3: compare the effectiveness of MRTS-BP and RTS-CFG

    For each case, from version v (v ≥ 2), following experimental steps are carried out.

    1. (1)

      With the results of step 1 in Experiment 2, compute recall R, precision P and F-measure F of MRTS-BP.

    2. (2)

      With the results of step 2 in Experiment 2, compute recall R, precision P and F-measure F of RTS-CFG (set NA when there is no available artifacts).

    3. (3)

      Compare R, P, F of MRTS-BP and RTS-CFG.

  • Experiment 4: analyze changing trends of EN, R, P and F with different selection strategies of MRTS-BP.

    For the last version of each case, following steps are carried out.

    1. (1)

      Find out the CIT of step 2 in Experiment 1 with smallest number of test cases and R = 100% (set NA when there is no R = 100%). Test cases are selected respectively with complete satisfaction strategy and k-existent satisfaction strategy (k = 2,3,4), if the maximal number of services contained in test paths is smaller than four, then the upper of k is set as such maximal number.

    2. (2)

      For each test case selected, compute EN, R, P and F.

    3. (3)

      Compare values of EN, R, P and F of different strategies by bar charts.

Results and discussion

Data and analysis

Data of Experiment 1 are shown in Table 3. Let Ri (i = 1,2…10) represent the value of R corresponding to ci of step 1, it can be seen that R can reach 100% with frequent threshold c assigned an appropriate value in each version of each case. That is because existent satisfaction strategy means that test cases including any services affected by changes will be selected. So it indicates that MRTS-BP can ensure safety when c is set appropriate with existent satisfaction strategy. To show change trend of R with c intuitively, the line chart Fig. 6 is drawn from the last version of each case. From Fig. 6, when c changes from minimum value to maximum value, R will gradually decrease. This is because when c becomes larger, less frequent patterns are mined, that is, less possible service dependencies are obtained, which leads to more zero elements in SDM, and further leads to less directed edges of directed graph. Since less edges of directed graph has, less nodes will be considered in change propagation computing, which may lead to that less services are appended to CIT. Then, less test paths will be selected with existent satisfaction strategy, which leads to the smaller value of R. Therefore, in order to ensure the safety of MRTS-BP, the value of c should be close to its lower bound value.

Table 3 Data of Experiment 1
Fig. 6
figure 6

The change trends of R with c in Experiment 1

Data of Experiment 2 are shown in Table 4. From EN of MRTS-BP, value ranges from 40% to 57%, and mean value is 50%, that is, the number of test cases is apparently reduced by applying MRTS-BP. Similarly, EN of RTS-CFG ranges from 34% to 56% with mean value 48%. It can be seen that, in terms of percentage reduction of test suite scale, MRTS-BP can save testing cost like the typical RTS technique. However, from values of ET, testing time cost saving rates of the two approaches are different. Since MRTS-BP has more data than RTS-CFG, boxplots in Fig. 7 are drawn to show the comparison of the two. From Fig. 7, MRTS-BP and RTS-CFG have similar mean values, standard deviation values and extreme values for EN. But for ET, mean value and extreme values of the former are apparently larger than those of the latter (the difference is at least 20%), that is, MRTS-BP can save more testing time cost than RTS-CFG. This is because RTS-CFG spends large amounts of time on artifacts processing, while MRTS-BP spends much less time on process log mining and compute change propagation. Consistency, comprehensibility, integrity, granularity other characteristics of artifacts seriously affect processing performance of RTS-CFG, while logs are structural and the process of MRTS-BP can be easy to automate. Therefore, MRTS-BP spends much less time than RTS-CFG during the selection phase. To RQ2, on the premise of guaranteeing safety, MRTS-BP can reduce testing cost of regression testing not only in the number reduction of test cases but also in time cost saving.

Table 4 Data of Experiment 2
Fig. 7
figure 7

The boxplot of EN and ET in Experiment 2

The Data of Experiment 3 are shown in Table 5. From values of R, MRTS-BP and RTS-CFG can ensure all test cases revealing faults are selected for each version of each case, that is, MRTS-BP and RTS-CFG are both safe. To intuitively compare P and F of the two techniques, line charts are drawn for each case in Fig. 8. From the line charts of case m-Ticket, z-Shop and Need, for P and F, it is can be seen that values and their change trends of MRTS-BP and RTS-CFG are almost the same. This is due to similar abilities of the two approaches to reduce the number of test cases and cover impact scopes of changes. Essentially, MRTS-BP and RTS-CFG both identify change impact scopes based on service dependencies. The difference lies in that, the former adopts impact propagation calculation while the latter adopts edge analysis based on control flow models. And it also indicates that BP-like algorithm worked in regression testing selection. On the other hand, since artifacts based RTS approaches are not adapted to the case which artifacts are difficult to obtain, such as JOA, the scalability of MRTS-BP is obviously better than that of RTS-CFG in practice.

Table 5 Data of Experiment 3
Fig. 8
figure 8

The line charts of each case in Experiment 3

The Data of Experiment 4 are shown in Table 6, and corresponding bar charts are shown in Fig. 9. From values of EN with different selection strategies in each case, it can be seen that more strict selection strategy is, less test cases are selected, and more testing cost are saved. However, that less test cases are selected means more test cases affected by changes are ignored, which can make MRTS-BP be not safe, as shown by the values of R in each cases. That is, EN and R are a pair of trade-off with different selection strategies, and one should choose strategies according to the actual case. From Fig. 9, existent satisfaction strategy can always ensure the safety of MRTS-BP, but EN and F with such strategy are not the best, which means MRTS-BP can be applied in the case of high safety. Complete satisfaction strategy save more testing cost than others, but it is far less safe than others, which means such strategy can be applied in the case of tight schedule. The efficiency of k-existent satisfaction strategy is determined by the value of k. When k is larger, the number of selected test cases will be significantly reduced, but the safety will also be worse, or even unavailable. When k is smaller, MRTS-BP has better safety, but testing cost reduction rate becomes lower. It is worth noting that when k is 2, R reaches 100% in three cases, and EN and F are better compared with those of complete satisfaction strategy. This indicates that 2-existent satisfaction strategy can be applied and may bring more efficiency.

Table 6 Data of Experiment 4
Fig. 9
figure 9

The bar charts of each case in Experiment 4

Threats to validity

As with most empirical studies, there are some risks to apply the conclusions directly of experiments above, and threats to validity mainly are manifested in two aspects:

  1. (1)

    Cases selection. Although our experiments consider factors such as size, complexity, and domain when selecting cases, there may be some limitations that do not cover all types of microservice systems. At the same time, methods of test cases generation, and whether faults records are comprehensive or not may also affect experimental results. It needs to be validated by more cases in different areas, different scales and different data distribution characteristics.

  2. (2)

    Comparison RTS techniques selection. The method for comparison, RTS-CFG, comes from related work, which also has validity risks that will be introduced into our experiments. At the same time, though RTS-CFG is a typical RTS technique relying on artifacts, it does not represent all artifacts based RTS techniques and our approach need to be validated against with more different artifacts based RTS techniques.

Conclusions and future work

This paper proposes a microservice regression testing selection approach MRTS-BP, describes the whole process in detail, and verifies its effectiveness through experiments. MRTS-BP conquer the challenges of artifacts based RTS approaches by processing API gateway logs instead of artifacts. For acquisition issue, API gateway logs are automatically and centrally recorded without business-specific data, which avoids additional communication costs and security risks. For processing issue, API gateway logs are structural and consistent, which leads to processing automatically with MRTS-BP. For maintaining issue, API gateway logs are identified clearly and accurately correspond to each version of services. Thus, MRTS-BP can be fully automated and is applicable to microservice systems regression testing in practice.

In future, two aspects which include granularity of MRTS-BP and service dependencies from API gateway logs must give insight overview to improve accordingly. Also, MRTS-BP in different fields and patterns like mesh service to collect more cases for empirical study.

Availability of data and materials

The data used to support the findings of this study are included within the article.


  1. Fan CY, Ma SP (2017) Migrating monolithic mobile application to microservice architecture: an experiment report. In: Proceedings of 2017 IEEE International Conference on AI & Mobile Services (AIMS), pp 109–112

    Chapter  Google Scholar 

  2. Lewis J, Fowler M (2014) Microservices: a definition of this new architectural term.

    Google Scholar 

  3. Larrucea X, Santamaria I, Colomo-Palacios R, Ebert C (2018) Microservices. IEEE Softw 35(3):96–100

    Article  Google Scholar 

  4. Newman S (2015) Building microservices: designing fine-grained systems. O’Reilly Media, Sevastopol

  5. Yoo S, Harman M (2012) Regression testing minimization, selection and prioritization: a survey. Softw Test Verif Reliab 22(2):67–120

    Article  Google Scholar 

  6. Gao C, Zheng W, Deng Y, Lo D (2019) Emerging app issue identification from user feedback: experience on WeChat. In: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP) 2019, Montreal, QC, Canada, May 25-31

    Google Scholar 

  7. Roberto P (2020) On the testing resource allocation problem: research trends and perspectives. J Syst Softw 161:110462

    Article  Google Scholar 

  8. Qiu D, Li B, Ji S, Leung H (2014) Regression testing of web service: a systematic mapping study. ACM Comput Surv 47(2):1–46

    Article  Google Scholar 

  9. Kazmi R, Jawawi DNA, Mohamad R, Ghani I (2017) Effective regression test case selection: a systematic literature review. ACM Comput Surv 50(2):29

    Google Scholar 

  10. Gligoric M, Eloussi L, Marinov D (2015) Practical regression test selection with dynamic file dependencies, ISSTA. ACM, New York, pp 211–222

  11. Ali S, Hafeez Y, Hussain S, Yang S (2019) Enhanced regression testing technique for agile software development and continuous integration strategies. Softw Qual Control.

  12. Rothermel G, Harrold MJ, Dedhia J (2015) Regression test selection for C++ software. Softw Test Verif Reliab 10(2):77–109

    Article  Google Scholar 

  13. Spoon SA, Jones JA, Li T (2001) Regression test selection for Java software. ACM Sigplan Not 36(11):312–326

    Article  Google Scholar 

  14. Zhong H, Zhang L, Khurshid S (2019) TestSage: regression test selection for large-scale web service testing. In: 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp 430–441.

    Chapter  Google Scholar 

  15. Yedidia JS, Freeman WT, Weiss Y (2002) Understanding belief propagation and its generalizations. Morgan Kaufmann, San Mateo

  16. Ruth M, Oh S, Loup A (2007) Towards automatic regression test selection for web services. In: Computer software and applications conference. COMPSAC, New York

  17. Meszaros G (2007) xUnit: test patterns refactoring test code. Addison-Wesley, Boston

    Google Scholar 

  18. Kaczanowski T. Practical unit testing with JUnit and Mockito., 2013

    Google Scholar 

  19. Li ZJ, Tan HF, Liu HH, Zhu J, Mitsumori NM (2008) Business-process-driven gray-box SOA testing. IBM Syst 47(3):457–472

    Article  Google Scholar 

  20. Khan TA, Heckel R (2011) On model-based regression testing of web-services using dependency analysis of visual contracts. In: Proceedings of the 14th International Conference on Fundamental Approaches to Software Engineering: Part of the Joint European Conferences on Theory and Practice of Software (FASE’11/ETAPS’11), pp 341–355

    Chapter  Google Scholar 

  21. Li B, Qiu D, Leung H, Wang D (2012) Automatic test case selection for regression testing of composite service based on extensible BPEL flow graph. Syst Softw 85(6):1300–1324

    Article  Google Scholar 

  22. Liu H, Li Z, Zhu J, Tan H (2007) Business process regression testing. In: Proceedings of the 5th International Conference on Service-Oriented Computing (ICSOC’07), pp 157–168

    Google Scholar 

  23. Harrold MJ, Soffa ML (1989) Interprocedual data flow testing. In: Proceedings of the Symposium on Software Testing, Analysis, and Verification, pp 158–167

    Google Scholar 

  24. Fisher M, Jin D, Rothermel G (2002) Test reuse in the spreadsheet paradigm. In: Proceedings of the International Symposium on Software Reliability Engineering, pp 257–268

    Google Scholar 

  25. Rothermel G, Harrold MJ (1993) A safe, efficient algorithm for regression test selection. In: Proceedings of International Conference on Software Maintenance, pp 358–367

    Google Scholar 

  26. Rothermel G, Harrold MJ (1994) Selecting tests and identifying test coverage requirements for modified software. In: Proceedings of International Symposium on Software Testing and Analysis, pp 169–184

    Google Scholar 

  27. Rothermel G, Harrold MJ (1997) A safe, efficient regression test selection technique. ACM Trans Softw Eng Methodol 6(2):173–210

    Article  Google Scholar 

  28. Beydeda S, Gruhn V (2001) Integrating white-and black-box techniques for class-level regression testing. In: Proceedings of the International Computer Software and Applications Conference, pp 357–362

    Google Scholar 

  29. White L, Robinson B (2004) Industrial real-time regression testing and analysis using firewalls. In: Proceedings of the International Conference on Software Maintenance, pp 18–27

    Google Scholar 

  30. White L, Jaber K, Robinson B (2008) Extended firewall for regression testing: an experience report. J Softw Maint Evol 20(6):419–433

    Article  Google Scholar 

  31. Paul R, Yu L, Tsai WT, Bai X (2001) Scenario-based functional regression testing. In: Proceedings of the International Computer Software and Applications Conference, (COMPSAC 2001), pp 496–501

    Google Scholar 

  32. Tarhini A, Fouchal H, Mansour N (2006) Regression testing web services-based applications. In: Proceedings of the IEEE International Conference on Computer Systems and Applications, (COMPSAC 2006), pp 163–170

    Chapter  Google Scholar 

  33. Aggarwal CC, Han J (2014) Frequent pattern mining. Springer International Publishing, Switzerland, pp 19–36

  34. Mao LI, Zhi-Gang Z, Tao W (2019) Multiuser detection scheme for SCMA systems based on stability of belief propagation. Computer Science, Beijing

  35. Yamazaki E, Farsad N, Goldsmith A (2019) Low noise non-linear equalization using neural networks and belief propagation

    Google Scholar 

  36. Tan X, Xu W, Be’Ery Y (2018) Improving massive MIMO belief propagation detector with deep neural network

    Google Scholar 

  37. Shan B, Fang Y (2020) GPU accelerated parallel algorithm of sliding-window belief propagation for LDPC codes. Int J Parallel Prog 48(3):566–579

    Article  Google Scholar 

  38. Knoll C, Mehta D, Chen T (2018) Fixed points of belief propagation -- an analysis via polynomial homotopy continuation. IEEE Trans Pattern Anal Mach Intell 9:2124–2136

    Article  Google Scholar 

  39. Lu Z et al (2003) Web Log Mining. Web Intelligence. Springer Berlin, Heidelberg, pp 173–194

Download references


All those who have contributed to this paper have been included in the list of authors.


This paper is supported by the scholarship project set up in Beihang University.

Author information

Authors and Affiliations



LIZHE CHEN contributed to the conception of the study. JI WU contributed significantly to analysis and manuscript preparation. HAIYAN YANG performed the experiments. KUI ZHANG performed the data analyses and wrote the manuscript. The author(s) read and approved the final manuscript.

Authors’ information

LIZHE CHEN received his master degree in computer application technology from National Defense University of Science and Technology, Hefei, China, in 2010. He is currently a Ph.D candidate in the oftware Engineering Institute (SEI) at Beihang University. His research interests include model driven engineering, data mining, machine learning, model base safety analysis, and software testing.

JI WU is associate professor of software engineering at Beihang University. He received his PhD degree from Beihang University in 2003 and MS degree from the Second Research Institute of the China Aerospace Science and Industry Group in 1999. His research interests include embedded system and software modeling and verification, software requirement and architecture modeling and verification, safety and reliability assessment, and software testing.

HAIYAN YANG is lecture of software engineering at Beihang University. She received her master degree in Computer Software and Theory at Beihang University in 2000. Her research interests include software modeling and verification,software testing, software measurement.

KUI ZHANG received his master degree in information management and information system from Beijing Institute of Technology, Beijing, China, in 2010. He is currently a Ph.D candidate in the oftware Engineering Institute (SEI) at Beihang University. His research interests include model driven engineering, model based real time analysis, airworthiness certification, model base safety analysis, and general model based software engineering.

Corresponding author

Correspondence to Ji Wu.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

All authors approved the final manuscript and the submission to this journal.

Competing interests

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, Lz., Wu, J., Yang, Hy. et al. A microservice regression testing selection approach based on belief propagation. J Cloud Comp 12, 20 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Microservice
  • Regression testing selection
  • Log mining
  • Belief propagation