Implementation of Mobile UI Automation Rerun Based on Tesng+Cucumber

UI automation often needs maintenance because of its instability. In order to reduce the cost of maintenance, it is very important to restart configuration failure.

There are two main things to do in this failed re-run: the failed scene re-run and the report update de-duplication.

1. Failure and Rerun

Tesng has a retry method that provides the interface of re-run. It needs to implement the IRetryAnalyr interface. This method controls the number of re-run. The code is as follows:

public class Retry implements IRetryAnalyzer {
    private static Logger logger = Logger.getLogger(Retry.class);
    private int retryCount = 1;
    private static int maxRetryCount = 2;//Control the Number of Failure Reruns

    @Override
    public boolean retry(ITestResult result){
        logger.info("====Execution use cases:" + result.getName() + ",The first" + retryCount + "Second failure!");
        if(retryCount < maxRetryCount){
            retryCount ++;
            return true;
        }
        retryCount = 1;
        return false;
    }
}

Among them:

  • retryCount=1 is a must! Because the number of retries is common to all use cases, if the previous failed use cases use up the number of retries, and then there are failed use cases, then there will be no re-run, so when the number of re-runs reached the upper limit, the number of retries will be set to 1, so that there will be failed use cases, judging that the discovery is less than the maximum number of executions. Will continue to run again, unaffected.
  • Of course, there is a problem here. If the maximum number of executions is 2, this method is not a problem. If the maximum number of executions is greater than 2, then when the maximum number of executions is not 1, and there are failure cases, the number of re-runs will become smaller.
  • There is an implementation scheme on the Internet which adds retry times to onTestFailue, onTestSucess and other methods in the listener. This method is not feasible after I try, because if an interface has been failed, after each execution, the number of re-runs is set to 1, judging that the discovery is less than the maximum number of executions, it is still not feasible. It will continue to execute and fall into an infinite loop.

In addition to implementing the retry method, a listener is needed to monitor the retry method:

public class RetryListener implements IAnnotationTransformer {
    @Override
    public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod){
        IRetryAnalyzer retry = annotation.getRetryAnalyzer();
        if(retry == null){
            annotation.setRetryAnalyzer(Retry.class);
        }
    }
}

However, the re-run will inevitably lead to duplication of reports. The method of de-duplication implements onFinish method by adding a testng monitor. Only the last result is recorded. The code is as follows:

@Override
public void onFinish(ITestContext testContext) {
    super.onFinish(testContext);
    //Rerun after failure and record the last result
    Iterator<ITestResult> listOfFailedTests = testContext.getFailedTests().getAllResults().iterator();
    while (listOfFailedTests.hasNext()) {
        ITestResult failedTest = listOfFailedTests.next();
        ITestNGMethod method = failedTest.getMethod();
        if (testContext.getFailedTests().getResults(method).size() > 1) {
            listOfFailedTests.remove();
        }else if(testContext.getPassedTests().getResults(method).size() > 0) {
                listOfFailedTests.remove();
        }
    }
}

Then configure the listener to listen in testng.xml, otherwise the re-run and de-run will not work.~

<listeners>
        <listener class-name="com.netease.study.retry.RetryListener" />
        <listener class-name="com.netease.study.retry.TestngListener" />
</listeners>

If the generated report is a testng report, the above method has perfectly solved the problem, but because of the addition of cucumber, the generated report is a cucumber report (because the report is cool, really do not want to give up), so still need to duplicate the cucumber report. The report looks roughly as follows:

The cucumber report is actually generated by reading the cucumber.json file, which is roughly as long as this:

There are several scenarios in a file, and several jsons in elements. There are two scenarios in the screenshot above. If there is a failure and a re-run, two elements sub-json with id entical IDS will be generated in the elements in this file, which only needs to be de-duplicated.

Steps:

1. Traverse all the cucumber_xxx.json files under the cucumber Report

2. Read json files one by one for duplicate element s, and if so, put them in the remove list

3. Delete the element json data on which the first id matches (because the latest data needs to be retained after the re-run, so the data on the first run is removed). It just happens to be implemented by calling the remove method.

The code is as follows:

def handleDuplicateData(path):
    json_data = getJsonData(path)
    list = []
    removelist=[]
    elements = json_data[0]['elements']
    for element in elements:
        tmpId = element['id']
        if tmpId not in list:
            list.append(tmpId)
        else:
            removelist.append(tmpId)
    print(list)
    print("===removeList:{}".format(removelist))

    for e in elements:
        if e['id'] in removelist:
            elements.remove(e)
            removelist.remove(e['id'])
    json_data[0]['elements'] = elements

    f2 = open(path,"w+",encoding="utf-8")
    f2.write(str(json.dumps(json_data)))
    f2.close()

At this point, the automation of the re-run and the report of the heavy.

Tags: JSON less xml encoding

Posted on Thu, 29 Aug 2019 02:06:57 -0700 by evmace