Implement appium multiprocess compatibility test based on pytest


In practice, if you want to test the compatibility of multiple devices with appium, you may think of "multi-threading", but due to the influence of GIL in python, multi-threading can not achieve "multi-machine parallelism", so you can consider the use of multi-process at this time.

Why based on pytest

We know that in pytest can define different fixtures, and test case methods can call these fixtures to share data.The idea of the previous framework was that in the Common directory defines the method for generating driver --> calls the former to generate driver --> test case calls fixture under TestCases to implement driver sharing.But now it's different. We have several devices. If the information of these devices is simply written in yml, it doesn't seem convenient when we take them in parallel. Where can we write them? doesn't seem to be a good place to write device information either, leaving only at the end, and it's a good place to use as the gateway to multiple processes
But here's the problem. If we want to start multiple appium services, we need to consider the following points:

  1. How does appium start?
  2. How device information is passed to base_driver method to generate driver

The first point is very clear, the way the client starts the appium server seems a bit out of place. If you want to test five mobile phones at the same time, do you want to start the client one by one?The best way is to start the command line, because it's easier and faster to start
Before moving on to the second point, let's sort out the idea: defines multiple device information --> base_driver method calls, generates multiple drivers --> test cases under TestCases call fixture s, but how does the device information get passed to the base_driver method?This is when pytestconfig in pytest comes in handy

Using pytestconfig

The built-in pytestconfig controls pytest through command line parameters, options, configuration files, plug-ins, run directories, and so on.Pytest config is a shortcut to request.config and is sometimes referred to in the pytest documentation as the "pytest configuration object"
To understand how pytestconfig works, you can see how to add a custom command line option and read it in a test case.You can read custom command line options directly from pytestconfig, but in order for pytest to parse it, you also need to use the hook function pytest_addoption
Here are a few command line options added using pytest's hook function pytest_addoption

def pytest_addoption(parser):
	parser.addoption("--myopt", action="store_true", help="some boolean option")
	parser.addoption("--foo", action="store", default="bar", help="foo: bar or baz")

Now you can use these options in your test cases

import pytest

def test_option(pytestconfig):
	print("'foo' set to:", pytestconfig.getoption('foo'))
	print("'myopt' set to:", pytestconfig.getoption('myopt'))

Let's see how it works

E:\virtual_workshop\pytest-demo\test_demo7\pytestconfig>pytest -s -q
'foo' set to: bar
'myopt' set to: False
1 passed in 0.02s

E:\virtual_workshop\pytest-demo\test_demo7\pytestconfig>pytest -s -q --myopt
'foo' set to: bar
'myopt' set to: True
1 passed in 0.01s

E:\virtual_workshop\pytest-demo\test_demo7\pytestconfig>pytest -s -q --myopt --foo baz
'foo' set to: baz
'myopt' set to: True
1 passed in 0.01s

Because pytestconfig is a fixture, it can also be used by other fixtures.You can also create fixtures for these options if you like

def foo(pytestconfig):
def myopt(pytestconfig):
	return pytestconfig.option.myopt
def test_fixtures_for_options(foo, myopt):
	print("'foo' set to: ", foo)
	print("'myopt' set to: ", myopt)

Specific implementation


Now that you can use pytest command line parameters, you only need to add the parameter--cmdopt to pytest.main, which is similar:

import pytest, os
from multiprocessing import Pool

device_infos = [{"platform_version": "5.1.1", "server_port": 4723, "device_port": 62001, "system_port": 8200},
                {"platform_version": "7.1.2", "server_port": 4725, "device_port": 62025, "system_port": 8201}]

def run_parallel(device_info):
                 "--alluredir", "Reports"])
    os.system("allure generate Reports -o Reports/html --clean")

if __name__ == "__main__":
    with Pool(2) as pool:, device_infos)

Why do I only write four messages about the device?platform_version, server_port, device_port, system_port.Where are the others like appPackage, appActivity, platformName, etc?Of course, you can also write here. Other devices should be the same. I wrote it in the configuration information of yml

  • It is worth noting that server_port cannot be duplicated for multiple devices here. This is the port number that appium server starts. If server_port is duplicated for multiple devices, then only one service can be started, so it is different
  • What is system_port?This is to prevent the phenomenon of "competing for each other".When multiple processes and multiple devices are in parallel, if multiple devices use the same appium remote port at the same time (e.g. 8200).For multiple devices, they do not know to use the same port with each other, so there will be confusion in testing due to the incompatibility of Request and Action received from multiple devices, and an error "Original error:Can not proxy command to remote server" may appear

Define caps.yml under Caps

Basically, this defines the common part of desired_caps that is the same for multiple devices

platformName: Android
appPackage: com.xxzb.fenwoo
appActivity: com.xxzb.fenwoo.activity.addition.WelcomeActivity
newCommonTimeout: 500
noReset: False

Define under Common

Here are a few points to note:

  • When a multiprocess calls the base_driver method of a BaseDriver class, it should start the appium server from the command line when instantiating. Imagine what would happen if the appium server was started and placed in get_base_driver?Every time the get_base_driver method is called in conftest, a cmd window opens trying to start the appium server
  • The yaml.load method takes note of the new writing and adds the parameter Loader=yaml.FullLoader, which is said to be safer
from appium import webdriver
from .conf_dir import caps_dir
import yaml
import os

class BaseDriver:

    def __init__(self, device_info):
        self.device_info = device_info
        cmd = "start appium -p {0} -bp {1} -U{2}".format(self.device_info["server_port"], self.device_info["server_port"] + 1, self.device_info["device_port"])

    def base_driver(self, automationName="appium"):
        fs = open(f"{caps_dir}//caps.yml")
        #Platform name, package name, Activity name, timeout, reset, server_ip,
        desired_caps = yaml.load(fs, Loader=yaml.FullLoader)
        #Version Information
        desired_caps["platform_version"] = self.device_info["platform_version"]
        #Device Name
        desired_caps["deviceName"] = f"{self.device_info['device_port']}"
        #System Port Number
        desired_caps["systemPort"] = self.device_info["system_port"]

        if automationName != "appium":
            desired_caps["automationName"] = automationName

        driver = webdriver.Remote(f"{self.device_info['server_port']}/wd/hub", desired_capabilities=desired_caps)
        return driver


The key point is the use of pytest_addoption and request.config.getoption, which add a command line and parse a command line, but which still requires attention:

  • eval(cmdopt): eval is used to convert cmdopt to a dictionary because cmdopt itself is a string, like this: "{'platform_version':'7.1.2','server_port': 4725,'device_port': 62025,'system_port': 8201}", which is more inconvenient to use.
  • Additionally, there is a problem to solve. If there are multiple fixtures, the fixture used by the first test case must be guaranteed to instantiate BaseDriver, and the result of the instantiation, base_driver, must be used as a global variable for all fixtures to share. Otherwise, the problem of starting multiple cmd windows and launching multiple appium server s will arise
from common.base_driver import BaseDriver
import pytest

driver = None

def pytest_addoption(parser):
    parser.addoption("--cmdopt", action="store", default="device_info", help=None)

def cmdopt(pytestconfig):
    #Two Writing Styles
    return pytestconfig.getoption("--cmdopt")
    #return pytestconfig.option.cmdopt

#Define a common fixture
def common_driver(cmdopt):
    global driver
    base_driver = BaseDriver(eval(cmdopt))
    driver = base_driver.base_driver()
    yield driver

Because pytestconfig is a shortcut to request.config, cmdopt can also be written

def cmdopt(request):
    return request.config.getoption("--cmdopt")

Multiprocess Running

Run, showing a screenshot of a multiprocess run

remaining problems

Multiprocess compatibility testing can also cause some problems:

  • How test reports better differentiate multiple devices
  • For models with different resolutions, robustness and stability of some operation methods should be guaranteed.If the A phone screen is large, make sure the button is visible on the screen. The B phone screen is small and needs to be slid several times to see the button, which requires robust enough to define the method.
  • Business logic issues.If parallel de-operation (calling the same interface), will there be any business logic limitations, such as grabbing a voucher-free ticket, the same ip a day, and the same device can only grab one, at which point there should only be one success and the other will undoubtedly fail.This requires either adjusting restrictions or methods

Tags: Python Mobile Android Windows

Posted on Mon, 11 May 2020 21:59:27 -0700 by bmcewan