22-allure特性 丨总览中的Environment和Categories设置 1 Environment设置 2 Categories设置 3 关于Flaky test 3.1 Flaky test介绍 3.2 产生Flaky Tests的原因 3.3 Flaky安装 3.4 Flaky使用 3.5 小结
如下图，我们可以看到allure报告的总览，里边的一些特性是可以自定义设置的。 Environment可以理解为环境变量；默认为空； 可以自己设置。 在存放测试报目录下创建environment.properties或者environment.xml文件； 而测试报告目录是使用--alluredir指定的目录，比如： - - alluredir allure- results
pytest - n auto - - alluredir= allure- results test_xdist. py
Browser= Chrome
Browser. Version= 111.0 .5563 .65
Env= Test
IP= 192.168 .1 .133
Allure- Pytest. Version= 2.8 .12
运行后查看Environment： 或者创建environment.xml： environment> < parameter> < key> Browser< / key> < value> Chrome< / value> < / parameter> < parameter> < key> Browser. Version< / key> < value> 111.0 .5563 .65 < / value> < / parameter> < parameter> < key> Env< / key> < value> Test< / value> < / parameter> < parameter> < key> IP< / key> < value> 192.168 .1 .133 < / value> < / parameter> < parameter> < key> Allure- Pytest. Version< / key> < value> 2.8 .12 < / value> < / parameter>
< / environment>
Categories即分类，测试用例结果的分类；默认有两种分类：
和environment方式一样，在allure-results目录中创建categories.json文件
[ { "name" : "Ignored tests" , "matchedStatuses" : [ "skipped" ] } , { "name" : "Infrastructure problems" , "matchedStatuses" : [ "broken" , "failed" ] , "messageRegex" : ".*bye-bye.*" } , { "name" : "Outdated tests" , "matchedStatuses" : [ "broken" ] , "traceRegex" : ".*FileNotFoundException.*" } , { "name" : "Product defects" , "matchedStatuses" : [ "failed" ] } , { "name" : "Test defects" , "matchedStatuses" : [ "broken" ] }
]
name：分类名称
matchedStatuses：测试用例的运行状态，默认[ "failed" , "broken" , "passed" , "skipped" , "unknown" ]
messageRegex：测试用例运行的错误信息，默认. * ，通过正则匹配
traceRegex：测试用例运行的错误堆栈信息，默认. * ，通过正则匹配
import pytest
import timeclass TestCase01 ( ) : def test_case_01 ( self) : time. sleep( 1 ) print ( "case01$$$$$$$$$$$$$$$$$$$$$" ) assert 1 == 2 def test_case_02 ( self) : time. sleep( 1 ) print ( "case02$$$$$$$$$$$$$$$$$$$$$" ) assert 3 == 3 def test_case_03 ( self) : time. sleep( 1 ) print ( "case03$$$$$$$$$$$$$$$$$$$$$" ) assert "is" in "is_you" def test_case_04 ( self) : time. sleep( 1 ) print ( "case04$$$$$$$$$$$$$$$$$$$$$" ) assert 5 < 10 def test_case_05 ( self) : time. sleep( 1 ) print ( "case05$$$$$$$$$$$$$$$$$$$$$" ) assert 222 == 333 def test_case_06 ( self) : time. sleep( 1 ) print ( "case06$$$$$$$$$$$$$$$$$$$$$" ) assert 444 > 666 class TestCase02 ( ) : def test_case_07 ( self) : time. sleep( 1 ) print ( "case07$$$$$$$$$$$$$$$$$$$$$" ) assert 10 / 2 == 5.0 def test_case_08 ( self) : time. sleep( 1 ) print ( "case08$$$$$$$$$$$$$$$$$$$$$" ) assert "num" in "num_list" def test_case_09 ( self) : time. sleep( 1 ) print ( "case08$$$$$$$$$$$$$$$$$$$$$" ) assert "num1" in "num_list" if __name__ == '__main__' : pytest. main( [ "-s" , "test_yyy.py" ] ) pytest - n auto - - alluredir= allure- results test_yyy. pyallure serve allure- results
查看结果： Flaky test在被测对象和测试条件都不变的情况下，有时候失败、有时候成功的测试；实际上就是不稳定的测试，或者随机失败(随机成功)的测试； 标记成Flaky是为了当用例失败的情况下，我们能获取足够详细的信息。 pip3 install pytest- ignore- flaky
C: \Users\Administrator> pip3 install pytest- ignore- flaky
Looking in indexes: https: // pypi. tuna. tsinghua. edu. cn/ simple
Collecting pytest- ignore- flakyDownloading https: // pypi. tuna. tsinghua. edu. cn/ packages/ 22 / bf/ 4a670d28c8c37569e26536c068d83b37a01aea9fff9a45a03ae3be5344b9/ pytest_ignore_flaky- 2.0 .0 - py3- none- any . whl ( 3.9 kB)
Requirement already satisfied: pytest>= 6.0 in d: \python37\lib\site- packages ( from pytest- ignore- flaky) ( 6.2 .4 )
Requirement already satisfied: py>= 1.8 .2 in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 1.10 .0 )
Requirement already satisfied: colorama in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 0.4 .4 )
Requirement already satisfied: attrs>= 19.2 .0 in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 20.3 .0 )
Requirement already satisfied: atomicwrites>= 1.0 in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 1.4 .0 )
Requirement already satisfied: iniconfig in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 1.1 .1 )
Requirement already satisfied: importlib- metadata>= 0.12 in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 2.1 .1 )
Requirement already satisfied: toml in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 0.10 .2 )
Requirement already satisfied: packaging in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 20.8 )
Requirement already satisfied: pluggy< 1.0 . 0a1, >= 0.12 in d: \python37\lib\site- packages ( from pytest>= 6.0 - > pytest- ignore- flaky) ( 0.13 .1 )
Requirement already satisfied: zipp>= 0.5 in d: \python37\lib\site- packages ( from importlib- metadata>= 0.12 - > pytest>= 6.0 - > pytest- ignore- flaky) ( 1.2 .0 )
Requirement already satisfied: pyparsing>= 2.0 .2 in d: \python37\lib\site- packages ( from packaging- > pytest>= 6.0 - > pytest- ignore- flaky) ( 2.4 .7 )
Installing collected packages: pytest- ignore- flaky
Successfully installed pytest- ignore- flaky- 2.0 .0
import pytest
import timeclass TestCase01 ( ) : def test_case_01 ( self) : time. sleep( 1 ) print ( "case01$$$$$$$$$$$$$$$$$$$$$" ) assert 1 == 2 def test_case_02 ( self) : time. sleep( 1 ) print ( "case02$$$$$$$$$$$$$$$$$$$$$" ) assert 3 == 3 def test_case_03 ( self) : time. sleep( 1 ) print ( "case03$$$$$$$$$$$$$$$$$$$$$" ) assert "is" in "is_you" def test_case_04 ( self) : time. sleep( 1 ) print ( "case04$$$$$$$$$$$$$$$$$$$$$" ) assert 5 < 10 def test_case_05 ( self) : time. sleep( 1 ) print ( "case05$$$$$$$$$$$$$$$$$$$$$" ) assert 222 == 333 def test_case_06 ( self) : time. sleep( 1 ) print ( "case06$$$$$$$$$$$$$$$$$$$$$" ) assert 444 > 666 class TestCase02 ( ) : def test_case_07 ( self) : time. sleep( 1 ) print ( "case07$$$$$$$$$$$$$$$$$$$$$" ) assert 10 / 2 == 5.0 def test_case_08 ( self) : time. sleep( 1 ) print ( "case08$$$$$$$$$$$$$$$$$$$$$" ) assert "num" in "num_list" @pytest. mark. flaky def test_case_09 ( self) : time. sleep( 1 ) print ( "case08$$$$$$$$$$$$$$$$$$$$$" ) assert "num1" in "num_list" if __name__ == '__main__' : pytest. main( [ "-s" , "test_yyy.py" ] ) pytest - n auto - - alluredir= allure- results test_yyy. py
== == == == == == == == == == == == == == == == == == == == == short test summary info == == == == == == == == == == == == == == == == == == == == == =
FAILED test_yyy. py: : TestCase01: : test_case_06 - assert 444 > 666
FAILED test_yyy. py: : TestCase01: : test_case_01 - assert 1 == 2
FAILED test_yyy. py: : TestCase01: : test_case_05 - assert 222 == 333
FAILED test_yyy. py: : TestCase02: : test_case_09 - AssertionError: assert 'num1' in 'num_list'
== == == == == == == == == == == == == == == == == == 4 failed, 5 passed, 1 rerun in 5 . 99s == == == == == == == == == == == == == == == == == ==
从上发现被我们使用@pytest.mark.flaky标记的用例，断言是失败的，也正常标准失败： 命令行加上代码：--ignore-flaky重新运行： pytest - n auto - - alluredir= allure- results test_yyy. py - - ignore- flaky
发现被标记的用例变成了xfailed而不是失败了： 默认情况下， @pytest.mark.flaky 装饰器标记的测试用例默认会执行； 当用例执行结果成功时正常执行正常显示用例结果； 当用例执行结果失败时，测试用例默认失败重跑一次。 pytest命令行参数 --ignore-flaky 运行 @pytest.mark.flaky 标记的测试用例：当用例执行成功时执行结果显示正常；当用例执行失败时执行结果显示XFAIL(skip flaky test failure)。 
