📊Visualize Test Results with Allure
How to record and visualize your test results with Allure Reports
Last updated
How to record and visualize your test results with Allure Reports
Last updated
Pylenium uses pytest as the Test Framework and uses pytest-allure for reporting. However, you can use your preferred reporting strategy since it's easy to extend the framework
Visit Allure's websites for more info and details:
Official website: https://qameta.io/allure-report/
Official docs: https://docs.qameta.io/allure-report/
Pylenium comes with Allure natively integrated, so all you have to do is add the --alluredir argument to the pytest command. Besides that, continue using Pylenium and pytest normally 😄
Visit their installation docs and install allure using the appropriate method. For example, on Macs, you can use homebrew:
Pylenium also includes a CLI command to try and do the installation for you:
It's recommended to visit Allure's docs and install it from there. For example, the Linux instructions require the use of sudo and you want to make sure you trust it first before executing 👍🏽
⬆️ That creates a folder called ./allure-report and stores the test results there
Then you generate the results into an Allure Report and "serve" it:
It's recommended to use the allure command directly instead of through Pylenium:
By default, allure will open a new window or browser tab in your default browser and show you something like this:
Take some time to review all of the data that you get from this report, but some notable mentions:
pass/fail percentages
trends in historical test results
Yes, the more you run tests and save them to the same folder, you'll get more data in the report!
detailed steps on setup and teardown of each test
logs automatically included for each test
on test failure, a screenshot is automatically taken and attached to the test
test case durations and test run durations
other visuals in the Graphs tab
and more!
If you haven't checked out their official docs yet, you should because they go over what you can send, how you send it, and how to use the Allure Report once it's served!
This guide will cover more common scenarios
By default, Pylenium already takes a screenshot and attaches it to allure on any test failure so you can see the last "view" of the page before the test failed. However, you can add more screenshots.
For example, if you want to take a screenshot after every page transition, this is what the scripted version of the test would look like:
Run the test
Serve the report
And observe that we have both screenshots in the Test Body section of our test report:
allure.attach() is all you need, meaning that you can add it to your Page Objects, flow functions, or even as a reusable decorator!
allure knows the current test that is running and attaches the screenshot appropriately. You don't have to worry about "assigning" it to the right test.
pytest tags (aka marks) tests using its mark feature and allure simply leverages it. Take the following example:
We can run all tests in the file
Or we can run a subset of tests given their tag or mark using the -m flag
Either way, we now see the tag(s) on the test in our report
Also notice how the docstring in the test function has been converted to a Description in our report! 👀
In pytest, you can use the mark feature to tag tests. However, in Allure, they have a unique way to categorize tests based on each test's results.
To integrate the Allure Report with a bug tracker or test management system, Allure has @allure.link, @allure.issue and @allure.testcase descriptors.
The severity mark is part of Allure, not pytest. For example:
This is another way to mark and filter tests!
