Pytest
Warning
This page is under development.
-
Running tests locally
Installation of testing dependencies and running the implemented tests.
-
Testing framework
Explanation of the current testing framework and addition of test cases.
Running tests locally¶
Tests can run locally using pytest. Ensure that the package and the testing requirements listed in the pyproject.toml test section are installed in python environment. In order to install PVGIS with the testing dependencies a user can run:
Tip
Find more details on PVGIS installation here.
Testing Framework¶
About¶
PVGIS is currenly supported by Pydantic dynamic generated objects as a return type
of all functions. These objects share some specific attributes such as .value that contains always array like information and .unit, and some case specific metadata attributes. Based on this common design a generic class for testing is implememented in the main conftest.py containing generic tests including testing of object type, data dtype, unit, shape of the array like information and value in multiple tolerance levels. Bellow the content of the generic class is presented:
import pytest
from numpy import isclose
TOLERANCE_LEVELS = [1, 1e-1, 1e-2]
class ValidateDataModel:
"""A test class for native PVGIS data models
This class provides a set of static methods and pytest fixtures
to validate the structure and attributes of native PVGIS data models.
It is designed to check the consistency between an actual object (`in_`)
and an expected object (`expected`) by comparing their `type`, `value`,
`unit`, data type (`dtype`), and `shape`.
PVGIS' data models are generated dynamically based on Pydantic's BaseModel.
A generic layout for PVGIS' native data models comprises the following
fields, as per Pydantic's terminology :
- value
- unit
- symbol
- description
and then other object-specific fields such as for example :
- min_radians
- max_radians
for angular measurements
See Also
--------
calculated: Input parameters
expected: The expected output
tolerance: Refers to NumPy's `isclose() function where `atol` (float) is
the absolute tolerance parameter (see Notes). See also `numpy.isclose()`.
function: The function
Notes
-----
This class draws inspiration from the Pyxu project [0]_.
References
----------
.. [0] @software{pyxu-framework,
author = {Matthieu Simeoni and
Sepand Kashani and
Joan Rué-Queralt and
Pyxu Developers},
title = {pyxu-org/pyxu: pyxu},
publisher = {Zenodo},
doi = {10.5281/zenodo.4486431},
url = {https://doi.org/10.5281/zenodo.4486431}
}
Examples
--------
Methods
-------
_check_type(calculated, expected):
Assert that the types of the input and expected objects are the same.
_check_value(calculated, expected, tolerance: float):
Assert that the values of the input and expected objects are close within a given tolerance level.
_check_unit(calculated, expected):
Assert that the units of the input and expected objects are identical.
_check_dtype(calculated, expected):
Assert that the data types of the input and expected object values are the same.
_check_shape(calculated, expected):
Assert that the shapes of the input and expected object values match.
Fixtures
--------
calculated(function, cases):
Returns an actual object created from an function and the first case from the provided test cases.
expected(cases):
Returns the expected object based on the second case in the provided test cases.
Test Methods
-------------
test_type(calculated, expected):
Tests that the input and expected objects are of the same type.
test_value(calculated, expected, tolerance: float):
Tests that the input and expected object values are close within the specified tolerance.
test_unit(calculated, expected):
Tests that the units of the input and expected objects match.
test_dtype(calculated, expected):
Tests that the data types of the input and expected object values match.
test_shape(calculated, expected):
Tests that the shapes of the input and expected object values are identical.
"""
@staticmethod
def _check_type(calculated, expected):
assert type(calculated) == type(expected)
@staticmethod
def _check_value(calculated, expected, tolerance:float):
assert isclose(calculated.value, expected.value, atol = tolerance).all()
@staticmethod
def _check_unit(calculated, expected):
assert calculated.unit == expected.unit
@staticmethod
def _check_dtype(calculated, expected):
assert calculated.value.dtype == expected.value.dtype
@staticmethod
def _check_shape(calculated, expected):
assert calculated.value.shape == expected.value.shape
@pytest.fixture
def calculated(self, function, cases):
return function(**cases[0])
@pytest.fixture
def expected(self, cases):
return cases[1]
def test_type(self, calculated, expected):
self._check_type(calculated, expected)
@pytest.mark.parametrize('tolerance', TOLERANCE_LEVELS)
def test_value(self, calculated, expected, tolerance:float):
self._check_value(calculated, expected, tolerance=tolerance)
def test_unit(self, calculated, expected,):
self._check_unit(calculated, expected)
def test_dtype(self, calculated, expected,):
self._check_dtype(calculated, expected)
def test_shape(self, calculated, expected):
self._check_shape(calculated, expected)
This generic class is then inherited by a test class specificically desinged to test a function of the software. For example for testing pvgis.algorithms.noaa.fractional_year.calculate_fractional_year_series_noaa function the test class should be like:
import pytest
from pvgis.algorithms.noaa.fractional_year import calculate_fractional_year_series_noaa
from .cases.fractional_year_noaa import cases_fractional_year_noaa
from .cases.fractional_year_noaa import cases_fractional_year_noaa_ids
from .cases.fractional_year_noaa import cases_fractional_year_noaa_invalid
from ..conftest import ValidateDataModel
class TestCalculateFractionalYearNOAA(ValidateDataModel):
@pytest.fixture(params=cases_fractional_year_noaa, ids=cases_fractional_year_noaa_ids)
def cases(self, request):
return request.param
@pytest.fixture
def function(self):
return calculate_fractional_year_series_noaa
@pytest.fixture(params=cases_fractional_year_noaa_invalid)
def calculated_invalid(self, request):
return request.param
def test_invalid_input_datetime_string(self, calculated_invalid, function):
with pytest.raises(calculated_invalid[1]):
_ = function(**calculated_invalid[0])
where class TestCalculateFractionalYearNOAA inherits all the methods from the generic class, thus all the predifined tests and contains as well the fixtures to read the cases from file .cases.fractional_year_noaa. Also contains an extra test for testing invalid input (test_invalid_input_datetime_string) and a fixture for reading those cases as well (in_invalid).
Generating cases for testing¶
For generating cases for testing mostly information from external sources is used and is converted to a python list containing multiple python dictonaries with the input of the function and the expected values.
For example using the excel Algorithms_NOAA.xlsx and the script in tests/generators/algorithms_noaa_test_generator.py the file tests/algorithms/noaa/cases/cases_fractional_year.py is generated. Then this file is used for reading the cases from the pytest fixtures.