9.10. How To: Test in EVerest¶
This is a tutorial on how to setup your test environment for EVerest.
9.10.1. Requirements¶
A successful build of everest-core is required. Refer to everest-core for this.
Some test cases require the installation of everestpy. You can do this after a successful build of everest-core:
cd everest-core
cmake --build build --target everestpy_pip_install_dist
Sometimes the everstpy module might require manual installation.
That can be done by running:
cd everest-framework/everestpy
python3 -m pip install .
Install the python package containing the EVerest test utilities from everest-testing.
Install the requirements of this repository:
python3 -m pip install -r requirements.txt
Some test cases require the installation of Josev. This should be located in your EVerest workspace:
cd <path-to-josev>
python3 -m pip install -r requirements.txt
python3 -m pip install .
For most test cases you need a correct setup of certificates and configs within EVerest. You can use
cd test_sets/ocpp_tests/everest_aux
./install_certs.sh <path/to/everest-core>
./install_configs.sh <path/to/everest-core>
to install the certificates within everest-aux into the correct location of everest-core.
An MQTT server is needed for testing. Docker can be used for this. Refer to docker-setup for this.
9.10.2. Test sets¶
Run any test set (e.g. ocpp_compliance_tests.py) with:
python3 -m pytest test_sets/ocpp_tests/ocpp16/ocpp_compliance_tests.py --everest-prefix <path-to-everest-core>/build/dist --libocpp <path-to-libocpp>
or run a single test case with:
python3 -m pytest test_sets/ocpp_tests/ocpp16/ocpp_compliance_tests.py --everest-prefix <path-to-everest-core>/build/dist/ --libocpp <path-to-libocpp> -k 'test_remote_start_first' -s
or run all tests in parallel with:
./run-testing.sh
If running from the everest-core/tests/ocpp_tests directory, the path to
everest-core or libocpp can be relative, for example:
--everest-prefix ../../../everest-core/build/dist --libocpp ../../../libocpp/
Note
Here is a TODO: We have to update the documentation for known failing tests.
9.10.3. View OCPP test logs¶
While running the tests, EVerest logs OCPP message to its log directory. These logs are stored in HTML files.
When you open the following directory in your web browser you can view to logs:
/tmp/everest_ocpp_test_logs/
9.10.4. VS Code Debugging¶
Debugging can have various layers depending on the feature tested. This part will focus on debugging inside VS Code.
9.10.4.1. Python debugging¶
Tests can be manually launched by adding the proper entries to the
‘launch.json’ file.
Example for debugging a test from ocpp_compliance_tests.py:
"configurations": [
{
"name": "Python: OCPP Compliance Test",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [
"test_sets/ocpp_tests/ocpp16/ocpp_compliance_tests.py",
"--libocpp", "../libocpp/",
"--everest-prefix", "../everest-core/build/dist",
"-s", "-vv", "-k", "your_test_here",
],
"cwd": "${workspaceFolder}/ocpp-testing/",
"console": "integratedTerminal",
"justMyCode":false
}
]
The paths can differ based on the workspace setup.
9.10.4.2. C++ debugging¶
When a certain test case executes, there is a chance that C++ code can be faulty, requiring a GDB attach in order to detect an issue. In that case, the following steps can be followed:
setup for python debugging
setup for c++ debugging with the following config entry:
"configurations": [
{
"name": "(gdb) Attach PID",
"type": "cppdbg",
"request": "attach",
"program": "${workspaceFolder}/everest-core/build/dist/bin/manager",
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
},
{
"description": "Set Disassembly Flavor to Intel",
"text": "-gdb-set disassembly-flavor intel",
"ignoreFailures": true
}
]
}
]
build everest-core with debugging enabled:
cmake .. -DCMAKE_BUILD_TYPE=Debugrun the desired test from python debugging (
`Python: OCPP Compliance Test`) and place a breakpoint near the test’s entry pointcopy the PID from the variable ‘test_controller: TestController’ from the test’s function:
test_controller->_everest_core->process->pidrun
`pstree -pT ${pid}`with the retrieved pid for example 102590:
pstree -pT 102590
manager(102590)─┬─auth:Auth(102653)
├─car_simulator:J(102654)
├─connector_1:Evs(102655)
├─controller(102595)
├─energy_manager:(102656)
├─evse_security:E(102657)
├─grid_connection(102658)
├─ocpp:OCPP(102660)
├─python3(102659)
├─slac:JsSlacSimu(102661)
├─system:System(102662)
├─token_provider_(102663)
└─yeti_driver:JsY(102664)
while the test is in the breakpoint, run the (gdb) Attach PID configuration with the desired module to attach to
in the VS code terminal prompt input ‘y’ and insert the root password
unpause the python Python: OCPP Compliance Test debug session
9.10.5. External integration¶
The main motive for external integration is the flexibility of running EVerest outside of the SIL environment. Therefore, the ocpp-tests can be run with any out-of-tree versions of EVerest.
Currently, there are different versions of EVerest that should be able to run ocpp-tests:
SIL
BaseCamp
The test_sets folder can be embedded in any external repository that uses
EVerest.
The external EVerest does not have to include all the components required by
the SIL version.
Due to the limitations of pytest, the conftest.py file must not be included, in order to preserve the needs of the external project.
In order to use the proper fixtures for the [test_sets](test_sets) inside an
external repository, a custom conftest.py specific for that project has to
provide all the necessary fixtures for running the tests.
9.10.5.1. Example fixture¶
The test_F01_F02_F03` requires the following fixtures:
async def test_F01_F02_F03(charge_point_v201: ChargePoint201, test_controller: TestController, test_utility: TestUtility):
The project-specific conftest.py must provide the proper fixtures that are custom for the project - in our case the test_controller fixture:
# Add necessary handling in here
class ExternalTestControllerAdapter(TestController):
def __init__(self):
pass
def start_thread(self):
pass
def stop_thread(self):
pass
def start(self):
pass
def stop(self):
pass
def plug_in(self, connector_id=1):
pass
def plug_in_ac_iso(self, payment_type, connector_id):
raise NotImplementedError()
def plug_out(self, connector_id=1):
pass
async def swipe_async(self, token):
pass
@pytest.fixture
def test_controller(everest_core: EverestCore):
controller = ExternalTestControllerAdapter(
everest_core
)
controller.start()
yield controller
controller.stop()
9.10.5.2. Example marker injection¶
Sometimes the tests do not have all the required
pytest markers.
The test test_F01_F02_F03 might require additional markers in the context
of an external repo.
That can be achieved at runtime using pytest hooks.
For example if the test might require the
pytest.mark.use_temporary_persistent_store
pytest.mark.ocpp_config(Path("path-to-config"))
markers that are not present in the test header. They can be injected by adding the following code to the conftest.py of the specific external project:
def pytest_collection_modifyitems(session, config, items):
marks = (
pytest.mark.use_temporary_persistent_store,
pytest.mark.ocpp_config(Path("path-to-config"))
)
for item in items:
if "ocpp_testing" in item.path.as_posix():
for marker in marks:
item.add_marker(marker)
pass
The result is that before running each selected test under test_sets, the
markers will be applied, modifying the default behavior of the tests.
9.10.6. Required mocks¶
In order to properly run the tests, certain mocks have to be implemented. Different versions of EVerest might require different mock implementations, that might include but not be limited to:
charge_point_v16
charge_point_v201
test_controller
ocpp_test_mocks
test_utility
Get Support
YouTube Channel
Twitter