Release checks
Before each release, NEEV runs every path in this guide end to end: it adds sources, sends real messages over MLLP and HTTP, opens them as each role, breaks a field on purpose, changes a mapping, and routes a test message through the orchestration. It also tries the mistakes people make, such as a wrong key, a port that is taken or a message that is not HL7, and checks that each one is refused with a clear reason.
This page is for the NEEV administrator who runs those checks on a staging environment.
What is checked
| Area | Steps | What it proves |
|---|---|---|
| Access | 4 | People outside the workspace, or without a session, see nothing. Each role sees only what it may. |
| Sources | 56 | MLLP, webhook, SFTP, FTP and database sources: add, test, switch on, receive, refuse bad input, switch off. MLLP over real TCP, webhooks over real HTTP. |
| Messages | 13 | Search and filters, plain-English detail, raw content for members only, send again, run again, export. |
| Logs | 4 | One message's timeline, plain labels, filters, refused sign-ins recorded. |
| Queue | 6 | Failed deliveries are queued, sent again or discarded, by the right roles. |
| Health | 2 | The dashboard and its time windows. |
| Field watch | 10 | A field stops arriving, the anomaly is found and alerted, then resolves when the field returns. |
| Alerts | 8 | Create, test, switch off and delete rules; invalid rules refused. |
| Mapping | 15 | Edit a field, test, promote, apply to the node, see it in the output, roll back, restore a snapshot. |
| Orchestration | 16 | Save, refuse stale and broken saves, test destinations, run, refuse real destinations, saving loads the route into the gateway. |
| Ask Neev | 9, or 10 with --with-llm | Questions answered without changes, edge cases refused, only admins can save or load the route, a route built from one sentence. |
| Profiles | 6 | Baselines are read-only, new versions are pinned, only admins create. |
Every check runs in its own throwaway workspace with an admin, a member and a viewer, plus a second workspace to prove one workspace never sees another's data. Everything it creates is removed at the end. Keys and addresses are never printed.
Run the checks
The checks need the backend, the MLLP listener service and the source test kit running on the same environment.
python manage.py neev_integration_check
Each step prints PASS, FAIL or SKIP with a short reason, then a
summary such as 147 passed, 0 failed, 2 skipped (or
149 passed, 0 failed, 1 skipped with --with-llm). The exit code is the
number of failures, so a pipeline can stop on anything other than 0.
| Option | Use it to |
|---|---|
--only sources,messages | Run some areas only: access, sources, messages, logs, queue, health, field_watch, alerts, mapping, orchestration, assistant, profiles. |
--with-llm | Also run the Ask Neev requests that call the language model. |
--keep | Keep the throwaway workspace afterwards, to look at what it did. |
--base-url | Point at a backend that is not on http://localhost:8000. |
--cleanup | Remove what an interrupted run left behind, then stop. |
A step is skipped, not failed, when the environment cannot run it, for
example when the test database accepts any password, or when --with-llm
is not given. The reason is printed next to it.
The checks create and delete workspaces, users and sources. Run them on a staging or local environment only.