Advanced Verify AI Tests

Standard Verify AI tests — the Diff check and Description check — are a powerful first line of defense for validating your evidence attachments in real time. But for organizations that want even deeper automated validation, Advanced Tests take things further by testing your evidence against other related evidence in your compliance program.

With Advanced Tests, Strike Graph's Security Assistant analyzes the relationships between your evidence items and recommends custom test definitions that run automatically each time a new attachment is added. This is especially useful for catching consistency issues across evidence that should align — like confirming that your access review covers the same systems described in your access control policy, or that your vulnerability scan results are consistent with the scope defined in your risk assessment.

How Advanced Tests work

Advanced Tests are built on two components: a generation step and an execution step.

When you ask Strike Graph to recommend advanced tests for an evidence item, the Security Assistant analyzes that evidence alongside its related evidence. Security Assistant looks at items that share controls as well as other evidence with similar/related coverage of compliance program.

Based on that analysis, it generates a set of recommended test definitions, each describing a specific check to run between the source evidence and a related (target) evidence item.

Each test definition includes:

• A test category — the type of check being performed (e.g., consistency, coverage, alignment)

• A test description — a plain-language explanation of what the test is checking

• Claims to confirm — a list of specific assertions that Strike Graph will attempt to verify when the test runs

Once you've reviewed and accepted a test definition, you can run it against new attachments as they are added to the evidence, just like the standard Diff and Description checks.

Please note: Currently, while in beta, Advanced Verify AI tests must be run manually from the Evidence Preview modal.

Prerequisites

Advanced Tests require:

• Verify AI to be enabled on the evidence item

• The evidence item to have at least one related evidence identified by Strike Graph's Security Assistant

• Access to the Advanced Tests feature (check with your Customer Success Manager if you don't see this option)

Generating test recommendations

To get started, navigate to the evidence item where you want to set up advanced tests. In the Verify AI card under the evidence description, look for the Security Assistant: Cross-Evidence Tests section.

If Strike Graph has identified related evidence for this item, you'll see a "Recommend advanced tests" button. Click it to kick off the generation process.

Note: If you see "Pending related evidence analysis" instead of the button, Strike Graph is still completing its analysis of related evidence for this item. Check back in a few minutes.

If no related evidence is found, or if generation fails for any reason, you'll see an error message and can try again.

Reviewing and accepting test recommendations

The Advanced Tests modal displays all of the test definitions Strike Graph has generated, grouped by the related (target) evidence they reference.

Target evidence is the related evidence item that Verify AI will test your attachments from your current evidence against.

For each target evidence item, you'll see its name, description, and owner, along with one or more recommended test definitions. Each definition shows the test category, a description of what will be checked, and a list of specific claims to confirm.

For each test definition, you have two options:

• Add to Verify AI — Accepts the test definition and adds it to the active test suite for this evidence. Once accepted, the test can be run against new attachments. You can un-accept a test at any time by clicking "Added to Verify AI" to toggle it back to suggested.

• Mark as reviewed — Dismisses the suggestion without adding it to the test suite. This is useful for tests that aren't relevant to your compliance program. You can un-dismiss a suggestion the same way.

There's no requirement to accept all suggestions. Add the ones that are meaningful for your compliance goals and skip the rest.

Running and viewing advanced test results

Once you've accepted one or more test definitions, they'll appear alongside the standard Diff and Description checks in the Verify AI card on the evidence detail page. Accepted tests are labeled with the Security Assistant icon and display their test category.

To manually run Advanced tests:

• Click on the attachment you'd like to run the tests against to open the Evidence Preview modal

• Click on the tab that says "Tests"

• Select which of the added Advanced Tests you'd like to run

• Find the 'Run' button to initiate the test

• Once the test completes, you'll see the results in this same modal

Like standard tests, advanced tests return one of the following statuses:

• Passed (green) — the test's claims were confirmed against the current attachments

• Needs attention (red) — one or more claims could not be confirmed; the evidence owner will receive an email alert

• Skipped (gray) — the test could not be processed, usually due to a missing or unreadable attachment on either the source or target evidence

The test count in the Verify AI header (e.g., "3/4 TESTS COMPLETED") will now include both standard and advanced tests.

Tips and considerations

Both evidences need attachments. Advanced tests compare the source evidence attachment against the most recent attachment of the related (target) evidence. If either evidence is missing a current attachment, the test will be skipped.

Regenerating recommendations replaces previous ones. If you click "Recommend advanced tests" again, Strike Graph will delete all previous test definitions — including any you've accepted — and start fresh. Re-generate only if you want a completely new set of recommendations.

Test descriptions are fixed at generation time. The test definitions Strike Graph generates are based on a snapshot of your evidence at generation time. If your evidence or its related items change significantly, you may want to regenerate recommendations to get updated suggestions.

Advanced tests run on the most recent attachment. Like standard Verify AI tests, advanced tests always evaluate the most recently added attachment, not historical ones.