Rule Review Workflow Guide

1. Getting Started

What is Rule Review?

The Rule Review workflow allows you to examine, validate, and refine policy rules that have been automatically extracted from documents. During review, you can approve accurate rules, reject incorrect ones, flag ambiguous cases, edit rule content, and add contextual annotations.

Accessing the Review Page

After uploading a document and completing the extraction process, navigate to the results page and click the "Review Rules" button. This opens the interactive review interface for that extraction job.

Understanding Review Statuses

Each rule has a review status that tracks its progress through the review workflow:

• Pending: Rule has not been reviewed yet.
• Approved: Rule has been validated and confirmed as correct.
• Rejected: Rule is incorrect or not applicable.
• Flagged: Rule needs expert review due to complexity or uncertainty.
• Needs Edit: Rule has been modified during review (automatically assigned after editing).
• Skipped: Rule was skipped during review and may need attention later.

2. Reviewing Rules

Viewing Rules

Rules are displayed as cards in the main content area. Each card shows:

• Rule name: A unique identifier or descriptive name for the rule
• Summary: A brief human-readable explanation of what the rule does
• Status badge: Current review status
• Rule type: Category such as Eligibility, Calculation, Validation, or Workflow
• Confidence score: AI-generated confidence level (high, medium, low)

Opening Rule Details

Click on any rule card to open the detailed view panel. The detail panel displays:

• Full rule content: The complete Rego policy code
• Source mapping: References to the original document sections that informed this rule
• Edit history: If the rule has been modified, view all previous versions and edit reasons
• Annotations: Any contextual notes, warnings, or references added during review

Understanding Source Mapping

The right panel in the rule detail view shows source mappings. Each mapping includes:

• Source text: Excerpts from the original document
• Document location: Page and section references
• Relevance: How strongly this source text relates to the rule

Use source mappings to verify that the rule accurately reflects the policy intent expressed in the document.

Making Review Decisions

In the rule detail panel, you can take the following actions:

• Approve: Click the "Approve" button (or press a) to mark the rule as correct and complete.
• Reject: Click the "Reject" button (or press r) to mark the rule as incorrect or not applicable. You'll be prompted to provide a reason.
• Flag for Review: Click the "Flag" button (or press f) to mark the rule for expert attention. Use this for complex or ambiguous cases.

3. Editing Rules

Opening the Edit Modal

In the rule detail view, click the "Edit Rule" button (or press e) to open the edit modal. This allows you to modify the rule content and summary.

Making Changes

1. Edit rule content: Modify the Rego policy code in the text area. The editor preserves formatting and syntax.
2. Update summary: Revise the human-readable summary if needed to reflect your changes.
3. Provide edit reason: This field is required. Explain why you made the changes (e.g., "Fixed incorrect threshold value" or "Clarified condition logic").
4. Save changes: Click "Save Changes" to apply your edits.

Version History

After editing a rule, all previous versions are preserved. In the rule detail view, expand the "Edit History" section to see:

• All previous versions of the rule content
• Timestamps of when each edit was made
• Edit reasons provided by reviewers

4. Adding Annotations

What Are Annotations?

Annotations allow you to add contextual information, warnings, and references to rules without modifying the rule content itself. Annotations are visible to other reviewers and can help document policy intent, edge cases, and important considerations.

Annotation Types

• Clarification: Explain ambiguous aspects of the rule or policy
• Policy Intent: Document the underlying purpose or goal of the rule
• Edge Case: Note scenarios or conditions that may require special handling
• Warning: Flag potential issues, conflicts, or risks
• Reference: Link to external documents, regulations, or related rules
• Other: Any other contextual information that doesn't fit the above categories

Adding an Annotation

1. Open the rule detail view
2. Click the "Add Annotation" button
3. Select the annotation type from the dropdown
4. Enter your annotation text (up to 1000 characters)
5. Click "Save Annotation"

5. Bulk Operations

Selecting Multiple Rules

You can select multiple rules for batch operations:

• Click the checkbox on individual rule cards to select them
• Use the "Select All" checkbox to select all visible rules matching current filters

When you select one or more rules, a bulk actions toolbar appears at the top of the rules list.

Bulk Actions

The bulk actions toolbar provides the following operations:

• Approve Selected: Mark all selected rules as approved
• Reject Selected: Mark all selected rules as rejected (you'll be prompted for a reason)
• Flag for Review: Flag all selected rules for expert attention

Filtering Before Bulk Operations

Use the filter controls to narrow down the rules before selecting:

• Search: Filter by rule name or summary keywords
• Status filter: Show only rules with a specific review status
• Rule type filter: Show only rules of a certain type (Eligibility, Calculation, etc.)

Filtering before bulk selection helps you target specific sets of rules, such as "all pending eligibility rules".

6. Re-extraction

When to Use Re-extraction

Re-extraction is useful when:

• Multiple rules were rejected or flagged due to systematic issues
• You've edited many rules and want to see if the AI can improve them based on your feedback
• The initial extraction missed important rules or made consistent errors

How Re-extraction Works

When you trigger re-extraction, the system:

1. Collects feedback from all rejected, edited, and flagged rules
2. Generates a feedback summary describing patterns and issues
3. Re-runs the extraction pipeline with this feedback to improve results
4. By default, preserves approved rules and only re-extracts problematic ones

Starting Re-extraction

1. Click the "Re-extract with Feedback" button at the top of the review page
2. Review the auto-generated feedback summary
3. Optionally provide additional context or instructions
4. Choose whether to preserve approved rules (recommended)
5. Optionally retry only flagged rules instead of all problematic rules
6. Click "Start Re-extraction" to begin

7. Keyboard Shortcuts

The review interface supports keyboard shortcuts to speed up your workflow. Press ? at any time to view the shortcuts modal.