Skip to content

Latest commit

 

History

History
286 lines (235 loc) · 29.8 KB

File metadata and controls

286 lines (235 loc) · 29.8 KB

Development Conventions

Introduction

This document outlines the standard operating procedure for all AI agents contributing to this project. Adhering to these conventions ensures a systematic, verifiable, and robust development process.

Tool Usage

When you suggest commands in your regular response (NOT inside a SEARCH/REPLACE block), format them inside bash blocks:

cmd1
cmd2

These commands are then presented to the user, who can accept them for execution, and the results are returned to you.

Key tools available for you to suggest:

  • rg (ripgrep): Your primary tool for fast, powerful code searching. Use it to find definitions, locate files, or understand code relationships.
  • tree: List files to understand directory structures.
  • head: Inspect the beginning of files to quickly understand their structure and content.
  • ls: Check file sizes to determine if they should be read or inspected.
  • psql: Run arbitrary SQL for debugging or inspection (e.g., echo 'SELECT * FROM sql_saga.era;' | psql -d sql_saga_regress).

For file system operations and large-scale edits, prefer suggesting shell commands over generating SEARCH/REPLACE blocks where appropriate. This is faster and more efficient.

  • Use rm to delete files and git mv to move or rename them.
  • For simple content replacement (e.g., replacing an entire file's contents), echo "new content" > filename can be used instead of a large SEARCH/REPLACE block.
  • For large-scale, repetitive search-and-replace operations across multiple files, powerful tools like ruplacer and renamer are available and should be used.

Shell Command Rules

  • All shell commands must be single-line. The execution environment splits commands by newline.
  • Do not suggest multi-line echo commands. To write multi-line content to a file, echo a single-line string with \n characters to a temporary file first.

Coding Standards

C Code (PostgreSQL Extensions)

  • C99 Compliance: All C code must be compatible with the C99 standard. The build process uses the -Wdeclaration-after-statement flag, which will generate a warning if variable declarations are mixed with code.
    • Rule: Declare all variables at the beginning of a block (immediately after a {). Do not mix declarations and executable statements.
  • PostgreSQL Coding Conventions: Adhere to the formatting and naming conventions outlined in the official PostgreSQL Documentation. This includes conventions for variable names, function names, and code layout.

psql Conventions

  • You must never add an E before the echo string, as it outputs. So instead of echo E'...\n...' you must use echo '...\n...', the \n works regardless.

SQL Conventions

  • Function/Procedure Definitions:

    • Use the function/procedure name in the literal string quote for the body (e.g., AS $my_function_name$).
    • Specify LANGUAGE plpgsql (or other) before the body.
    • Use the long form for parameters for documentation clarity (e.g., param_name param_type).
    • Example:
      CREATE FUNCTION public.example(email text) RETURNS void LANGUAGE plpgsql AS $example$
      BEGIN
        -- Use function_name.parameter_name for clarity if needed, e.g., example.email
        SELECT * FROM some_table st WHERE st.email = example.email;
      END;
      $$;
  • Function Calls: For calls with 3+ arguments, use named arguments (e.g., arg1 => val1).

  • String Literals for format():

    • Always prefer dollar-quoting (e.g., format($$ ... $$)) for the main dynamic SQL string. This avoids having to escape single quotes inside the SQL.
    • Nesting: When nesting dollar-quoted strings (e.g., a dynamic SQL string that itself contains another format() call), use named dollar quotes for the outer string to avoid conflicts. The convention is to use a descriptive name like $SQL$ or $jsonb_expr$. This is especially common inside function bodies, which already use a named quote (e.g., $function_name$).
    • For format() calls with multiple parameters, especially if parameters repeat, use numbered placeholders for clarity:
      • %1$I for the 1st parameter as an identifier, %2$L for the 2nd as a literal, %3$s for the 3rd as a plain string, etc.
      • Example: format($$Testing %3$s, %2$s, %1$s$$, 'one' /* %1 */, 'two' /* %2 */, 'three' /* %3 */);
    • Prefer parameter binding with EXECUTE ... USING for large arrays or values rather than interpolating with %L where possible.

    Good

    -- Dollar-quoted format() string; identifier and literal are numbered; batch array is passed via USING.
    EXECUTE format($$
        UPDATE public.%1$I AS dt SET
            last_completed_priority = %2$L
        WHERE dt.row_id = ANY($1) AND dt.action = 'skip'
    $$, v_data_table_name /* %1$I */, v_step.priority /* %2$L */)
    USING p_batch_row_ids;

    Also good (embedding quotes safely without doubling them)

    EXECUTE format($$
        UPDATE public.%1$I AS dt SET
            error = COALESCE(dt.error, '{}'::jsonb) ||
                    jsonb_build_object('status_code', 'Provided status_code not found and no default available'),
            state = 'error'
        WHERE dt.row_id = ANY($1) AND dt.status_code IS NOT NULL
    $$, v_data_table_name)
    USING p_batch_row_ids;

    Also good (nested dollar-quoting)

    -- The outer string uses a named quote ($jsonb_expr$) to avoid conflicting with the inner $$
    v_error_sql := $jsonb_expr$
        jsonb_build_object('latitude', CASE ... THEN format($$Value %1$s out of range$$, lat_val) ... END)
    $jsonb_expr$;

    Avoid

    -- Hard to read and easy to break: manual '' escaping inside single-quoted format string
    EXECUTE format('UPDATE public.%s SET note = ''it'''''s broken'' WHERE id = %s', tbl, id);

    Notes

    • Use %I for identifiers, %L for SQL literals, and %s for raw string insertion.
    • Keep the SQL readable by aligning numbered placeholders with inline comments that show which parameter they refer to.
  • Table Aliases: Prefer explicit AS for table aliases, e.g., FROM my_table AS mt. For common data table aliases in import procedures, AS dt is preferred.

  • Temporal Logic: When writing conditions involving time, always order the components chronologically for readability (e.g., start <= point AND point < end). Avoid non-chronological forms like point >= start.

  • Temporary Table Management:

    • To ensure procedures are idempotent and to avoid noisy NOTICE messages in logs, use the following pattern to clean up temporary tables at the beginning of a procedure:
      -- The explicit 'pg_temp.' schema ensures we only check for session-local tables.
      IF to_regclass('pg_temp.my_temp_table') IS NOT NULL THEN DROP TABLE my_temp_table; END IF;
      CREATE TEMP TABLE my_temp_table (...) ON COMMIT DROP;
    • This pattern has several advantages:
      1. Silent Operation: It avoids the NOTICE: table "..." does not exist, skipping message that DROP TABLE IF EXISTS would generate on the first run.
      2. Co-location: It keeps the cleanup logic directly beside the creation logic, improving readability.
      3. Debuggability: If the code does not behave as expected, then a test running in the same transaction can inspect those temporary tables to determine where the faulty logic lies.

Build System

  • Makefile and Source Files: The Makefile uses a glob pattern (src/[0-9][0-9]_*.sql) to automatically discover and concatenate SQL source files. When adding a new feature, simply create a new numbered .sql file in the src/ directory (e.g., src/31_new_feature.sql). It will be included in the build automatically without any need to edit the Makefile.

SQL Naming conventions

  • x_id is a foreign key to table x
  • x_ident is an external identifier, not originating from the database
  • x_at is a TIMESTAMPTZ (with timezone)
  • x_on is a DATE
  • Temporal Columns: To ensure consistency and intuitive use, all temporal periods must follow the [) semantic (inclusive start, exclusive end). This aligns with the default behavior of PostgreSQL's native range types (e.g., daterange) and ensures compatibility with operators like OVERLAPS.
    • Column names for the start of a period must be named valid_from (or a similarly descriptive name ending in _from).
    • Column names for the end of a period must be named valid_until (or a similarly descriptive name ending in _until).
    • Metadata columns in sql_saga that store these column names will be named valid_from_column_name and valid_until_column_name.

Guiding Principles

1. Maintain a Stateless Mindset

With each new message from the user, especially one containing test results or command output, you must clear all previous assumptions and hypotheses. Your analysis must be based only on the new information provided in the latest message. Do not refer back to your own previous hypotheses if they have been contradicted by data. Treat each interaction as a fresh start to the analysis cycle.

2. Follow a Hypothesis-Driven Cycle

All development work is an iterative process of forming hypotheses and verifying them with real-world data. The core principle is to replace assumption with verification.

The Iterative Development Cycle

All development work, especially bug fixing, must follow a rigorous, hypothesis-driven cycle to ensure progress is verifiable, correct, and does not waste time on flawed assumptions. A task is not complete until the final step of this cycle has been successfully executed. A hypothesis is not confirmed until it is supported by direct, empirical observation.

This cycle is designed to optimize for the shortest overall development time, not just the shortest "happy path". In a complex system, the probability of any given change being perfectly correct on the first attempt is low. A process that optimizes for the happy path (e.g., CHANGE -> CLEANUP -> DONE) results in a very long and costly sad path when the change is incorrect (INCORRECT CHANGE -> CLEANUP -> ERROR -> RE-INVESTIGATE -> ...).

Our process deliberately adds verification steps (CHANGE -> VERIFY -> CLEANUP -> DONE), which makes the happy path slightly longer. However, it makes the sad path significantly shorter (INCORRECT CHANGE -> VERIFY -> RE-HYPOTHESIZE -> ...). By catching errors early and cheaply, we minimize wasted effort and ensure the average time to a correct solution is much lower.

This cycle is designed to optimize for the shortest overall development time, not just the shortest "happy path". In a complex system, the probability of any given change being perfectly correct on the first attempt is low. A process that optimizes for the happy path (e.g., CHANGE -> CLEANUP -> DONE) results in a very long and costly sad path when the change is incorrect (INCORRECT CHANGE -> CLEANUP -> ERROR -> RE-INVESTIGATE -> ...).

Our process deliberately adds verification steps (CHANGE -> VERIFY -> CLEANUP -> DONE), which makes the happy path slightly longer. However, it makes the sad path significantly shorter (INCORRECT CHANGE -> VERIFY -> RE-HYPOTHESIZE -> ...). By catching errors early and cheaply, we minimize wasted effort and ensure the average time to a correct solution is much lower.

1. Hypothesize: Formulate and State a Hypothesis

  • Action: Before making any code changes, clearly state your hypothesis about the root cause of the problem in tmp/journal.md. This creates a locally persistent log of your thought process.
  • Example: "Hypothesis: The import job hangs because the batch selection query is inefficient and confuses the query planner."

2. Isolate: Create or Identify a Reproducing Test

  • Action: Ensure a test case exists that isolates the bug. This can be an existing pg_regress test or a temporary SQL script (tmp/debug_*.sql) that demonstrates the failure (e.g., a query that is slow or returns incorrect data).

3. Prototype: Propose a Non-Destructive Verification

  • Action: Before proposing a permanent fix, create a temporary, non-destructive script (e.g., tmp/verify_fix.sql) to test your proposed change. This script must use tools like EXPLAIN (ANALYZE, BUFFERS) or read-only SELECT queries to gather performance data or check logic without altering the database state.
  • Example: Create a script that runs EXPLAIN ANALYZE on a simplified query to prove it is fast and returns the expected number of rows.

4. Observe: Gather Empirical Evidence from the Prototype

  • Action: Suggest the user run the verification script from Step 3. Do not proceed until you have observed the results. This step is mandatory.
  • Standard Command: psql -d sql_saga_regress < tmp/verify_fix.sql

5. Analyze & Refine: Analyze Prototype Results

  • Action: Carefully inspect the output from the verification script.
    • If Successful: The prototype confirms the hypothesis (e.g., the new query is fast). You can now proceed to propose the permanent change.
    • If Unsuccessful: The hypothesis was incorrect. The prototype failed (e.g., the query was still slow or returned zero rows). Analyze the new data, update tmp/journal.md with a new hypothesis, and return to Step 1.

6. Implement: Propose the Permanent Change

  • Action: Only after the prototype has been successfully verified in Step 5, propose the specific code changes (using SEARCH/REPLACE blocks) for the permanent files (e.g., migrations, functions). State the expected outcome.
  • Example: "This change replaces the complex query with the simplified version that was verified to be fast in tmp/verify_fix.sql."

7. Validate: Run Full Regression Tests

  • Action: After the user applies the permanent changes, request that they run the relevant test suite to ensure the fix works and has not introduced any regressions.
  • Standard Command Format: Always use the following command structure to run tests. This ensures the extension is installed before testing and that any failures are immediately diffed for analysis.
    • Debugging Tip: For complex SQL or C functions, prefer adding RAISE DEBUG statements over RAISE NOTICE. In the corresponding test file, temporarily wrap the relevant commands with SET client_min_messages TO DEBUG; and RESET client_min_messages;. This provides targeted diagnostic output without permanently cluttering the test results.
    • Command: make install && make test ...; make diff-fail-all
    • Test Output Review: You must never propose SEARCH/REPLACE blocks for expected/*.out files. If a test fails due to intended changes, instruct the user to review and accept the new output interactively by running make diff-fail-all vim.
    • Usage:
      • To run all tests: make install && make test; make diff-fail-all
      • To run fast tests (excluding benchmarks): make install && make test fast; make diff-fail-all
      • To run specific tests: make install && make test TESTS="01_install 51_quoted_identifiers"; make diff-fail-all
    • Note on Self-Contained Tests: The 01_install.sql test handles setup required by older tests. Newer tests should be self-contained.
      • Include \i sql/include/test_setup.sql at the beginning to create necessary roles and permissions.
      • Include \i sql/include/test_teardown.sql at the end to clean up.
      • For tests that verify transactional behavior, the BEGIN/ROLLBACK block should wrap only the test logic, not the setup/teardown of non-transactional objects like roles.
      • When running a specific older test, 01_install must still be run first.
      • Test Isolation Patterns: To ensure tests are robust and independent, use SAVEPOINTs to manage transactional state within a test file.
        • For successful test cases: Wrap each distinct scenario in a SAVEPOINT <name> and RELEASE SAVEPOINT <name> block. This creates a nested transaction for the scenario. If it succeeds, its changes are committed to the main transaction.
          SAVEPOINT scenario_A;
          -- ... test logic for scenario A ...
          RELEASE SAVEPOINT scenario_A;
        • For expected failures: To test a command that is expected to raise an ERROR, wrap it in a SAVEPOINT and ROLLBACK TO SAVEPOINT block. This allows the test to capture and display the raw error message without aborting the entire test script, enabling subsequent verification steps. This is the preferred pattern for negative testing.
          SAVEPOINT expect_error;
          -- This command is expected to fail
          SELECT my_buggy_function();
          ROLLBACK TO SAVEPOINT expect_error;
          -- This command will now execute successfully
          SELECT 'The transaction is still active';

temporal_merge Planner Tracing Protocol

The temporal_merge planner includes a permanent trace column in its output (temporal_merge_plan). This is a critical architectural feature for debugging and must be handled with a strict protocol.

  1. Trace is Permanent and Performant: The trace column must never be removed from the planner's output or from any regression test that queries it. Its presence, even as NULL, is required for consistency and to allow for easy activation of tracing. The implementation is designed to have near-zero performance impact when tracing is disabled. The planner uses two patterns to achieve this:

    • Short-circuiting || operator: When tracing is off, the trace seed is NULL. Subsequent trace || jsonb_build_object(...) operations are short-circuited by the PostgreSQL planner, which knows that NULL || anything is NULL and therefore does not execute the expensive jsonb_build_object call.
    • CASE statements: In other areas, an explicit CASE WHEN ... statement is used, which also guarantees the expensive trace-building logic is skipped. Both approaches ensure that the architectural decision to have a permanent trace column incurs no runtime overhead when tracing is not active.
  2. Trace is Toggled via GUC: Tracing is activated for a specific operation by setting the session-level GUC: SET sql_saga.temporal_merge.enable_trace = true;. When false (the default), the planner populates the trace column with NULL.

  3. Debugging Protocol: When a temporal_merge test fails and the root cause is not immediately obvious from the plan output, the following steps are mandatory: a. Enable Trace: In the test file, wrap the failing CALL sql_saga.temporal_merge(...) with SET sql_saga.temporal_merge.enable_trace = true; and SET sql_saga.temporal_merge.enable_trace = false;. b. Isolate Trace in Diffs: Run the test and inspect the diff in trace to gain understanding. The expected has NULL in trace, and the actual has the trace due to the GUC variable setting. The SELECTs must remain unchanged, such that when the the issue is fixed and the trace is disabled again it will mach. This technique ensures that if the plan's data is wrong, the test will fail, and the diff output will contain the full trace from the actual plan for analysis.

  4. Trace Aggregation: When multiple atomic time segments are coalesced into a single final plan operation, their individual traces must be aggregated into a jsonb array using jsonb_agg. This provides a complete diagnostic record of all atomic segments that were merged into the final operation. The trace for each atomic segment includes the Allen Interval Relation (s_t_relation) between the covering source and target rows, providing detailed insight into the initial temporal interaction that created the segment.

8. Conclude: Update Documentation

  • Action: Only after the fix has been successfully validated in Step 7, update todo.md to move the task to a "done" state (e.g., [x]).

General Development Principles

  • Embrace Falsifiability: Treat every hypothesis and plan as provisional until proven by direct, empirical observation. Frame plans as "Current Plan" or "Next Step," not "Final Plan." The goal is to be prepared to be wrong and to allow evidence to guide the development process. Optimism should be rooted in the rigor of the process itself, not in the assumed correctness of any single solution. This mindset is the primary defense against hubris and wasted effort.
  • Fail Fast:
    • Functionality that is expected to work should fail immediately and clearly if an unexpected state or error occurs.
    • Do not mask or work around problems; instead, provide sufficient error or debugging information to facilitate a solution. This is crucial for maintaining system integrity and simplifying troubleshooting, especially in backend processes and SQL procedures.
  • Declarative Transparency:
    • Where possible, store the inputs and intermediate results of complex calculations directly on the relevant records. This makes the system's state self-documenting and easier to debug, inspect, and trust, rather than relying on dynamic calculations that can appear magical.
    • Example: The temporal_merge planner was refactored to separate ephemeral and non-ephemeral data into distinct payloads (ephemeral_payload, data_payload). This makes the core logic clearer, as the complex coalescing operations now work on a clean data_payload, instead of requiring repeated, on-the-fly stripping of ephemeral keys. This separation of concerns makes the algorithm's intent explicit.
    • Example: The time_points_unified CTE is the key to this. It uses a FIRST_VALUE window function to establish a single, authoritative corr_ent for each entity, which is then used as a stable partitioning key throughout the rest of the plan. This prevents the timeline for a single conceptual entity from being fragmented.
    • Example: The planner's logic for determining if a source row represents a "new" entity is a critical declarative step. The new_ent flag is derived from information that is already available (stable_key IS NULL and target_entity_exists), rather than being passed down through complex state. This makes the logic for partitioning timelines robust and correct across multiple batches.

The "Observe, Verify, Implement" Protocol for Complex Changes

When a task is complex or has a history of regressions (e.g., the rename_following trigger), a more rigorous, data-driven protocol is required to avoid speculative fixes. This protocol is a practical application of the "Observe first, then change" principle.

  1. Establish a Baseline (Observe):

    • Action: Before writing any new logic, instrument the existing, passing code with diagnostic logging (RAISE NOTICE).
    • Goal: Capture the exact inputs, outputs, and event sequences that occur during the relevant tests. This baseline data serves as a definitive specification for the new logic. The tests are expected to fail due to the new logging output.
  2. Verify New Logic in Read-Only Mode (Verify):

    • Action: Implement the new logic, but keep it "read-only." It should calculate its decisions but only log them, without altering the program's control flow (e.g., no early RETURN statements).
    • Goal: Run the tests again. Analyze the new logs to compare the baseline data against the logic's decisions. This step proves the logic is correct against real-world test data before it is activated.
  3. Activate the Verified Logic (Implement):

    • Action: Only after the logic has been verified in read-only mode, remove all diagnostic logging and enable the new control flow (e.g., add the early RETURN statement).
    • Goal: Run the tests a final time to confirm that the now-active logic works as intended and that no regressions have been introduced.

This protocol transforms debugging from a cycle of "guess-and-check" into a methodical, scientific process of data gathering and verification, dramatically reducing the number of failed attempts.

Development Journaling

For complex, multi-step tasks like major refactoring, a detailed plan should be maintained in tmp/journal.md. This file outlines the sequence of steps, the actions required for each step, and the expected outcome. It serves as a roadmap for the task, ensuring a systematic approach. The journal should be cleared when you begin on a new major todo item. To clear the journal, you must propose the shell command echo -n > tmp/journal.md (or the relevant journal file). You must not attempt to clear it using a SEARCH/REPLACE block, as its contents are ephemeral and cannot be reliably searched. This helps for bug fixing and iterative development by logging hypotheses and outcomes, providing a low-level history of the debugging process, and to spot patterns of failure or repetition, upon wich you can suggest a clearing of the context for improved focus.

Known Pitfalls and Falsified Assumptions

This section documents incorrect assumptions that have been disproven through testing. Reviewing these can help avoid repeating past mistakes.

  • MVCC and Transaction Visibility in PL/pgSQL: The set of rows visible to a pl/pgsql function is determined when the function begins execution. Within a single function, a SET-based query and a LOOP that executes queries will both operate on the same data snapshot. It is a flawed assumption to think that a LOOP is somehow more robust to transaction visibility issues than a single complex query within the same function. If a set-based query is failing, the bug is in the query's logic, not in a fundamental limitation of MVCC for set-based operations.

  • regclass Type Casting in C Triggers: The regclass data type is a symbolic reference to a relation OID, not just the OID itself. When a C function using SPI executes a query involving a regclass column, the PostgreSQL backend may attempt to format this value, which involves looking up the object's name. This behavior can be dangerous and lead to subtle bugs.

    • The Problem in sql_drop Event Triggers: An sql_drop trigger fires after the object has been deleted from the system catalogs. If an SPI query in the trigger function selects a regclass column that refers to a now-deleted object, the backend's attempt to look up the object's name will fail. This causes SPI_execute to return an error, leading to unpredictable behavior if not handled precisely. This can manifest as cascading, incorrect error messages or the appearance that queries are returning rows when they have actually failed.
    • The Problem in Regular Triggers: In regular DML triggers, using regclass in a WHERE clause (e.g., WHERE table_oid = $1) can be sensitive to the current search_path and rely on implicit casting rules. This can make queries less robust.
    • The Solution: For maximum robustness in any C-based trigger function using SPI:
      1. Always CAST to oid in SQL: In all SPI queries, explicitly cast regclass columns to oid for both SELECT lists and WHERE clauses (e.g., SELECT table_oid::oid ..., ... WHERE table_oid::oid = $1).
      2. Use OIDOID for Parameters: When using SPI_execute_with_args, specify the parameter type as OIDOID for relation identifiers.
      3. Perform Lookups in C: Fetch raw OIDs from the database. Any conversion from an OID to a textual name for error messages must be done manually in the C code (e.g., using regclassout) after the query has successfully completed.
  • IGNORE NULLS in Window Functions: PostgreSQL does not support the IGNORE NULLS clause for window functions like LAG, LEAD, or FIRST_VALUE. This is a common pitfall.

    • The Problem: Attempting to use FIRST_VALUE(my_col) IGNORE NULLS OVER (...) will result in a syntax error.
    • The Solution: The standard and most robust workaround is a "gaps-and-islands" approach. This involves creating a new grouping key using a conditional SUM() window function that increments every time a non-NULL value is encountered. This partitions the data into "islands" where a value can be safely propagated to adjacent NULLs using an aggregate like MAX() or a carefully framed FIRST_VALUE().
    • Example: The diff_with_propagated_ids CTE in the temporal_merge planner is a canonical example of this pattern, using both forward (look_behind_grp) and backward (look_ahead_grp) running sums to find the nearest non-NULL value in either direction.

When the Cycle Fails: Changing Strategy

When repeated iterations of the hypothesis-driven cycle fail to resolve a persistent and complex bug (such as a memory corruption crash), it is a sign that the underlying assumptions are wrong and a change in strategy is required.

The "Simplify and Rebuild" Approach

  1. Formulate a new meta-hypothesis: State that the complexity of the current implementation is the source of the problem.
  2. Strip to a non-crashing stub: Drastically simplify the faulty code to a minimal, stable state that is guaranteed not to crash, even if it returns an incorrect result. This changes the failure mode from a crash to a predictable test failure.
  3. Verify stability: Run the tests to confirm that the server is now stable and the crash is gone. This establishes a new, reliable baseline.
  4. Incrementally re-introduce logic (Logical Binary Search): Re-introduce functionality piece by piece, running tests at each step.
    • Start by restoring the first half of the original logic. If the system remains stable, the bug is in the second half. If it crashes, the bug is in the first half.
    • Continue this process, dividing the suspicious code block in half with each iteration, until the single line or small section of code causing the instability is isolated.
    • This process is a logical "binary search" on the code's functionality.
  5. Resume the normal cycle: Once the root cause is identified, resume the standard hypothesis-driven cycle to fix the specific issue.