[Enhancement] Detect and manage overlapping IntelliJ editor guides

[YangSiJun528]

Summary

Manage IntelliJ's native guide rendering when it can appear beside a multiline
active vertical guide from Bracket Pair Guides.

This issue now also covers the two-line report in
#27.
The report is not caused by Current scope alone:

• IntelliJ's regular indent guide is the physical native line in the New UI.
• Matched-brace and Current scope markers can emphasize that existing line.
• At a brace boundary, matched-brace highlighting can emphasize the line even
  when Current scope is disabled.

Provide one IntelliJ-integration gate, one native-highlight mode, and one
independent indent-guide option. The default should reduce competing native
highlighting while preserving useful regular indent guides:

manageNativeVisuals = true
nativeHighlightMode = SUPPRESS_MATCHED_BRACE_AND_CURRENT_SCOPE
hideNativeIndentGuides = false

This default can still show a dim IntelliJ indent guide beside the plugin's
active guide. Users who require exactly one vertical line can opt into hiding
IntelliJ indent guides.

Preference-schema migration from earlier plugin versions is not required for
this design. Runtime ownership and restoration of native values are still
required.

Native rendering model

The relevant IntelliJ values are:

Symbol	IntelliJ value	Effect
B	CodeInsightSettings.HIGHLIGHT_BRACES	Enables native matched-brace processing and gates the standard Current scope path
S	CodeInsightSettings.HIGHLIGHT_SCOPE	Enables Current scope while B is enabled
I	affectedEditor.settings.isIndentGuidesShown	Controls effective regular indent-guide visibility for that editor

The global suppression target for I is
EditorSettingsExternalizable.isIndentGuidesShown. Detection must use the
affected editor's effective value because an editor may override the global
default.

Marker creation

For a standard monolithic editor, the relevant marker conditions are:

matchedBraceMarker =
    B && caretTouchesMatchedBrace && multilinePair

currentScopeMarker =
    B && S && scopeFallbackResolvesEnclosingPair && multilinePair

marker = matchedBraceMarker || currentScopeMarker

HIGHLIGHT_SCOPE does not control the direct matched-brace marker at a brace
boundary. IntelliJ processes direct brace highlighting before evaluating
HIGHLIGHT_SCOPE, and both paths can install the same line marker for a
multiline range.

Relevant platform paths:

• On the minimum supported 2024.1 platform,
  BackgroundHighlightingUtil.needMatching()
  rejects the standard matching update when HIGHLIGHT_BRACES is disabled.
• BraceHighlightingHandler
  handles a direct brace first and evaluates HIGHLIGHT_SCOPE afterward.
• A multiline match installs a
  DefaultLineMarkerRenderer.

New UI visibility

In the New UI, the marker does not directly paint the code-area line.
IndentGuideRenderer
uses it to choose the color of an existing indent guide.

Conceptually:

ordinaryNativeLine = I && indentDescriptorExists

highlightedNativeLine =
    ordinaryNativeLine && marker && matchingStartLine

Consequences:

• Disabling Show indent guides removes the native code-area line.
• Disabling matched-brace highlighting removes its emphasis but leaves a dim
  regular indent guide when I remains enabled.
• Disabling Current scope does not remove direct matched-brace emphasis at a
  brace boundary.
• In the New UI, matched-brace and Current scope highlighting are not separate
  physical code-area lines; they can recolor the existing indent guide.

Classic UI visibility

In the Classic UI, DefaultLineMarkerRenderer can paint a gutter-side marker
directly. That marker can remain visible independently of Show indent
guides. Detection and tests must not assume the New UI rendering path for
every supported IDE configuration.

Reproduction from #27

The verified New UI states are:

State at a multiline brace boundary	Result
B=On, S=On, I=On	Emphasized native indent guide + plugin guide
B=On, S=Off, I=On	Same result; the direct matched-brace marker remains
B=Off, I=On	Dim regular indent guide + plugin guide
I=Off	Plugin guide only in the code area

Settings model

Use a parent gate, a mode for the dependent native highlight settings, and a
separate indent-guide Boolean:

enum class NativeHighlightMode {
    SUPPRESS_MATCHED_BRACE_AND_CURRENT_SCOPE,
    SUPPRESS_CURRENT_SCOPE_ONLY,
    LEAVE_INTELLIJ_HIGHLIGHTING_UNCHANGED,
}

data class IntelliJIntegrationPreferences(
    val manageNativeVisuals: Boolean = true,
    val nativeHighlightMode: NativeHighlightMode =
        NativeHighlightMode.SUPPRESS_MATCHED_BRACE_AND_CURRENT_SCOPE,
    val hideNativeIndentGuides: Boolean = false,
)

The parent is an operational gate, not a persisted mirror of whether every
child is selected:

• Parent off: release every owned native value and leave IntelliJ rendering
  unchanged while retaining the selected child configuration.
• Parent on: apply the selected highlight mode and indent-guide option.
• Bracket Pair Guides' existing global Enabled option remains the outer
  lifecycle gate. Disabling the plugin also releases every native value.

Highlight-mode mapping

The UI describes rendered effects, while ownership must change only the native
value required for that mode:

Mode	HIGHLIGHT_BRACES ownership	HIGHLIGHT_SCOPE ownership
Suppress matched-brace and Current scope	Own as false	Do not own or rewrite; Current scope is suppressed by the B gate
Suppress Current scope only	Do not own or rewrite	Own as false
Leave IntelliJ highlighting unchanged	Do not own; restore if necessary	Do not own; restore if necessary

Do not silently set HIGHLIGHT_SCOPE=false when suppressing matched-brace
highlighting. The rendered Current scope effect is dependent, but the native
settings and restore values remain independently owned.

Only the selected highlight slot may remain owned in steady state. When
switching modes, acquire the new suppressing slot before releasing the old one
inside one coordinator transaction so the intermediate state does not briefly
re-enable both native effects.

hideNativeIndentGuides=true independently owns the global
EditorSettingsExternalizable.isIndentGuidesShown value as false.

The complete "plugin guide only" configuration is:

manageNativeVisuals = true
nativeHighlightMode = SUPPRESS_MATCHED_BRACE_AND_CURRENT_SCOPE
hideNativeIndentGuides = true

Settings UI

Use one shared explanation rather than explanatory text under every control:

IntelliJ integration

[x] Adjust IntelliJ guide rendering while Bracket Pair Guides is enabled

Native guide highlighting:
[ Hide matched-brace and Current scope highlighting  v ]

[ ] Hide regular IntelliJ indent guides

IntelliJ visuals left enabled may appear beside the active bracket guide.
Keeping indent guides can result in two adjacent vertical lines.

Highlight-mode choices:

1. Hide matched-brace and Current scope highlighting — default
2. Hide Current scope highlighting only
3. Leave IntelliJ highlighting unchanged

When the parent is off, disable the child controls without clearing their
values.

Show the shared informational text only when all of the following are true:

pluginVerticalGuideCanAppear =
    pluginEnabled && activeGuideEnabled && verticalGuideEnabled

potentialNativeVisualRemains =
    !manageNativeVisuals ||
    !hideNativeIndentGuides ||
    nativeHighlightMode != SUPPRESS_MATCHED_BRACE_AND_CURRENT_SCOPE

showSharedInfo =
    pluginVerticalGuideCanAppear && potentialNativeVisualRemains

This is neutral informational text, not a warning: retaining IntelliJ indent
guides is useful and is the default. Always show one separate common restoration
note for the group:

Original IntelliJ settings are restored when this integration or Bracket Pair
Guides is disabled.

Runtime conflict detection

The Settings-page information describes the possible coexistence of regular
indent guides. The runtime notification has a narrower purpose: alert the user
only when native matched-brace or Current scope highlighting can visibly
emphasize another line beside a multiline plugin guide.

Regular IntelliJ indent guides being enabled is intentional under the default
configuration and is not sufficient to trigger a notification.

Before notifying, require all of the following:

1. Bracket Pair Guides is enabled for the current editor and language.
2. Active guide and Vertical are enabled.
3. A multiline active vertical guide has actually been displayed.
4. Native matched-brace or Current scope highlighting remains effective after
   the selected management settings have been applied.
5. A supported editor path confirms that the native highlighted marker can be
   visible beside the plugin guide.

The native highlight sources must be distinguished:

directMatchedBraceCapability =
    HIGHLIGHT_BRACES && caretTouchesMatchedBrace && multilinePair

currentScopeCapability =
    HIGHLIGHT_BRACES && HIGHLIGHT_SCOPE &&
    scopeFallbackResolvesEnclosingPair

Do not report a Current scope conflict from HIGHLIGHT_SCOPE=true alone, and do
not assume disabling Current scope resolves the direct brace-boundary case.

Classify visibility by UI path:

• New UI: require an effective indent guide and a matching indent
  descriptor beside the plugin geometry. A native marker without that carrier
  is not a visible code-area conflict.
• Classic UI: evaluate direct matched-brace and scope-fallback gutter-marker
  capability separately from indent guides because the marker can paint
  directly.

Use supported editor APIs and the plugin's computed guide geometry. Do not
inspect private renderer internals. If the UI path, marker source, or relevant
geometry cannot be classified safely, skip the notification.

Initial compatibility scope

The initial implementation covers standard monolithic editors only.

Do not enable conflict notifications or native-setting ownership in Remote
Development, Code With Me, or local-editor-support paths until their settings
context is handled explicitly. CodeInsightSettings and
EditorSettingsExternalizable can be client-specific, and a shared restore
value can restore the wrong client's setting.

Notification behavior

Use one IntelliJ notification balloon with informational severity. Do not use
an editor banner: this is guidance about an IDE-wide setting combination, not a
file-specific error requiring persistent editor space.

Suggested copy:

IntelliJ guide highlighting may overlap

IntelliJ highlighting may draw another line beside Bracket Pair Guides.
Review the integration settings if this is unintended.

Provide one action:

• Review settings: open the IntelliJ integration settings group.

The notification is advisory. Publishing, closing, or opening it must not
change IntelliJ settings or plugin preferences.

Show it only once, with persisted application-level state. Do not show it again
on every IDE process or project after it has been published. Mark the state when
the notification is successfully published, not when detection first starts.
If the plugin exposes a notification reset, keep that reset in the IntelliJ
integration settings and use the same persisted state as the single source of
truth.

The ordinary indent-guide coexistence described in the Settings page must not
publish this notification. It is intended only for the prominent native
highlighting case that a user may have enabled unintentionally.

Suppression lifetime

Suppression is preference-based, not toggled per caret position or active pair.
Once requested, hold the corresponding native value while both Bracket Pair
Guides and the IntelliJ-integration gate are enabled. Per-guide mutation would
cause global-setting churn and visible flicker across editors.

Disabling the integration gate, disabling Bracket Pair Guides, changing to a
mode that no longer owns a value, dynamic plugin unload, or application exit
releases the relevant ownership.

For indent-guide changes, use the public
EditorSettingsExternalizable.setIndentGuidesShown(...) setter and refresh open
editors on the EDT. Re-evaluate the affected editor afterward because an
explicit per-editor override can remain effective.

HIGHLIGHT_BRACES and HIGHLIGHT_SCOPE expose no change-notifying setter, and
IntelliJ exposes no public API dedicated to removing an already installed scope
marker. Do not call internal brace or indent renderers. Request supported
editor/daemon recomputation where available; the change must be visible by the
next native brace-highlighting update and must not require an IDE restart.

Ownership and restoration

Use one application-level lifecycle coordinator with three independent
Boolean ownership slots:

• HIGHLIGHT_BRACES
• HIGHLIGHT_SCOPE
• EditorSettingsExternalizable.isIndentGuidesShown

Each slot must:

1. Persist the exact original Boolean before first enforcing false.
2. Distinguish original false from no ownership, for example with Boolean?.
3. Change only its assigned native value.
4. Restore only when the current value is still the plugin-owned false.
5. Release on parent disable, plugin disable, mode/option change, dynamic unload,
   and application exit.
6. Treat an external change to true as the newer user choice: preserve it,
   release that slot, and update only the corresponding plugin configuration.
7. Persist once after releasing all applicable slots during shutdown/unload.

Mode updates after an external override:

• External HIGHLIGHT_BRACES=true while the first mode owns it: switch the
  highlight mode to Leave IntelliJ highlighting unchanged.
• External HIGHLIGHT_SCOPE=true while Current-scope-only mode owns it: switch
  the highlight mode to Leave IntelliJ highlighting unchanged.
• External indent guides true while owned: set
  hideNativeIndentGuides=false.
• Keep manageNativeVisuals=true; release only the overridden child target.

These preference updates must be compare-and-set operations against the mode
that still owns the overridden target. Delayed cleanup for a previous mode must
not replace a newer mode selected by the user.

Native writes and restores performed by the coordinator must not be observed as
external overrides. Use a scoped reentrancy/write guard, including around the
indent-guide setter and any change event it produces.

An external write of false while the plugin already owns false is
observationally indistinguishable from the plugin-owned value. Document this as
a limitation of Boolean value-based ownership.

Non-goals

• Treating the three native values as independent rendered effects
• Mutating HIGHLIGHT_SCOPE merely because HIGHLIGHT_BRACES is suppressed
• Hiding regular IntelliJ indent guides by default
• Repositioning the plugin guide to mask a native indicator
• Depending on private renderer implementation details solely for detection
• Acquiring and releasing application-wide settings on every caret movement
• Supporting Remote Development settings ownership in the initial version

Acceptance criteria

• manageNativeVisuals defaults to true.
• The default highlight mode suppresses matched-brace and standard Current
  scope rendering by owning only HIGHLIGHT_BRACES=false.
• Regular IntelliJ indent guides remain enabled by default.
• Parent off restores all owned native values without clearing the selected
  mode or indent-guide option.
• Current-scope-only mode owns only HIGHLIGHT_SCOPE=false and does not
  modify or own HIGHLIGHT_BRACES.
• Leave-unchanged mode owns neither highlight value.
• A completed mode transition owns at most one highlight value and does not
  briefly re-enable both native effects while switching.
• The shared Settings information appears only when the plugin vertical
  guide can appear and a potentially overlapping native visual remains.
• The Repeated vertical lines #27 brace-boundary case is not attributed to Current scope alone.
• With Current scope off at a brace boundary, direct matched-brace emphasis
  can still be reproduced.
• With matched-brace suppression and indent guides enabled, the New UI shows
  a dim native indent guide rather than the emphasized guide.
• With effective indent guides disabled, the New UI code area shows only the
  plugin guide.
• Regular indent guides alone never trigger the runtime notification.
• The notification requires an actually displayed multiline plugin vertical
  guide and effective native matched-brace or Current scope highlighting.
• New UI notification eligibility requires a matching effective indent
  descriptor, not only the global indent-guide flag or an internal marker.
• Classic UI notification eligibility accounts for a directly painted
  gutter marker independently of the indent-guide setting.
• An unclassifiable editor or marker path does not produce a notification.
• The notification uses informational severity and appears as a balloon,
  not an editor banner.
• The notification is published only once across IDE processes and
  projects unless its persisted state is explicitly reset.
• The only notification action opens the IntelliJ integration settings.
• Publishing, closing, or opening the notification does not mutate any
  IntelliJ setting or plugin preference.
• Original native values of both true and false restore correctly for
  all three slots.
• Changing modes releases obsolete ownership without rewriting an unrelated
  native value.
• An external override updates only the corresponding child configuration
  and leaves the parent enabled.
• Delayed external-override cleanup cannot replace a newer highlight mode.
• Plugin-originated acquire and restore writes do not clear or rewrite the
  selected mode or indent-guide option.
• Indent-guide changes refresh globally bound open editors; an explicit
  per-editor override remains detectable and is not silently overwritten.
• New UI and Classic UI marker behavior are tested separately on supported
  platform versions.
• Remote-development and other excluded editor paths receive neither a
  notification nor native-setting mutation.
• Documentation explains the difference between reducing native highlight
  emphasis and removing the physical native indent guide.

[YangSiJun528]

Note: It would be possible to adjust the horizontal pixel offset so the two lines align, but that would only mask the conflict rather than solve it. It is also not the behavior I want, so I decided not to pursue that approach.