Preserve trailing comments when removing an import

[MBoegers]

Problem

An end-of-line comment on an import line is stored in the LST as the prefix of the next import, not as a suffix of its own. RemoveImport carries a removed import's prefix forward to the following import so blank lines between groups are preserved (#3868), but it only forwarded getLastWhitespace() — any comments in that prefix were silently discarded.

Because of where comments are stored, removing an import touched comments that belong to neighbouring lines:

• the trailing comment of the line before the removed import was dropped (it lives in the removed import's prefix);
• standalone / commented-out lines before the removed import were dropped (same reason);
• the removed import's own trailing comment (which lives in the next import's prefix) dangled onto the previous line, misattributing it.

The most visible casualty was a Kotlin // ktlint-disable suppression on a wildcard import being removed when the import on the next line was renamed, which then broke the build with a fresh lint failure.

Fix

When removing an import:

• forward the removed import's prefix comments (the previous line's trailing comment and any standalone / commented-out lines) onto the next import, still preserving blank lines; and
• drop only the removed import's own end-of-line comment — the first comment of the next import's prefix when it sits on the same line as the removed import (no newline precedes it).

Behaviour, by example

Each example removes java.util.ArrayList. A comment is kept unless it described the removed line itself.

1. The removed import's own end-of-line comment → dropped (it described the import that's gone):

 import java.util.List;
-import java.util.ArrayList; // explains the removed import
 import java.util.UUID;

2. The previous line's end-of-line comment → preserved (e.g. a ktlint suppression on a wildcard import — the original bug):

 import java.util.List; // ktlint-disable no-wildcard-imports
-import java.util.ArrayList;
 import java.util.UUID;

3. A standalone / commented-out line before the removed import → preserved:

 import java.util.List;
 // import java.util.Date
-import java.util.ArrayList;
 import java.util.UUID;

4. The following line's end-of-line comment → preserved:

 import java.util.List;
-import java.util.ArrayList;
 import java.util.UUID; // explains UUID

Last import removed

When the last import is removed there is no following import to carry its prefix onto — the follower is the first class declaration (or the EOF when there are none). The same rules apply, except the class keeps its own spacing (an import-group blank line is not pushed in front of it). So a preceding line's comment is still preserved:

 import java.util.List; // ktlint-disable no-wildcard-imports
-import java.util.ArrayList;

 class A {
 }

…and the removed import's own comment is still dropped (rather than dangling onto the previous line):

 import java.util.List;
-import java.util.ArrayList; // explains the removed import

 class A {
 }

Tests

RemoveImportTest (rewrite-java-test): dropsTrailingCommentOfRemovedImport, preservesTrailingCommentOnLineBeforeRemovedImport, preservesTrailingCommentAndBlankLineAfterRemovedImport, preservesCommentedOutLineBeforeRemovedImport, preservesTrailingCommentOnLineAfterRemovedImport, and for the last-import case dropsTrailingCommentOfRemovedLastImport, preservesPreviousLineCommentWhenLastImportRemoved, preservesCommentedOutLineWhenLastImportRemoved.

RemoveImportTest (rewrite-kotlin): preservesKtlintSuppressionCommentOnLineBeforeRemovedImport, preservesPreviousLineCommentWhenLastImportRemoved.

Context: moderneinc/customer-requests#2437.

[sambsnyd]

If we're going to try and be smarter about this we should probably differentiate between "line ending" and "preceding line" comments .

import com.foo.Bar; // This comment is probably about Bar


// This comment is probably about Baz
import com.foo.Baz;

In this example the two comments are both part of the import Baz prefix, but should probably be treated separately.