Skip to content

feat: manual repository onboarding (diddo onboard) - #21

Open
drugoi wants to merge 11 commits into
mainfrom
feature/manual-onboarding-import
Open

drugoi wants to merge 11 commits into
mainfrom
feature/manual-onboarding-import

Conversation

@drugoi

@drugoi drugoi commented Mar 23, 2026

Copy link
Copy Markdown
Owner

Summary

Adds diddo onboard to backfill commit history for the current repo from a user-chosen cutoff date (on or after), with identity selection and optional saved aliases under [onboarding] in config.

What’s included

  • CLI: onboard subcommand (no summary output flags)
  • Config: OnboardingConfig, IdentityAlias, save_onboarding_aliases
  • Core: git log scan (GIT_LOG_FORMAT), filter/import, preselection from current git user + aliases + detected authors
  • Interactive flow: prompts for cutoff, numbered identities, comma-separated selection, optional save to config
  • Tests: unit tests + injected run_with integration-style tests
  • Docs: README section, design + implementation plan under docs/plans/

How to try

cargo run -- onboard

(from a git repo root; uses real git in cwd)

Verification

  • cargo test
  • cargo clippy -- -D warnings

Made with Cursor

drugoi added 7 commits March 23, 2026 15:07
- Add GitIdentity and build_preselected_identities (anchors, saved aliases,
  detected email matches; name-only only when no email in scan)
- Unit tests for alias preselection and name-only guard

Made-with: Cursor
- Implement run/run_with: git repo validation, git log scan, cutoff prompt,
  identity list + selection, filter + import with scanned/matched stats
- Extend import_commits with explicit scanned/matched counts for outcomes
- Optional merge of new identities into config via save_onboarding_aliases
- Tests: empty history, selective import with injected git; help lists onboard
- README: onboard section; add design + implementation plan docs

Made-with: Cursor
@drugoi drugoi added the enhancement New feature or request label Mar 23, 2026
- Store full git SHA in hook; scan with git log -z and %B for hook-parity messages
- Add committed_local_date from %cI for cutoff; name-only identity match is ASCII case-insensitive
- save_onboarding_aliases uses AppConfig::load_from_file (no env merge into TOML)
- Tests for dedupe, cutoff boundaries, identity selection, env key not persisted
- README cutoff wording; drop redundant field rustdoc

Made-with: Cursor

@maximgorbatyuk maximgorbatyuk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Well-structured PR with good separation of concerns, solid test coverage, and thorough documentation. The run_with injection pattern is great for testability.

However, there's one critical data-loss bug (onboarding overwrites hook-recorded diff stats) and a high-severity dedup issue (hash length change) that should be fixed before merge. Details in line comments.

Comment thread src/onboarding.rs
branch: branch.to_string(),
files_changed: 0,
insertions: 0,
deletions: 0,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical: onboarding will silently destroy hook-recorded diff stats

to_db_commit hardcodes files_changed: 0, insertions: 0, deletions: 0. Since db::insert_commit uses ON CONFLICT(repo_path, hash) DO UPDATE SET ... unconditionally, re-importing a commit that the hook already recorded will overwrite its real diff stats with zeroes.

The test import_is_deduped_when_commit_already_exists_from_hook_style_row passes only because commit_count stays the same — but the row was silently mutated.

Suggested fix: add a separate insert method (or flag) that uses INSERT ... ON CONFLICT DO NOTHING:

// in db.rs
pub fn insert_commit_if_new(&self, commit: &Commit) -> Result<bool> {
    let rows = self.connection.execute(
        "INSERT INTO commits (
            hash, message, repo_path, repo_name, branch,
            files_changed, insertions, deletions, committed_at, author_email
        ) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
        ON CONFLICT(repo_path, hash) DO NOTHING",
        params![
            &commit.hash, &commit.message, &commit.repo_path,
            &commit.repo_name, &commit.branch, commit.files_changed,
            commit.insertions, commit.deletions,
            commit.committed_at.to_rfc3339(), &commit.author_email,
        ],
    )?;
    Ok(rows > 0)
}

Then use it in import_commits and count inserted directly:

let mut inserted = 0;
for c in commits {
    let row = to_db_commit(c, repo_path, repo_name, branch);
    if database.insert_commit_if_new(&row)? {
        inserted += 1;
    }
}

This also eliminates the commit_count before/after race.

Prompt: Apply the fix above — add insert_commit_if_new to db.rs, use it in import_commits, remove the commit_count before/after logic, and update the dedup test to assert the existing row's files_changed was NOT overwritten.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7ce9f4a: added Database::insert_commit_if_new (INSERT … ON CONFLICT DO NOTHING) and switched onboarding import_commits to use it. Counts now come from rows inserted, not commit_count before/after. Extended import_is_deduped_when_commit_already_exists_from_hook_style_row to assert files_changed / insertions / deletions stay intact after a duplicate import attempt.

Comment thread src/hook.rs
G: FnMut() -> io::Result<String>,
{
let hash = trim_git_output(run_git(&["rev-parse", "--short", "HEAD"])?);
let hash = trim_git_output(run_git(&["rev-parse", "HEAD"])?);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High: hash length change breaks dedup with existing database rows

Switching from rev-parse --short HEAD (7-char) to rev-parse HEAD (40-char SHA) is a good move for correctness, but it's a silent breaking change for existing databases.

Old hook-recorded rows have short hashes (e.g. "abc1234"). Onboarding scans with %H (full SHA). The unique constraint is on (repo_path, hash), so these won't conflict — onboarding will insert duplicate entries for every commit that was recorded before this upgrade.

Suggested fix options (pick one):

  1. Prefix-match on conflict — before importing, check if a row with a matching hash prefix already exists:
fn commit_exists_by_prefix(db: &Database, repo_path: &str, full_hash: &str) -> Result<bool> {
    // check both exact match and short-prefix match
    db.connection.query_row(
        "SELECT COUNT(*) FROM commits WHERE repo_path = ?1 AND (hash = ?2 OR hash = ?3)",
        params![repo_path, full_hash, &full_hash[..7.min(full_hash.len())]],
        |row| row.get::<_, i64>(0),
    ).map(|c| c > 0)
}
  1. One-time migration — expand all short hashes in the DB to full hashes on startup (via git rev-parse <short>).

  2. At minimum — document this in the README/changelog so users know to expect duplicates if they onboard a repo with pre-upgrade hook data.

Prompt: Choose an approach and implement it. If going with option 1, integrate the prefix check into import_commits. If going with option 2, add a migration step in Database::open. If option 3, add a note to the README under the onboarding section.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We went with a one-time migration on DB open: run_commit_hash_migration expands abbreviated OIDs via git rev-parse when the repo path still exists, and drops the short row if a full-hash row already exists. Together with the hook now storing full rev-parse HEAD, onboarding imports align on the same key without duplicate rows after migration runs.

Comment thread src/onboarding.rs Outdated
}

fn identity_same(a: &IdentityCandidate, b: &IdentityCandidate) -> bool {
a.name == b.name && a.email == b.email

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: case-sensitivity inconsistency between identity_same and identity_matches

identity_same uses exact ==, but identity_matches (line 414) uses eq_ignore_ascii_case for both email and name. This means:

  • An alias saved as "Me@Example.com" and a detected identity "me@example.com" won't show as preselected in the UI (exact mismatch)
  • But they will match during import filtering (case-insensitive match)
  • Alias dedup in the save flow will also miss case variants, creating near-duplicate saved aliases

Suggested fix:

fn identity_same(a: &IdentityCandidate, b: &IdentityCandidate) -> bool {
    let email_eq = match (&a.email, &b.email) {
        (Some(ae), Some(be)) => ae.eq_ignore_ascii_case(be),
        (None, None) => true,
        _ => false,
    };
    let name_eq = match (&a.name, &b.name) {
        (Some(an), Some(bn)) => an.eq_ignore_ascii_case(bn),
        (None, None) => true,
        _ => false,
    };
    email_eq && name_eq
}

Prompt: Replace identity_same with the case-insensitive version above and add a test that verifies identity_same treats "Me@X.com" and "me@x.com" as equal.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7ce9f4a: shared optional_str_eq_ignore_case for email and name on identity_same, plus identity_alias_same for the “save aliases” merge / has_new_alias checks so preselection, import filtering, and config dedup stay consistent. Added identity_same_treats_ascii_case_variants_as_equal.

Comment thread src/onboarding.rs Outdated
use std::io;

let database = Database::open_in_memory().unwrap();
let config_path = std::env::temp_dir().join("diddo-onboard-empty.toml");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: fixed temp file paths can collide in parallel test runs

These integration tests use hard-coded filenames in std::env::temp_dir() (e.g. "diddo-onboard-empty.toml"). The rest of the test suite uses a temp_dir() helper that creates unique directories with timestamps.

With cargo test running tests in parallel, two runs hitting the same file can cause flaky failures.

Suggested fix: reuse the existing temp_dir helper pattern from config.rs tests:

fn temp_dir(label: &str) -> std::path::PathBuf {
    let dir = std::env::temp_dir().join(format!(
        "diddo-onboard-{}-{}",
        label,
        std::time::SystemTime::now()
            .duration_since(std::time::UNIX_EPOCH)
            .unwrap()
            .as_nanos()
    ));
    std::fs::create_dir_all(&dir).unwrap();
    dir
}

// then:
let temp = temp_dir("empty");
let config_path = temp.join("config.toml");
// ... at end:
let _ = std::fs::remove_dir_all(temp);

Apply to all three integration tests.

Prompt: Add a temp_dir helper to the onboarding test module and update all three integration tests to use unique temp directories with cleanup.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 7ce9f4a: onboarding integration tests now use a temp_dir_onboard helper (pid + nanos under std::env::temp_dir()) and remove the directory after each test, matching the pattern used in config tests.

Comment thread src/onboarding.rs
Comment thread docs/plans/2026-03-23-manual-onboarding-design.md
drugoi added 3 commits March 23, 2026 20:53
- Introduced rust-toolchain.toml to specify the Rust version (1.93.0).
- Updated GitHub workflows (release.yml and test.yml) to read the Rust toolchain channel from the new configuration file.
- Adjusted Rust installation steps to utilize the specified toolchain from the configuration.

Made-with: Cursor
- Added functionality to migrate abbreviated commit hashes to their full SHA-1 equivalents in the database.
- Introduced helper functions to validate hash formats and execute git commands for hash resolution.
- Implemented tests to ensure correct migration behavior and deduplication of commit entries.

Made-with: Cursor
…isolation

- Add insert_commit_if_new (ON CONFLICT DO NOTHING) so onboarding does not
  overwrite hook-recorded diff stats; import_commits counts inserts directly.
- Extend dedup test to assert files_changed/insertions/deletions unchanged.
- Make identity_same and alias merge dedup ASCII case-insensitive for email/name.
- Use unique temp dirs for onboarding integration tests to avoid parallel flakes.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants