Skip to content

test - #1

Open
liliu-z wants to merge 1 commit into
masterfrom
li_test
Open

test#1
liliu-z wants to merge 1 commit into
masterfrom
li_test

Conversation

@liliu-z

@liliu-z liliu-z commented Jun 24, 2025

Copy link
Copy Markdown
Owner

No description provided.

Comment thread configs/advanced/load_guard.yaml Outdated

@zhengbuqian zhengbuqian Jun 25, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.2 does not mean 20% of resources reserved for cache, actually it means for X amount of evictable data, reserve at least 0.2 * X for cache swapping

Comment thread configs/advanced/load_guard.yaml Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.0 means before loading inevictable data, we can use memory/disk to cache swap evictable data, but inevictable data may eventually be loaded and use up node resources, causing evictable data to not be able to be loaded at all, which is kind of expected behavior if the user choose to configure as such

Signed-off-by: Li Liu <li.liu@zilliz.com>
std::atomic<int64_t> actual_size_;
bool evictable_;
std::atomic<bool> deep_loaded_{false};
uid_t uid_;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cell does not need the notion of uid.

~Cell() = default;

// Size management
int64_t GetEstimatedSize() const { return estimated_size_; }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simply use estimated size in this version, remove all attempts towards using actual size.

with this, we can compute and cache size of segment in ctor, so we don't need a lock in Segment's accessors.

}

bool SegmentLoadGuard::LoadSegment(std::shared_ptr<Segment> segment) {
if (!CanLoadSegment(*segment)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this entire CanLoadSegment, if so UpdateShallowUsage be guarded by a lock?

} CCellInfo;

// SegmentLoadGuard lifecycle
CSegmentLoadGuard NewSegmentLoadGuard(CLoadGuardConfig config);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have only a single load guard instance globally

// ResourceSyncConfig contains configuration for resource synchronization
type ResourceSyncConfig struct {
SyncInterval time.Duration
MemoryCalibInterval time.Duration

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memory calib is a task in the future, do not do that in this PR and rely solely on OS reported memory usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants