Skip to content

Latest commit

 

History

History
294 lines (171 loc) · 84.5 KB

File metadata and controls

294 lines (171 loc) · 84.5 KB

Embedding MeshSync An all-in-one model of Meshery deployment

Status Draft
Date 2025-05-01
Authors Nikita B Lee Calcote Rafayel Juwe…
Reviewers [Add your name here] …

Proposal Overview 2

Summary 2

Motivation 2

Current Situation 2

Proposed Enhancement 2

Proposal Pros and Cons 2

Implementation 3

MeshSync 3

Meshery Operator 3

Meshery Server 3

Changes needed for MeshSync function as a library 3

Add an option to Meshery to call run MeshSync on start 4

MeshSync Messaging Protocol 4

Kubernetes Connections 4

Meshery CLI 4

Meshery UI 4

Upgrading / Scaling Meshery Deployments 4

Rationale / Alternatives Considered 4

Impact / Risks 4

Unresolved Questions 5

Boot Time or Connection Time? 5

Access to the k8s cluster 5

Appendices 5

Proposal Overview {#proposal-overview}

Summary {#summary}

With a vigil eye to reducing time-to-value for initial user deployment of Meshery, this design specification proposes an all-in-one model of Meshery deployment, in which clusters quickly come under management and their resource details are continuously synchronized without the need for administrative or cluster-level permissions on the users’ behalf.

Motivation {#motivation}

User attention span is short. This is particularly true for new, would-be Meshery users, who have yet to see the value in the solution (in a full deployment of Meshery). It is incumbent upon Meshery as a project to give users an option to be able quickly to try Meshery with their infrastructure.

Current Situation {#current-situation}

Currently, Meshery users have a number of configuration choices to make while deploying Meshery. These configuration choices enable users with different deployment models; e.g. “full” with all adapters enabled, “secure” with enforcement of use of a single provider, and so on.

As of Meshery v0.8, if a user would like to do this, they need to install and deploy Meshery Operator, MeshSync, Broker, and CRDs. Challenges with this scaled mode of deployment include:

  1. Permissions to install cluster-level resources (Meshery Operator and CRDs).
  2. Time necessary to install and time to configure and troubleshoot failed deployments.

Proposed Enhancement {#proposed-enhancement}

Allow users to immediately have information about cluster resources without need to install and deploy Meshery Operator, MeshSync, Broker, and CRDs by:

supporting an embedded mode of MeshSync deployment or in other words, supporting a new mode of Kubernetes connection.

Proposal Pros and Cons {#proposal-pros-and-cons}

Benefits of an an all-in-one deployment model are:

  1. Reduced cognitive overhead by lessening the number of moving parts.
  2. Elimination of part of all of the need for administrative privileges over the cluster.
  3. Overall a reduced time-to-value.

Drawbacks of an all-in-one deployment model are:

  1. The Meshery Server container and its single process undertakes more load, potentially causing performance degradation and poor UX.
  2. “Demo” or “simple” mode deployments often find themselves in Production situations; A path to upgrading or scaling deployment modes is needed.

Implementation {#implementation}

Any number of Meshery components will be affected by this proposed change. Therefore, all components need to be considered.

  • Add new package under meshsync/pkg/embedded  
    
  • Use client-go with a set of informers or List calls to gather resources  
    
  • Collect: Pods, Deployments, Services, Ingresses, CRDs, Namespaces, etc.  
    
  • Inject EmbeddedMeshSync service into Meshery Core
    

MeshSync {#meshsync}

Support one-time snapshot with MeshSync data written to local file\
…todo
The proposed idea is to:

Define the Embedded Mode

Objective: Introduce a new deployment mode for MeshSync called “embedded” or “snapshot” mode, where MeshSync runs temporarily to capture cluster state and sync it with Meshery Server, injects EmbeddedMeshSync service into Meshery Core. then terminates.

***Characteristics**:*  
    *Read-only access to the Kubernetes cluster.*  
    *Temporary deployment, similar to kubectl-meshsync-snapshot.*  
    *No persistent connection to Meshery Server or Meshery Broker.*  
    *Outputs a snapshot (e.g., JSON or YAML) that Meshery can import for offline management.*

Potential Challenges

  • *Snapshot Scalability: Large clusters may generate large snapshots, requiring optimization in*    
    
                *MeshSync’s discovery logic.*
    
  • Snapshot Format: Ensure the snapshot format is compatible with Meshery’s database and UI

            *for seamless import.*
    
  • Operator Complexity: Adding Job-based deployment to the Meshery Operator may increase
    its complexity, requiring careful testing.

Meshery Operator {#meshery-operator}

…todo
Kubernetes Operator to manages the lifecycle of Meshery components like MeshSync and Meshery Broker,

1 Add Support for MeshSync Snapshot Mode
Introduce a new "mode" flag or CRD spec option for the Meshery Operator to trigger snapshot mode deployment:

*apiVersion: meshery.io/v1alpha1*  
*kind: MeshSync*  
*metadata:*  
  *name: meshsync-snapshot*  
*spec:*  
  *mode: snapshot*

2. Deploy MeshSync as a Kubernetes Job

*Instead of running MeshSync as a Deployment (long-running), the Operator would:*

    *Deploy it as a Job (temporary).*

    *Attach a read-only service account (minimal RBAC).*

    *Use emptyDir or ConfigMap to export data.*

    *Output the snapshot to a file or API endpoint (stdout, JSON file, etc).*

3. Skip CRD Installation

*In snapshot mode, the operator:*

    *Skips installing cluster-scoped resources like Meshery CRDs.*

    *Only deploys namespace-scoped, ephemeral components.*

4. Deployment Flow for Users

*Lightweight Installation: Users install just the Meshery Operator (no MeshSync/Broker by default)*

*Request Snapshot: Create a MeshSync CR with mode: snapshot*

*Automatic Cleanup: Operator removes resources after completion*

5. Addressing Potential Challenges

*Snapshot Scalability: Operator can implement chunking for large snapshots*

*Format Compatibility: Operator can transform data before final output*

*Complexity Management: Isolate snapshot logic in separate controllers*
Problem Solution by Enhanced Operator
Cluster permissions required Uses Job with read-only access; avoids cluster-scoped installs
Time to install Lightweight deployment using ephemeral job
Persistent components None; snapshot mode only
Format compatibility Operator ensures snapshot output is Meshery-compatible
Complexity of deployment One-liner mesheryctl support or CRD
Failures in CRDs and Broker Entirely bypassed in this mode

Meshery Server {#meshery-server}

With the implementation of this issue: Support one-time snapshot with MeshSync data written to local file there will be decoupling of MeshSync from nats.

The proposed idea is to:

  • Arrange meshsync to provide its functionality as library.
  • Add an integration test to Meshery Server which converts MeshSync integration to be sure that after updates the old functionality didn’t have regression.
  • Add an option to Meshery to call run MeshSync on start.
  • Update integration with MeshSync to be able to consume events from MeshSync library instead of NATs.

Changes needed for MeshSync function as a library {#changes-needed-for-meshsync-function-as-a-library}

  • Arrange what is now in main as a function which then could be called from the library client (most probably as a goroutine);
  • Keep existing idea of stop after N seconds (implemented in snapshot to local file), so that client code do not need to handle of termination of goroutine;
  • Add an output mode to a golang channel.

Add an option to Meshery to call run MeshSync on start {#add-an-option-to-meshery-to-call-run-meshsync-on-start}

Meshery Server needs to honor both environment variable and meshconfig property.
F.e. introduce new env variable COLECT_CLUSTER_SNAPSHOT_WITH_MeshSync=true;

MeshSync Messaging Protocol {#meshsync-messaging-protocol}

Consideration is to be given to the messaging protocol (and hopefully, not the messaging format) of events emitting from MeshSync and the mechanism by which Meshery Server consumes them. Currently, an enterprise message bus (NATS) is that mechanism: a subscription mechanism.

References

Details of implementation to be discussed . . .
Description to be given then.

Kubernetes Connections {#kubernetes-connections}

….todo

Meshery CLI {#meshery-cli}

New property in meshconfig? Enhancement to `system context`.
…todo

Meshery UI {#meshery-ui}

Understanding of the connection states and operations available to internal versus external Meshery Operator / MeshSync deployments.
todo

Upgrading / Scaling Meshery Deployments {#upgrading-/-scaling-meshery-deployments}

…todo

Rationale / Alternatives Considered {#rationale-/-alternatives-considered}

TODO: Assessment of necessary privilege level to install Meshery Operator/CRDs.

Impact / Risks {#impact-/-risks}

Risk: Regression of integration between Meshery server and MeshSync: Existing integration between Meshery server and MeshSync is not covered with integration tests.
Mitigation Plan: The proposed solution to implement an integration test before doing any refactoring. Add an integration test to test integration between Meshery Server and MeshSync to ensure there is no regression after implementation of this proposal.
Mitigation Plan: Complementary we could implement a solution form this proposal completely standalone without any update to existing integration with MeshSync.

Risk Mitigation
Divergence in logic between two modes Reuse data models and interfaces as much as possible
Incomplete resource discovery Document supported types and limitations clearly
Confusion between full vs. embedded mode UI/CLI should highlight active mode with descriptions

Unresolved Questions {#unresolved-questions}

Boot Time or Connection Time? {#boot-time-or-connection-time?}

Is this a boot time concern or a connection time concern?
[Lee] Despite my positioning of embedded MeshSync as an all-in-one deployment model, I will argue that it is easier for us to implement and more flexible for the user to have this functionality exposed as a configurable option of every individual Kubernetes’ connection.

Access to the k8s cluster {#access-to-the-k8s-cluster}

The current approach of running MeshSync as a binary suggests that you run it on your local environment, and MeshSync can get access to .kube/config.
Hence the approach suggested in this proposal will work if Meshery server is run on the local environment (with access to .kube/config file).
If Meshery is running in the k8s cluster itself, we will need to consider how to provide access to MeshSync to the k8s cluster in question.

Appendices {#appendices}

todo