client: implement set_db_change_aware

[halfcrazy]

fix monitor losing event if db schema change, and reconnect/inactivity probe cannot handle

[halfcrazy]

https://github.com/openvswitch/ovs/blob/fed82453a0a42085cb9c04c3e7d74ae6199b870e/ovsdb/jsonrpc-server.c#L761-L786

/* Makes all of the JSON-RPC sessions managed by 'remote' disconnect.  (They
 * will then generally reconnect.).  'comment' should be a human-readable
 * explanation of the reason for disconnection, for use in log messages, or
 * NULL to suppress logging.
 *
 * If 'force' is true, disconnects all sessions.  Otherwise, disconnects only
 * sesions that aren't database change aware. */
static void
ovsdb_jsonrpc_session_reconnect_all(struct ovsdb_jsonrpc_remote *remote,
                                    bool force, const char *comment)
{
    struct ovsdb_jsonrpc_session *s;

    LIST_FOR_EACH_SAFE (s, node, &remote->sessions) {
        if (force || !s->db_change_aware) {
            jsonrpc_session_force_reconnect(s->js);
            if (comment && jsonrpc_session_is_connected(s->js)) {
                VLOG_INFO("%s: disconnecting (%s)",
                          jsonrpc_session_get_name(s->js), comment);
            }
            jsonrpc_session_disconnect(s->js); // <--------- i think we need to disconnect immediatly
            if (!jsonrpc_session_is_alive(s->js)) {
                ovsdb_jsonrpc_session_close(s);
            }
        }
    }
}

https://github.com/openvswitch/ovs/blob/fed82453a0a42085cb9c04c3e7d74ae6199b870e/Documentation/ref/ovsdb-server.7.rst?plain=1#L516-L519

If the conversion is successful, the server notifies clients that use the
``set_db_change_aware`` RPC introduced in Open vSwitch 2.9 and cancels their
outstanding transactions and monitors.  The server disconnects other clients,
enabling them to notice the change when they reconnect.  

@dceara could you take a look from ovs side? I found ovsdb stopped sending monitor events, but the connection was not disconnected.

[dceara]

Hi @halfcrazy,

https://github.com/openvswitch/ovs/blob/fed82453a0a42085cb9c04c3e7d74ae6199b870e/ovsdb/jsonrpc-server.c#L761-L786

/* Makes all of the JSON-RPC sessions managed by 'remote' disconnect.  (They
 * will then generally reconnect.).  'comment' should be a human-readable
 * explanation of the reason for disconnection, for use in log messages, or
 * NULL to suppress logging.
 *
 * If 'force' is true, disconnects all sessions.  Otherwise, disconnects only
 * sesions that aren't database change aware. */
static void
ovsdb_jsonrpc_session_reconnect_all(struct ovsdb_jsonrpc_remote *remote,
                                    bool force, const char *comment)
{
    struct ovsdb_jsonrpc_session *s;

    LIST_FOR_EACH_SAFE (s, node, &remote->sessions) {
        if (force || !s->db_change_aware) {
            jsonrpc_session_force_reconnect(s->js);
            if (comment && jsonrpc_session_is_connected(s->js)) {
                VLOG_INFO("%s: disconnecting (%s)",
                          jsonrpc_session_get_name(s->js), comment);
            }
            jsonrpc_session_disconnect(s->js); // <--------- i think we need to disconnect immediatly

Yes, if the client didn't have "db_change_aware" set, due to jsonrpc_session_force_reconnect(), the server should've disconnected in the next jsonrpc_session_run() already without the need to immediately disconnect here. See:

https://github.com/openvswitch/ovs/blob/fed82453a0a42085cb9c04c3e7d74ae6199b870e/lib/jsonrpc.c#L1100

        if (!jsonrpc_session_is_alive(s->js)) {
            ovsdb_jsonrpc_session_close(s);
        }
    }
}

}

https://github.com/openvswitch/ovs/blob/fed82453a0a42085cb9c04c3e7d74ae6199b870e/Documentation/ref/ovsdb-server.7.rst?plain=1#L516-L519

If the conversion is successful, the server notifies clients that use the
set_db_change_aware RPC introduced in Open vSwitch 2.9 and cancels their
outstanding transactions and monitors. The server disconnects other clients,
enabling them to notice the change when they reconnect.

@dceara could you take a look from ovs side? I found ovsdb stopped sending monitor events, but the connection was not disconnected.

I don't see anything obviously wrong in OVS. Maybe @igsilya is a better person to ask an OVSDB expert. Also, logs of what went on might help identify any issues there might be.

Thanks,
Dumitru

[igsilya]

@dceara could you take a look from ovs side? I found ovsdb stopped sending monitor events, but the connection was not disconnected.

I don't see anything obviously wrong in OVS. Maybe @igsilya is a better person to ask an OVSDB expert. Also, logs of what went on might help identify any issues there might be.

The most likely cause is that the client didn't send a new monitor request after the monitor_canceled notification. You need to check what was the actual sequence of messages after the monitor_canceled .

[igsilya]

FWIW, I'm also not sure what this PR is trying to achieve, as implementing change awareness via disconnection doesn't make much sense to me.

[halfcrazy]

@dceara could you take a look from ovs side? I found ovsdb stopped sending monitor events, but the connection was not disconnected.

I don't see anything obviously wrong in OVS. Maybe @igsilya is a better person to ask an OVSDB expert. Also, logs of what went on might help identify any issues there might be.

The most likely cause is that the client didn't send a new monitor request after the monitor_canceled notification. You need to check what was the actual sequence of messages after the monitor_canceled .

Background is that this lib does not implement monitor_canceled and set_db_change_aware. So this belongs to other clients. The problem I'm trying to resolve is that after a DB schema change(eg, add a new table), the client monitor does not receive any new events, and local cache data is stale.

So refer to If the conversion is successful, the server notifies clients that use the set_db_change_aware RPC introduced in Open vSwitch 2.9 and cancels their outstanding transactions and monitors. The server disconnects other clients, enabling them to notice the change when they reconnect.

The client using this lib should be disconnected from the server. But from the pcap i don't see a fin or reset. (The echo jsonrpc call still works)

{"error":null,"result":["libovsdb echo"],"id":4}
{"method":"echo","params":["libovsdb echo"],"id":5}

{"error":null,"result":["libovsdb echo"],"id":5}{"method":"update3","params":[{"databaseName":"OVN_Northbound","id":"748d3761-bd95-4930-9c82-aad57440634a"},"458f8ef0-701a-4764-bd72-0dfab9b27722",{"Logical_Switch_Port":{"4d465e2b-5f1d-4223-bc1b-48abde6c2384":{"insert":{"name":"yz-p2"}}}}],"id":null}{"method":"update3","params":[{"databaseName":"OVN_Northbound","id":"748d3761-bd95-4930-9c82-aad57440634a"},"08ecc422-0845-450c-b6b7-db99ffa3862f",{"Logical_Switch_Port":{"4d465e2b-5f1d-4223-bc1b-48abde6c2384":{"modify":{"up":false}}}}],"id":null}
{"method":"echo","params":["libovsdb echo"],"id":6}

{"error":null,"result":["libovsdb echo"],"id":6}
{"method":"echo","params":["libovsdb echo"],"id":7}

{"error":null,"result":["libovsdb echo"],"id":7}
{"method":"echo","params":["libovsdb echo"],"id":8}

{"error":null,"result":["libovsdb echo"],"id":8}

reproduce:
1 start monitor client
2 insert a lsp yz-p2
3 ovsdb-client -t 30 convert tcp:10.255.161.217:6641 /usr/share/ovn/ovn-nb.ovsschema
4 then insert a new lsp yz-p3 // not receive an update event and tcp not disconnected

pcapnot-disconnect.zip