Mbed OS Socket test failures

[VeijoPesonen]

Description

Tested the driver by executing the Mbed OS greentea network test cases.

For further details about the tests please see

• mbed-os/TESTS/netsocket/README.md
• mbed-os/TESTS/network/wifi/README.md
• mbed-os/TESTS/network/interface/README.md

mbedgt: test case report:
| target            | platform_name | test suite              | test case                             | passed | failed | result | elapsed_time (sec) |
|-------------------|---------------|-------------------------|---------------------------------------|--------|--------|--------|--------------------|
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS                      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_CACHE                | 1      | 0      | OK     | 0.6                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_CANCEL               | 1      | 0      | OK     | 5.32               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 1      | 0      | OK     | 3.32               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_INVALID_HOST         | 1      | 0      | OK     | 1.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC  | 1      | 0      | OK     | 0.7                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS         | 1      | 0      | OK     | 1.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE   | 1      | 0      | OK     | 0.99               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT  | 1      | 0      | OK     | 4.62               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | ASYNCHRONOUS_DNS_TIMEOUTS             | 1      | 0      | OK     | 2.7                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | SYNCHRONOUS_DNS                       | 1      | 0      | OK     | 0.12               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | SYNCHRONOUS_DNS_INVALID               | 1      | 0      | OK     | 1.71               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-dns     | SYNCHRONOUS_DNS_MULTIPLE              | 1      | 0      | OK     | 0.6                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS                | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_INVALID        | 0      | 1      | FAIL   | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_NULL           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_PORT           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_PORT                   | 1      | 0      | OK     | 0.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_PORT_FAIL              | 0      | 1      | FAIL   | 0.09               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_UNOPENED               | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_BIND_WRONG_TYPE             | 0      | 1      | FAIL   | 0.08               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_CONNECT_INVALID             | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_ECHOTEST                    | 1      | 0      | OK     | 7.69               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 4.33               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK     | 4.42               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 3.12               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_ENDPOINT_CLOSE              | 0      | 1      | FAIL   | 0.7                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK     | 0.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.09               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.2                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_RECV_100K                   | 1      | 0      | OK     | 80.64              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_RECV_100K_NONBLOCK          | 1      | 0      | OK     | 82.92              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 1.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_SEND_REPEAT                 | 1      | 0      | OK     | 0.7                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_SEND_TIMEOUT                | 1      | 0      | OK     | 0.3                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_SETSOCKOPT_KEEPALIVE_VALID  | 0      | 1      | FAIL   | 0.6                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tcp     | TCPSOCKET_THREAD_PER_SOCKET_SAFETY    | 1      | 0      | OK     | 1.61               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_CONNECT_INVALID             | 1      | 0      | OK     | 0.8                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_ECHOTEST                    | 1      | 0      | OK     | 3.02               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 4.51               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK     | 4.43               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 3.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_ENDPOINT_CLOSE              | 1      | 0      | OK     | 0.97               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_HANDSHAKE_INVALID           | 1      | 0      | OK     | 0.41               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_NO_CERT                     | 1      | 0      | OK     | 0.4                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.01               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 1.31               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_SEND_CLOSED                 | 1      | 0      | OK     | 0.72               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_SEND_REPEAT                 | 1      | 0      | OK     | 1.38               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_SEND_TIMEOUT                | 1      | 0      | OK     | 0.91               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-tls     | TLSSOCKET_SEND_UNCONNECTED            | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS                | 1      | 0      | OK     | 0.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_INVALID        | 1      | 0      | OK     | 0.13               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_NULL           | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_PORT           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_PORT                   | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_PORT_FAIL              | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_UNOPENED               | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_BIND_WRONG_TYPE             | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_ECHOTEST                    | 1      | 0      | OK     | 2.32               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 194.29             |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_ECHOTEST_BURST_NONBLOCK     | 0      | 1      | FAIL   | 3.73               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 6.42               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.21               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.0                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 0.91               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_SENDTO_INVALID              | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_SENDTO_REPEAT               | 1      | 0      | OK     | 0.81               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-netsocket-udp     | UDPSOCKET_SENDTO_TIMEOUT              | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-interface | NETWORKINTERFACE_CONN_DISC_REPEAT     | 1      | 0      | OK     | 5.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-interface | NETWORKINTERFACE_STATUS               | 1      | 0      | OK     | 13.98              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-interface | NETWORKINTERFACE_STATUS_GET           | 1      | 0      | OK     | 11.13              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-interface | NETWORKINTERFACE_STATUS_NONBLOCK      | 1      | 0      | OK     | 5.14               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT                          | 1      | 0      | OK     | 6.02               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-DISCONNECT-REPEAT        | 1      | 0      | OK     | 12.14              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-NOCREDENTIALS            | 0      | 2      | FAIL   | 17.99              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-PARAMS-CHANNEL           | 1      | 0      | OK     | 0.08               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-PARAMS-CHANNEL-FAIL      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-PARAMS-NULL              | 0      | 2      | FAIL   | 30.17              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-PARAMS-VALID-SECURE      | 1      | 0      | OK     | 3.02               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-PARAMS-VALID-UNSECURE    | 1      | 0      | OK     | 3.01               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-SECURE                   | 1      | 0      | OK     | 2.02               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONNECT-SECURE-FAIL              | 1      | 0      | OK     | 17.94              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-CONSTRUCTOR                      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-GET-RSSI                         | 1      | 0      | OK     | 4.32               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-SCAN                             | 1      | 0      | OK     | 2.21               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-SCAN-NULL                        | 1      | 0      | OK     | 2.31               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-SET-CHANNEL                      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | tests-network-wifi      | WIFI-SET-CREDENTIAL                   | 0      | 7      | FAIL   | 0.28               |
mbedgt: test case results: 9 FAIL / 85 OK
mbedgt: completed in 876.77 sec

Version

ccbe1ed

Steps

mbed test --compile -t GCC_ARM -m GR_LYCHEE --app-config tools/test_configs/WiFiInterface.json -n tests-net*

mbedgt --test-spec=BUILD/tests/GR_LYCHEE/GCC_ARM/test_spec.json -V -n tests-netsocket-*,tests-network-wifi,tests-network-interface

Issue request type

[ ] Question
[ ] Enhancement
[X] Bug

[SeppoTakalo]

Please change the title to "Mbed OS Socket test failures"

[d-kato]

@VeijoPesonen @SeppoTakalo, I am checking the contents, but there was a test that I was worried about.
The following test "Expected return code" seems to be wrong. WIFI_SET_CREDENTIAL

Parameters	Password	Security	Expected return code
ssid="OK"	"12345678"	NSAPI_SECURITY_WEP	NSAPI_ERROR_OK or NSAPI_ERROR_UNSUPPORTED

The WEP only accepts passwords of 5 or 13 characters.
How about the following test.

Parameters	Password	Security	Expected return code
ssid="OK"	"12345"	NSAPI_SECURITY_WEP	NSAPI_ERROR_OK or NSAPI_ERROR_UNSUPPORTED

[SeppoTakalo]

WEP is 10 or 26 hexadecimal characters. Which is equivalent of 5 or 13 bytes in binary.

WifiInterface::set_credentials() take strings, so we can assume those to be hexadecimal characters.
Feeding in a binary would require length to be told also.

If this particular module accept only ascii character set, then it further limits the security of WEP by limiting the range of possible keys. I'm not sure if this is true.
Please check the data sheet.

If the device expects ascii, then convert hexadecimals to ascii, and hope that it would not break the AT parser of the receiving end.

[d-kato]

I understood. Assume the following.

• Character input (WEP-40: 5 characters, WEP-104: 13 characters).
• Hexadecimal character input (WEP-40: 10 characters, WEP-104: 26 characters).

So, in the case of 5 to 26 characters, return NSAPI_ERROR_OK.

[d-kato]

@VeijoPesonen @SeppoTakalo, Fix Mbed OS Socket test failures d897ebd

+-------------------+---------------+---------------------------------+---------------------------------------+--------+--------+--------+--------------------+
| target            | platform_name | test suite                      | test case                             | passed | failed | result | elapsed_time (sec) |
+-------------------+---------------+---------------------------------+---------------------------------------+--------+--------+--------+--------------------+
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS                      | 1      | 0      | OK     | 0.35               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_CACHE                | 1      | 0      | OK     | 0.64               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_CANCEL               | 1      | 0      | OK     | 5.33               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 1      | 0      | OK     | 3.51               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_INVALID_HOST         | 1      | 0      | OK     | 1.44               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC  | 1      | 0      | OK     | 1.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS         | 1      | 0      | OK     | 1.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE   | 1      | 0      | OK     | 0.94               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT  | 1      | 0      | OK     | 6.45               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | ASYNCHRONOUS_DNS_TIMEOUTS             | 1      | 0      | OK     | 2.83               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | SYNCHRONOUS_DNS                       | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | SYNCHRONOUS_DNS_INVALID               | 1      | 0      | OK     | 2.04               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-dns     | SYNCHRONOUS_DNS_MULTIPLE              | 1      | 0      | OK     | 0.84               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS                | 1      | 0      | OK     | 0.08               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_INVALID        | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_NULL           | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_ADDRESS_PORT           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_PORT                   | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_PORT_FAIL              | 1      | 0      | OK     | 0.04               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_UNOPENED               | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_BIND_WRONG_TYPE             | 1      | 0      | OK     | 0.07               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_CONNECT_INVALID             | 1      | 0      | OK     | 0.38               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_ECHOTEST                    | 1      | 0      | OK     | 8.32               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 9.44               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK     | 9.61               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 8.69               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_ENDPOINT_CLOSE              | 1      | 0      | OK     | 0.69               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK     | 0.07               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.04               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.22               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.04               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_RECV_100K                   | 1      | 0      | OK     | 7.85               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_RECV_100K_NONBLOCK          | 1      | 0      | OK     | 7.3                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 2.44               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_SEND_REPEAT                 | 1      | 0      | OK     | 0.97               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_SEND_TIMEOUT                | 1      | 0      | OK     | 0.51               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_SETSOCKOPT_KEEPALIVE_VALID  | 1      | 0      | OK     | 0.39               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tcp     | TCPSOCKET_THREAD_PER_SOCKET_SAFETY    | 1      | 0      | OK     | 5.18               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_CONNECT_INVALID             | 1      | 0      | OK     | 1.77               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_ECHOTEST                    | 1      | 0      | OK     | 12.01              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 13.07              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK     | 11.03              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 9.66               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_ENDPOINT_CLOSE              | 1      | 0      | OK     | 1.78               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_HANDSHAKE_INVALID           | 1      | 0      | OK     | 0.59               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_NO_CERT                     | 1      | 0      | OK     | 0.92               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.2                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 3.52               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_SEND_CLOSED                 | 1      | 0      | OK     | 1.74               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_SEND_REPEAT                 | 1      | 0      | OK     | 2.39               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_SEND_TIMEOUT                | 1      | 0      | OK     | 1.86               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-tls     | TLSSOCKET_SEND_UNCONNECTED            | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS                | 1      | 0      | OK     | 0.09               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_INVALID        | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_NULL           | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_ADDRESS_PORT           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_PORT                   | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_PORT_FAIL              | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_UNOPENED               | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_BIND_WRONG_TYPE             | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_ECHOTEST                    | 1      | 0      | OK     | 7.97               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_ECHOTEST_BURST              | 1      | 0      | OK     | 15.42              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_ECHOTEST_BURST_NONBLOCK     | 1      | 0      | OK     | 9.39               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_ECHOTEST_NONBLOCK           | 1      | 0      | OK     | 8.79               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_OPEN_CLOSE_REPEAT           | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_OPEN_DESTRUCT               | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_OPEN_LIMIT                  | 1      | 0      | OK     | 0.22               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_OPEN_TWICE                  | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_RECV_TIMEOUT                | 1      | 0      | OK     | 1.25               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_SENDTO_INVALID              | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_SENDTO_REPEAT               | 1      | 0      | OK     | 0.75               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-netsocket-udp     | UDPSOCKET_SENDTO_TIMEOUT              | 1      | 0      | OK     | 0.15               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-interface | NETWORKINTERFACE_CONN_DISC_REPEAT     | 1      | 0      | OK     | 6.14               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-interface | NETWORKINTERFACE_STATUS               | 1      | 0      | OK     | 13.38              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-interface | NETWORKINTERFACE_STATUS_GET           | 1      | 0      | OK     | 12.13              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-interface | NETWORKINTERFACE_STATUS_NONBLOCK      | 1      | 0      | OK     | 7.13               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT                          | 1      | 0      | OK     | 6.09               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-DISCONNECT-REPEAT        | 1      | 0      | OK     | 12.19              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-NOCREDENTIALS            | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-PARAMS-CHANNEL           | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-PARAMS-CHANNEL-FAIL      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-PARAMS-NULL              | 1      | 0      | OK     | 0.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-PARAMS-VALID-SECURE      | 1      | 0      | OK     | 2.11               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-PARAMS-VALID-UNSECURE    | 1      | 0      | OK     | 2.08               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-SECURE                   | 1      | 0      | OK     | 1.05               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONNECT-SECURE-FAIL              | 1      | 0      | OK     | 15.05              |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-CONSTRUCTOR                      | 1      | 0      | OK     | 0.06               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-GET-RSSI                         | 1      | 0      | OK     | 4.27               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-SCAN                             | 1      | 0      | OK     | 1.77               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-SCAN-NULL                        | 1      | 0      | OK     | 6.02               |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-SET-CHANNEL                      | 1      | 0      | OK     | 0.1                |
| GR_LYCHEE-GCC_ARM | GR_LYCHEE     | mbed-os-tests-network-wifi      | WIFI-SET-CREDENTIAL                   | 1      | 0      | OK     | 0.05               |
+-------------------+---------------+---------------------------------+---------------------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 94 OK
mbedgt: completed in 470.74 sec

Sometimes the connection to the echo server does not work well.
It may not matter, but changing line 37 of tcpsoket_echotest.cpp looks like it has been improved.

TCPSocket sock;
->
static TCPSocket sock;

[SeppoTakalo]

Excellent work.

@VeijoPesonen Can you verify the results in our network as well.

[VeijoPesonen]

Seems to be that there are still some issues left. Of course it might just be that the connection to AP is bad enough.

[1555030976.98][GLRM][INF] remote resources reset...
[1555030980.36][GLRM][TXD] mbedmbedmbedmbedmbedmbedmbedmbedmbedmbed
[1555030980.36][CONN][INF] sending up to 10 __sync packets (specified with --sync=10)
[1555030980.36][CONN][INF] sending preamble 'ad6c6a27-c7b7-4838-8750-51e6347e9569'
[1555030980.48][GLRM][TXD] {{__sync;ad6c6a27-c7b7-4838-8750-51e6347e9569}}
[1555030980.61][CONN][RXD] mbedmbedmbedmbedmbedmbedmbedmbed
[1555030980.61][CONN][INF] found SYNC in stream: {{__sync;ad6c6a27-c7b7-4838-8750-51e6347e9569}} it is #0 sent, queued...
[1555030980.61][CONN][INF] found KV pair in stream: {{__version;1.3.0}}, queued...
[1555030980.61][CONN][INF] found KV pair in stream: {{__timeout;360}}, queued...
[1555030980.61][HTST][INF] sync KV found, uuid=ad6c6a27-c7b7-4838-8750-51e6347e9569, timestamp=1555030980.609222
[1555030980.61][HTST][INF] DUT greentea-client version: 1.3.0
[1555030980.61][HTST][INF] setting timeout to: 360 sec
[1555030980.71][CONN][RXD] >>> Running 16 test cases...
[1555030980.71][CONN][INF] found KV pair in stream: {{__host_test_name;default_auto}}, queued...
[1555030980.71][HTST][INF] host test class: '<class 'mbed_os_tools.test.host_tests.default_auto.DefaultAuto'>'
[1555030980.71][HTST][INF] host test setup() call...
[1555030980.71][HTST][INF] CALLBACKs updated
[1555030980.71][HTST][INF] host test detected: default_auto
[1555030980.81][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONSTRUCTOR}}, queued...
[1555030980.81][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-NOCREDENTIALS}}, queued...
[1555030980.81][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-SET-CREDENTIAL}}, queued...
[1555030980.91][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-SET-CHANNEL}}, queued...
[1555030980.91][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-NULL}}, queued...
[1555030981.01][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-SCAN-NULL}}, queued...
[1555030981.01][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-SCAN}}, queued...
[1555030981.01][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-GET-RSSI}}, queued...
[1555030981.11][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-VALID-UNSECURE}}, queued...
[1555030981.11][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT}}, queued...
[1555030981.21][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-DISCONNECT-REPEAT}}, queued...
[1555030981.21][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-VALID-SECURE}}, queued...
[1555030981.31][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-CHANNEL}}, queued...
[1555030981.31][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-CHANNEL-FAIL}}, queued...
[1555030981.41][CONN][RXD] 
[1555030981.41][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-SECURE}}, queued...
[1555030981.41][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-SECURE-FAIL}}, queued...
[1555030981.51][CONN][RXD] >>> Running case #1: 'WIFI-CONSTRUCTOR'...
[1555030981.51][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONSTRUCTOR}}, queued...
[1555030981.61][CONN][RXD] >>> 'WIFI-CONSTRUCTOR': 1 passed, 0 failed
[1555030981.61][CONN][RXD] 
[1555030981.61][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONSTRUCTOR;1;0}}, queued...
[1555030981.71][CONN][RXD] >>> Running case #2: 'WIFI-CONNECT-NOCREDENTIALS'...
[1555030981.71][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-NOCREDENTIALS}}, queued...
[1555030981.81][CONN][RXD] >>> 'WIFI-CONNECT-NOCREDENTIALS': 1 passed, 0 failed
[1555030981.81][CONN][RXD] 
[1555030981.81][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-NOCREDENTIALS;1;0}}, queued...
[1555030981.91][CONN][RXD] >>> Running case #3: 'WIFI-SET-CREDENTIAL'...
[1555030981.91][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-SET-CREDENTIAL}}, queued...
[1555030982.01][CONN][RXD] >>> 'WIFI-SET-CREDENTIAL': 1 passed, 0 failed
[1555030982.01][CONN][RXD] 
[1555030982.01][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-SET-CREDENTIAL;1;0}}, queued...
[1555030982.11][CONN][RXD] >>> Running case #4: 'WIFI-SET-CHANNEL'...
[1555030982.11][CONN][RXD] :35::IGNORE: set_channel() not supported
[1555030982.12][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-SET-CHANNEL}}, queued...
[1555030982.22][CONN][RXD] >>> 'WIFI-SET-CHANNEL': 1 passed, 0 failed
[1555030982.22][CONN][RXD] 
[1555030982.22][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-SET-CHANNEL;1;0}}, queued...
[1555030982.31][CONN][RXD] >>> Running case #5: 'WIFI-CONNECT-PARAMS-NULL'...
[1555030982.31][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-NULL}}, queued...
[1555030982.42][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-NULL': 1 passed, 0 failed
[1555030982.42][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-NULL;1;0}}, queued...
[1555030982.56][CONN][RXD] 
[1555030982.56][CONN][RXD] >>> Running case #6: 'WIFI-SCAN-NULL'...
[1555030982.56][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-SCAN-NULL}}, queued...
[1555030989.25][CONN][RXD] >>> 'WIFI-SCAN-NULL': 1 passed, 0 failed
[1555030989.25][CONN][RXD] 
[1555030989.25][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-SCAN-NULL;1;0}}, queued...
[1555030989.35][CONN][RXD] >>> Running case #7: 'WIFI-SCAN'...
[1555030989.35][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-SCAN}}, queued...
[1555030991.66][CONN][RXD] >>> 'WIFI-SCAN': 1 passed, 0 failed
[1555030991.66][CONN][RXD] 
[1555030991.66][CONN][RXD] >>> Running case #8: 'WIFI-GET-RSSI'...
[1555030991.66][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-SCAN;1;0}}, queued...
[1555030991.76][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-GET-RSSI}}, queued...
[1555031008.74][CONN][RXD] :36::FAIL: Expected 0 Was -3004
[1555031011.75][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031011.85][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-GET-RSSI;0;2}}, queued...
[1555031011.95][CONN][RXD] >>> 'WIFI-GET-RSSI': 0 passed, 2 failed with reason 'Test Cases Failed'
[1555031011.95][CONN][RXD] 
[1555031011.95][CONN][RXD] >>> Running case #9: 'WIFI-CONNECT-PARAMS-VALID-UNSECURE'...
[1555031012.05][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-VALID-UNSECURE}}, queued...
[1555031027.12][CONN][RXD] :31::FAIL: Expected 0 Was -3004
[1555031027.12][CONN][RXD] :32::FAIL: Expected 0 Was -3004
[1555031030.04][CONN][RXD] :33::FAIL: Expected 0 Was -3010
[1555031030.14][CONN][RXD] :34::FAIL: Expected 0 Was -3004
[1555031030.14][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-VALID-UNSECURE;0;4}}, queued...
[1555031030.24][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-VALID-UNSECURE': 0 passed, 4 failed with reason 'Test Cases Failed'
[1555031030.24][CONN][RXD] 
[1555031030.34][CONN][RXD] >>> Running case #10: 'WIFI-CONNECT'...
[1555031030.34][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT}}, queued...
[1555031045.41][CONN][RXD] :37::FAIL: Expected 0 Was -3004
[1555031045.41][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031048.42][CONN][RXD] :41::FAIL: Expected 0 Was -3010
[1555031048.42][CONN][RXD] :42::FAIL: Expected 0 Was -3004
[1555031063.70][CONN][RXD] :47::FAIL: Expected 0 Was -3004
[1555031063.79][CONN][RXD] :48::FAIL: Expected 0 Was -3004
[1555031063.79][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT;0;6}}, queued...
[1555031063.90][CONN][RXD] >>> 'WIFI-CONNECT': 0 passed, 6 failed with reason 'Test Cases Failed'
[1555031063.90][CONN][RXD] 
[1555031063.99][CONN][RXD] >>> Running case #11: 'WIFI-CONNECT-DISCONNECT-REPEAT'...
[1555031063.99][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-DISCONNECT-REPEAT}}, queued...
[1555031066.51][CONN][RXD] :38::FAIL: Expected 0 Was -3010
[1555031066.51][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031081.58][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031081.58][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031084.59][CONN][RXD] :38::FAIL: Expected 0 Was -3010
[1555031084.59][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031115.51][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031115.61][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031146.62][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031146.62][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031177.71][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031177.71][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031192.73][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031192.73][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031195.75][CONN][RXD] :38::FAIL: Expected 0 Was -3010
[1555031195.75][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031210.81][CONN][RXD] :38::FAIL: Expected 0 Was -3004
[1555031210.81][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031213.82][CONN][RXD] :38::FAIL: Expected 0 Was -3010
[1555031213.82][CONN][RXD] :40::FAIL: Expected 0 Was -3004
[1555031213.92][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-DISCONNECT-REPEAT;0;20}}, queued...
[1555031214.03][CONN][RXD] >>> 'WIFI-CONNECT-DISCONNECT-REPEAT': 0 passed, 20 failed with reason 'Test Cases Failed'
[1555031214.03][CONN][RXD] 
[1555031214.03][CONN][RXD] >>> Running case #12: 'WIFI-CONNECT-PARAMS-VALID-SECURE'...
[1555031214.12][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-VALID-SECURE}}, queued...
[1555031217.24][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-VALID-SECURE': 1 passed, 0 failed
[1555031217.24][CONN][RXD] 
[1555031217.24][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-VALID-SECURE;1;0}}, queued...
[1555031217.34][CONN][RXD] >>> Running case #13: 'WIFI-CONNECT-PARAMS-CHANNEL'...
[1555031217.34][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-CHANNEL}}, queued...
[1555031217.44][CONN][RXD] :33::IGNORE: set_channel() not supported
[1555031217.44][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-CHANNEL;1;0}}, queued...
[1555031217.54][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-CHANNEL': 1 passed, 0 failed
[1555031217.54][CONN][RXD] 
[1555031217.54][CONN][RXD] >>> Running case #14: 'WIFI-CONNECT-PARAMS-CHANNEL-FAIL'...
[1555031217.64][CONN][RXD] :33::IGNORE: set_channel() not supported
[1555031217.64][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-CHANNEL-FAIL}}, queued...
[1555031217.74][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-CHANNEL-FAIL;1;0}}, queued...
[1555031217.84][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-CHANNEL-FAIL': 1 passed, 0 failed
[1555031217.84][CONN][RXD] 
[1555031217.84][CONN][RXD] >>> Running case #15: 'WIFI-CONNECT-SECURE'...
[1555031217.94][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-SECURE}}, queued...
[1555031219.95][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-SECURE;1;0}}, queued...
[1555031220.05][CONN][RXD] >>> 'WIFI-CONNECT-SECURE': 1 passed, 0 failed
[1555031220.05][CONN][RXD] 
[1555031220.05][CONN][RXD] >>> Running case #16: 'WIFI-CONNECT-SECURE-FAIL'...
[1555031220.15][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-SECURE-FAIL}}, queued...
[1555031235.21][CONN][RXD] >>> 'WIFI-CONNECT-SECURE-FAIL': 1 passed, 0 failed
[1555031235.21][CONN][RXD] 
[1555031235.21][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-SECURE-FAIL;1;0}}, queued...
[1555031235.31][CONN][RXD] >>> Test cases: 12 passed, 4 failed with reason 'Test Cases Failed'
[1555031235.31][CONN][RXD] >>> TESTS FAILED!
[1555031235.31][CONN][INF] found KV pair in stream: {{__testcase_summary;12;4}}, queued...
[1555031235.41][CONN][INF] found KV pair in stream: {{max_heap_usage;1748}}, queued...
[1555031235.41][CONN][INF] found KV pair in stream: {{reserved_heap;1851960}}, queued...
[1555031235.41][CONN][INF] found KV pair in stream: {{__thread_info;"0x0200230d8",2716,4096}}, queued...
[1555031235.41][HTST][ERR] orphan event in main phase: {{max_heap_usage;1748}}, timestamp=1555031235.412957
[1555031235.41][HTST][ERR] orphan event in main phase: {{reserved_heap;1851960}}, timestamp=1555031235.412963
[1555031235.41][HTST][ERR] orphan event in main phase: {{__thread_info;"0x0200230d8",2716,4096}}, timestamp=1555031235.412965
[1555031235.51][CONN][INF] found KV pair in stream: {{__thread_info;"0x020023228",168,512}}, queued...
[1555031235.51][CONN][INF] found KV pair in stream: {{__thread_info;"0x0200231e4",104,768}}, queued...
[1555031235.51][HTST][ERR] orphan event in main phase: {{__thread_info;"0x020023228",168,512}}, timestamp=1555031235.513171
[1555031235.51][HTST][ERR] orphan event in main phase: {{__thread_info;"0x0200231e4",104,768}}, timestamp=1555031235.513177
[1555031235.61][CONN][RXD] {{__cpu_info        up time;253072032}}
[1555031235.61][CONN][RXD] {{__cpu_info     sleep time;247305536}}
[1555031235.61][CONN][RXD] {{__cpu_info deepsleep time;0}}
[1555031235.71][CONN][RXD] {{__cpu_info  %  sleep/deep;97;0}}
[1555031235.71][CONN][INF] found KV pair in stream: {{end;failure}}, queued...
[1555031235.71][CONN][INF] found KV pair in stream: {{__exit;0}}, queued...
[1555031235.71][HTST][INF] __exit(0)
[1555031235.72][HTST][INF] __notify_complete(False)
[1555031235.72][HTST][INF] __exit_event_queue received
[1555031235.72][HTST][INF] test suite run finished after 255.11 sec...

[michalpasztamobica]

Small update: I cannot see the WIFI failures on my desk and I see that now they fail irregularly in the nightly runs, therefore I assume this might be a connectivity issue, rather than a SW bug.

New observation: Both the nightly tests and my local set up show a different mbed socket tests failure at the moment in tests-netsocket-udp:

[1560843500.22][CONN][RXD] >>> Running case #16: 'UDPSOCKET_ECHOTEST_NONBLOCK'...
[1560843500.23][CONN][INF] found KV pair in stream: {{__testcase_start;UDPSOCKET_ECHOTEST_NONBLOCK}}, queued...
[1560843532.17][CONN][RXD] Packets sent: 25, packets received 15, loss ratio 0.40
mbedgt: :205::FAIL: Values Not Within Delta
[1560843532.17][CONN][RXD] :205::FAIL: Values Not Within Delta
...
[1560843541.10][CONN][INF] found KV pair in stream: {{__testcase_finish;UDPSOCKET_SENDTO_REPEAT;1;0}}, queued...
[1560843541.11][CONN][RXD] >>> Running case #19: 'UDPSOCKET_ECHOTEST'...
[1560843541.11][CONN][INF] found KV pair in stream: {{__testcase_start;UDPSOCKET_ECHOTEST}}, queued...
[1560843542.17][CONN][RXD] [Round#15 - Receiver] error, returned 512
[1560843542.21][CONN][RXD] [Round#15 - Receiver] error, returned 88
[1560843542.29][CONN][RXD] [Round#15 - Receiver] error, returned 512
[1560843542.33][CONN][RXD] [Round#16 - Receiver] error, returned 88
[1560843542.40][CONN][RXD] [Round#16 - Receiver] error, returned 512
[1560843542.46][CONN][RXD] [Round#16 - Receiver] error, returned 88
[1560843542.53][CONN][RXD] [Round#17 - Receiver] error, returned 512
[1560843542.61][CONN][RXD] [Round#17 - Receiver] error, returned 188
[1560843542.69][CONN][RXD] [Round#17 - Receiver] error, returned 512
[1560843542.77][CONN][RXD] [Round#18 - Receiver] error, returned 188
[1560843542.84][CONN][RXD] [Round#18 - Receiver] error, returned 512
[1560843542.90][CONN][RXD] [Round#18 - Receiver] error, returned 188
[1560843542.97][CONN][RXD] [Round#19 - Receiver] error, returned 512
[1560843543.03][CONN][RXD] [Round#19 - Receiver] error, returned 288
[1560843543.10][CONN][RXD] [Round#19 - Receiver] error, returned 512
[1560843543.20][CONN][RXD] [Round#20 - Receiver] error, returned 288
[1560843543.31][CONN][RXD] [Round#20 - Receiver] error, returned 512
[1560843543.38][CONN][RXD] [Round#20 - Receiver] error, returned 288
[1560843543.51][CONN][RXD] [Round#21 - Receiver] error, returned 512
[1560843543.61][CONN][RXD] [Round#21 - Receiver] error, returned 388
[1560843543.72][CONN][RXD] [Round#21 - Receiver] error, returned 512
[1560843543.72][CONN][RXD] Packets sent: 36, packets received 15, loss ratio 0.58

I found that this commit is causing the failures: 74b5475

[michalpasztamobica]

Perhaps I made a mistake when bisecting. This change looks way more suspicious: 9817dc0 .
@d-kato , would you have a look at this, please?

[d-kato]

@michalpasztamobica , thank you. I have confirmed the problem.
I think that the cause is that by dividing the transmission, the received data is also divided.
Fixed: 4488aae

GR_LYCHEE has no problem because it uses RTS and CTS terminals, but in the case of a board that does not use CTS, data overflows. It has been fixed as follows:

1. When using CTS, 2048 bytes of data are sent at one time.
2. When not using CTS, it is divided into 512 bytes and sent.

I think that it is better than not being able to send even if it is divided.