diff --git a/README.md b/README.md index b8fccfb..2e9dcc0 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,43 @@ Home Assistant SunPower Integration using the local installer ethernet interface Original Integration is [https://github.com/krbaker/hass-sunpower](https://github.com/krbaker/hass-sunpower) -* If this is a fork, please add what's different here and fix up the badges below +## 🆕 What's Different in This Fork + +This fork adds automatic LocalAPI support for newer PVS firmware (build >= 61840) while maintaining full backwards compatibility with legacy CGI endpoints. + +### Key Enhancements +- Automatic API Detection: Queries firmware version and automatically selects the appropriate API +- LocalAPI for Newer Firmware: Uses the more efficient Varserver FCGI endpoints with authentication +- Direct LAN Access: LocalAPI works via the standard LAN IP - no proxy or special network setup required +- Legacy CGI Fallback: Maintains support for older firmware using traditional CGI endpoints +- Improved Performance: Caching mechanism reduces API calls and improves response times +- Automatic Credential Management: Serial suffix auto-fetched from PVS, no manual configuration needed + +### How It Works +1. During setup, the integration queries `/cgi-bin/dl_cgi/supervisor/info` to check firmware version +2. If firmware build >= 61840: Uses LocalAPI with session-based authentication +3. If older firmware: Uses legacy CGI endpoints (same as before) +4. Serial suffix (last 5 characters of PVS serial) is auto-fetched when possible +5. Field name mapping: LocalAPI returns camelCase field names (e.g., `pMppt1Kw`), which are automatically converted to snake_case (e.g., `p_mppt1_kw`) to match legacy CGI format, ensuring identical data structures for backwards compatibility +6. No configuration changes needed - it just works! + +### Benefits +- Faster response times with caching (caches variable paths, not values - data is always fresh) +- More reliable session management +- Better error handling and retry logic +- Reduced load on PVS system +- Cache resets on Home Assistant restart to ensure optimal performance + +### Recent Bug Fixes +- **Fixed LocalAPI reliability issues**: Simplified /vars endpoint calls by removing cache parameter. While LocalAPI supports caching, it adds complexity and the performance benefit is negligible at 120-second polling intervals. This change improves reliability and eliminates potential cache state issues. +- **Fixed KeyError crashes**: Added defensive checks to prevent crashes when device data is temporarily unavailable +- **Improved error logging**: Added detailed diagnostics to help troubleshoot connection issues +- **Better resilience**: Integration now gracefully handles missing PVS, inverter, or meter data + +### LocalAPI Documentation +For technical details on the LocalAPI implementation, see: +- [LocalAPI Documentation](https://github.com/SunStrong-Management/pypvs/blob/main/doc/LocalAPI.md) - Comprehensive guide to Varserver FCGI endpoints +- [pypvs Project](https://github.com/SunStrong-Management/pypvs) - Python library for PVS LocalAPI access [![GitHub Release][releases-shield]][releases] [![GitHub Activity][commits-shield]][commits] @@ -80,19 +116,23 @@ removed (This addition thanks to [@CanisUrsa](https://github.com/CanisUrsa)) ## Options (available from 'configure' once integration is setup) +**To change polling intervals:** Settings → Devices & Services → SunPower → Configure + ### Solar data update interval (seconds) +**Default: 120 seconds** | **Minimum: 60 seconds** + This sets how fast the integration will try to get updated solar info from the PVS. -The lowest "safe" rate looks like about 120 seconds. I am concerned some PVSs may fail -to work properly over time and I'm guessing it might be request or error logging filling -their memory. I am running with 300 seconds right now as I went through a heck of a time -with a PVS that began to fail pushing to Sunpower's cloud. + +**For LocalAPI (firmware >= 61840):** The official documentation recommends polling "once every few seconds" to avoid overloading the PVS CPU. The default 120 seconds is conservative and safe for long-term reliability. + +**For Legacy CGI (older firmware):** The PVS takes a very long time to return data. The lowest "safe" rate is about 120 seconds. Some PVSs may fail to work properly over time with aggressive polling, possibly due to request or error logging filling their memory. ### Energy storage update interval (seconds) -Should evenly divide into Solar data update interval or be an even multiple of it (this is due to the -currently silly way polling is handled through one timer). The original author of the ESS addon -[@CanisUrsa](https://github.com/CanisUrsa) had it as low as 20 seconds (see warning above) +**Default: 60 seconds** | **Minimum: 20 seconds** + +Should evenly divide into Solar data update interval or be an even multiple of it (this is due to the currently silly way polling is handled through one timer). The original author of the ESS addon [@CanisUrsa](https://github.com/CanisUrsa) had it as low as 20 seconds (see warning above) ## Network Setup @@ -260,6 +300,29 @@ Power Output. If you file a bug one of the most useful things to include is the output of > curl +### LocalAPI Authentication Issues + +If you have newer firmware (build >= 61840) and see authentication errors: + +1. Check firmware version: + ```bash + curl http://172.27.153.1/cgi-bin/dl_cgi/supervisor/info + ``` + Look for the `BUILD` number in the response. + +2. Verify serial suffix: The integration automatically fetches the serial suffix (last 5 characters of PVS serial) from the PVS during initialization. If auto-fetch fails, the integration will raise an error. The serial suffix is used as the password for LocalAPI authentication. + +3. Check logs: Look in Home Assistant logs for connection errors or authentication failures during integration setup. + +### API Type Detection + +The integration automatically detects and uses the appropriate API based on firmware version: +- Firmware build >= 61840: Uses LocalAPI automatically +- Older firmware: Uses legacy CGI endpoints automatically +- No configuration needed - the integration handles this transparently + +Note on Firmware Upgrades: API detection occurs when Home Assistant starts or when the integration reloads. If SunPower remotely upgrades your PVS firmware while Home Assistant is running, the integration will continue using the current API until you restart Home Assistant. After restart, it will automatically detect the new firmware and switch to LocalAPI if supported. + ### Missing solar production. Appears that the Sunpower meter has disappeared from the device list Run the debugging command and look for the METER entries. diff --git a/custom_components/sunpower/__init__.py b/custom_components/sunpower/__init__.py index ba16edc..2e85f5a 100644 --- a/custom_components/sunpower/__init__.py +++ b/custom_components/sunpower/__init__.py @@ -73,7 +73,13 @@ def create_vmeter(data): freq_avg = sum(freq) / len(freq) if len(freq) > 0 else None volts_avg = sum(volts) / len(volts) if len(volts) > 0 else None - pvs_serial = next(iter(data[PVS_DEVICE_TYPE])) # only one PVS + # Check if PVS device exists before trying to access it + pvs_devices = data.get(PVS_DEVICE_TYPE) + if not pvs_devices: + _LOGGER.warning("PVS device not found in data, skipping virtual meter creation") + return data + + pvs_serial = next(iter(pvs_devices)) # only one PVS vmeter_serial = f"{pvs_serial}pv" data.setdefault(METER_DEVICE_TYPE, {})[vmeter_serial] = { "SERIAL": vmeter_serial, @@ -98,8 +104,33 @@ def create_vmeter(data): def convert_sunpower_data(sunpower_data): """Convert PVS data into indexable format data[device_type][serial]""" data = {} + + # Log total device count + total_devices = len(sunpower_data.get("devices", [])) + _LOGGER.info(f"Processing {total_devices} devices from API") + for device in sunpower_data["devices"]: - data.setdefault(device["DEVICE_TYPE"], {})[device["SERIAL"]] = device + device_type = device.get("DEVICE_TYPE", "UNKNOWN") + serial = device.get("SERIAL", "UNKNOWN") + data.setdefault(device_type, {})[serial] = device + + # Log device types found for debugging + device_types = list(data.keys()) + device_counts = {dt: len(data[dt]) for dt in device_types} + _LOGGER.info(f"Device types found: {device_counts}") + + # Check for expected device types + if not data.get(PVS_DEVICE_TYPE): + _LOGGER.error(f"CRITICAL: PVS device type '{PVS_DEVICE_TYPE}' not found!") + _LOGGER.error(f"Available device types: {device_types}") + # Log ALL devices to see what we're getting + for i, dev in enumerate(sunpower_data["devices"][:10]): # First 10 devices + _LOGGER.error(f"Device {i+1}: TYPE='{dev.get('DEVICE_TYPE')}', SERIAL={dev.get('SERIAL')}, MODEL={dev.get('MODEL')}") + + if not data.get(INVERTER_DEVICE_TYPE): + _LOGGER.warning(f"No inverter devices found - this is normal for some PVS configurations") + _LOGGER.info(f"Available device types: {device_types}") + _LOGGER.info("Inverter data may be aggregated in meter readings or unavailable via LocalAPI") create_vmeter(data) @@ -117,24 +148,40 @@ def convert_ess_data(ess_data, data): sunvault_power_inputs = [] sunvault_power_outputs = [] sunvault_state = "working" + + # Ensure device types exist in data + if BATTERY_DEVICE_TYPE not in data: + _LOGGER.warning("BATTERY_DEVICE_TYPE not found in data, skipping battery ESS data conversion") + data[BATTERY_DEVICE_TYPE] = {} + if ESS_DEVICE_TYPE not in data: + _LOGGER.warning("ESS_DEVICE_TYPE not found in data, skipping ESS data conversion") + data[ESS_DEVICE_TYPE] = {} + if HUBPLUS_DEVICE_TYPE not in data: + _LOGGER.warning("HUBPLUS_DEVICE_TYPE not found in data, skipping HubPlus data conversion") + data[HUBPLUS_DEVICE_TYPE] = {} + for device in ess_data["ess_report"]["battery_status"]: - data[BATTERY_DEVICE_TYPE][device["serial_number"]]["battery_amperage"] = device[ + serial = device["serial_number"] + if serial not in data[BATTERY_DEVICE_TYPE]: + _LOGGER.warning(f"Battery {serial} not found in PVS data, skipping") + continue + data[BATTERY_DEVICE_TYPE][serial]["battery_amperage"] = device[ "battery_amperage" ]["value"] - data[BATTERY_DEVICE_TYPE][device["serial_number"]]["battery_voltage"] = device[ + data[BATTERY_DEVICE_TYPE][serial]["battery_voltage"] = device[ "battery_voltage" ]["value"] - data[BATTERY_DEVICE_TYPE][device["serial_number"]]["customer_state_of_charge"] = device[ + data[BATTERY_DEVICE_TYPE][serial]["customer_state_of_charge"] = device[ "customer_state_of_charge" ]["value"] - data[BATTERY_DEVICE_TYPE][device["serial_number"]]["system_state_of_charge"] = device[ + data[BATTERY_DEVICE_TYPE][serial]["system_state_of_charge"] = device[ "system_state_of_charge" ]["value"] - data[BATTERY_DEVICE_TYPE][device["serial_number"]]["temperature"] = device["temperature"][ + data[BATTERY_DEVICE_TYPE][serial]["temperature"] = device["temperature"][ "value" ] - if data[BATTERY_DEVICE_TYPE][device["serial_number"]]["STATE"] != "working": - sunvault_state = data[BATTERY_DEVICE_TYPE][device["serial_number"]]["STATE"] + if data[BATTERY_DEVICE_TYPE][serial]["STATE"] != "working": + sunvault_state = data[BATTERY_DEVICE_TYPE][serial]["STATE"] sunvault_amperages.append(device["battery_amperage"]["value"]) sunvault_voltages.append(device["battery_voltage"]["value"]) sunvault_temperatures.append(device["temperature"]["value"]) @@ -155,75 +202,89 @@ def convert_ess_data(ess_data, data): sunvault_power_inputs.append(0) sunvault_power_outputs.append(0) for device in ess_data["ess_report"]["ess_status"]: - data[ESS_DEVICE_TYPE][device["serial_number"]]["enclosure_humidity"] = device[ + serial = device["serial_number"] + if serial not in data[ESS_DEVICE_TYPE]: + _LOGGER.warning(f"ESS {serial} not found in PVS data, skipping") + continue + data[ESS_DEVICE_TYPE][serial]["enclosure_humidity"] = device[ "enclosure_humidity" ]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["enclosure_temperature"] = device[ + data[ESS_DEVICE_TYPE][serial]["enclosure_temperature"] = device[ "enclosure_temperature" ]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["agg_power"] = device["ess_meter_reading"][ + data[ESS_DEVICE_TYPE][serial]["agg_power"] = device["ess_meter_reading"][ "agg_power" ]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_a_current"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_a_current"] = device[ "ess_meter_reading" ]["meter_a"]["reading"]["current"]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_a_power"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_a_power"] = device[ "ess_meter_reading" ]["meter_a"]["reading"]["power"]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_a_voltage"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_a_voltage"] = device[ "ess_meter_reading" ]["meter_a"]["reading"]["voltage"]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_b_current"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_b_current"] = device[ "ess_meter_reading" ]["meter_b"]["reading"]["current"]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_b_power"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_b_power"] = device[ "ess_meter_reading" ]["meter_b"]["reading"]["power"]["value"] - data[ESS_DEVICE_TYPE][device["serial_number"]]["meter_b_voltage"] = device[ + data[ESS_DEVICE_TYPE][serial]["meter_b_voltage"] = device[ "ess_meter_reading" ]["meter_b"]["reading"]["voltage"]["value"] if True: device = ess_data["ess_report"]["hub_plus_status"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["contactor_position"] = device[ - "contactor_position" - ] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["grid_frequency_state"] = device[ - "grid_frequency_state" - ] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["grid_phase1_voltage"] = device[ - "grid_phase1_voltage" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["grid_phase2_voltage"] = device[ - "grid_phase2_voltage" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["grid_voltage_state"] = device[ - "grid_voltage_state" - ] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["hub_humidity"] = device[ - "hub_humidity" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["hub_temperature"] = device[ - "hub_temperature" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["inverter_connection_voltage"] = device[ - "inverter_connection_voltage" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["load_frequency_state"] = device[ - "load_frequency_state" - ] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["load_phase1_voltage"] = device[ - "load_phase1_voltage" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["load_phase2_voltage"] = device[ - "load_phase2_voltage" - ]["value"] - data[HUBPLUS_DEVICE_TYPE][device["serial_number"]]["main_voltage"] = device[ - "main_voltage" - ]["value"] + serial = device["serial_number"] + if serial not in data[HUBPLUS_DEVICE_TYPE]: + _LOGGER.warning(f"HubPlus {serial} not found in PVS data, skipping") + else: + data[HUBPLUS_DEVICE_TYPE][serial]["contactor_position"] = device[ + "contactor_position" + ] + data[HUBPLUS_DEVICE_TYPE][serial]["grid_frequency_state"] = device[ + "grid_frequency_state" + ] + data[HUBPLUS_DEVICE_TYPE][serial]["grid_phase1_voltage"] = device[ + "grid_phase1_voltage" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["grid_phase2_voltage"] = device[ + "grid_phase2_voltage" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["grid_voltage_state"] = device[ + "grid_voltage_state" + ] + data[HUBPLUS_DEVICE_TYPE][serial]["hub_humidity"] = device[ + "hub_humidity" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["hub_temperature"] = device[ + "hub_temperature" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["inverter_connection_voltage"] = device[ + "inverter_connection_voltage" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["load_frequency_state"] = device[ + "load_frequency_state" + ] + data[HUBPLUS_DEVICE_TYPE][serial]["load_phase1_voltage"] = device[ + "load_phase1_voltage" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["load_phase2_voltage"] = device[ + "load_phase2_voltage" + ]["value"] + data[HUBPLUS_DEVICE_TYPE][serial]["main_voltage"] = device[ + "main_voltage" + ]["value"] if True: # Generate a usable serial number for this virtual device, use PVS serial as base # since we must be talking through one and it has a serial - pvs_serial = next(iter(data[PVS_DEVICE_TYPE])) # only one PVS + # Check if PVS device exists before trying to access it + pvs_devices = data.get(PVS_DEVICE_TYPE) + if not pvs_devices: + _LOGGER.warning("PVS device not found in data, skipping SunVault virtual device creation") + return data + + pvs_serial = next(iter(pvs_devices)) # only one PVS sunvault_serial = f"sunvault_{pvs_serial}" data[SUNVAULT_DEVICE_TYPE] = {sunvault_serial: {}} data[SUNVAULT_DEVICE_TYPE][sunvault_serial]["sunvault_amperage"] = sum( @@ -283,6 +344,10 @@ def sunpower_fetch( except (ParseException, ConnectionException) as error: raise UpdateFailed from error + if not sunpower_data or "devices" not in sunpower_data: + _LOGGER.error("Invalid PVS data structure: %s", sunpower_data) + raise UpdateFailed("PVS returned invalid data structure - missing 'devices' key") + data = convert_sunpower_data(sunpower_data) if ESS_DEVICE_TYPE in data: # Look for an ESS in PVS data use_ess = True @@ -331,7 +396,12 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry): entry_id = entry.entry_id hass.data[DOMAIN].setdefault(entry_id, {}) - sunpower_monitor = SunPowerMonitor(entry.data[SUNPOWER_HOST]) + # Create monitor in executor since __init__ makes blocking calls + sunpower_monitor = await hass.async_add_executor_job( + SunPowerMonitor, + entry.data[SUNPOWER_HOST], + None, # Auto-fetch from PVS + ) sunpower_update_invertal = entry.options.get( SUNPOWER_UPDATE_INTERVAL, DEFAULT_SUNPOWER_UPDATE_INTERVAL, diff --git a/custom_components/sunpower/binary_sensor.py b/custom_components/sunpower/binary_sensor.py index 1f11b49..e296ec7 100644 --- a/custom_components/sunpower/binary_sensor.py +++ b/custom_components/sunpower/binary_sensor.py @@ -41,8 +41,8 @@ async def async_setup_entry(hass, config_entry, async_add_entities): else: _LOGGER.debug("Found No ESS Data") - if PVS_DEVICE_TYPE not in sunpower_data: - _LOGGER.error("Cannot find PVS Entry") + if PVS_DEVICE_TYPE not in sunpower_data or not sunpower_data[PVS_DEVICE_TYPE]: + _LOGGER.error("Cannot find PVS Entry or PVS data is empty") else: entities = [] @@ -152,6 +152,12 @@ def unique_id(self): @property def state(self): """Get the current value""" + # Check if device type and device exist in coordinator data + if ( + self._device_type not in self.coordinator.data + or self.base_unique_id not in self.coordinator.data[self._device_type] + ): + return None return self.coordinator.data[self._device_type][self.base_unique_id][self._field] @property diff --git a/custom_components/sunpower/config_flow.py b/custom_components/sunpower/config_flow.py index ea08c0b..9674155 100644 --- a/custom_components/sunpower/config_flow.py +++ b/custom_components/sunpower/config_flow.py @@ -72,7 +72,10 @@ def async_get_options_flow( async def async_step_user(self, user_input: dict[str, any] | None = None): """Handle the initial step.""" errors = {} - _LOGGER.debug(f"User Setup input {user_input}") + if user_input: + _LOGGER.debug(f"User Setup: host={user_input.get(CONF_HOST)}") + else: + _LOGGER.debug("User Setup: initial form display") if user_input is not None: try: info = await validate_input(self.hass, user_input) @@ -80,6 +83,8 @@ async def async_step_user(self, user_input: dict[str, any] | None = None): return self.async_create_entry(title=info["title"], data=user_input) except CannotConnect: errors["base"] = "cannot_connect" + except InvalidHost: + errors["base"] = "invalid_host" except Exception: # pylint: disable=broad-except _LOGGER.exception("Unexpected exception") errors["base"] = "unknown" @@ -107,7 +112,10 @@ async def async_step_init( user_input: dict[str, any] | None = None, ) -> config_entries.FlowResult: """Manage the options.""" - _LOGGER.debug(f"Options input {user_input} {self.config_entry}") + if user_input: + _LOGGER.debug(f"Options input: intervals={user_input.get(SUNPOWER_UPDATE_INTERVAL)}/{user_input.get(SUNVAULT_UPDATE_INTERVAL)}") + else: + _LOGGER.debug("Options: initial form display") options = dict(self.config_entry.options) errors = {} diff --git a/custom_components/sunpower/manifest.json b/custom_components/sunpower/manifest.json index 6e0bc46..4b0416e 100644 --- a/custom_components/sunpower/manifest.json +++ b/custom_components/sunpower/manifest.json @@ -10,6 +10,6 @@ "issue_tracker": "https://github.com/krbaker/hass-sunpower/issues", "requirements": ["requests"], "ssdp": [], - "version": "2025.8.1", + "version": "2025.10.3", "zeroconf": [] } diff --git a/custom_components/sunpower/sensor.py b/custom_components/sunpower/sensor.py index a7308ec..3e62dc8 100644 --- a/custom_components/sunpower/sensor.py +++ b/custom_components/sunpower/sensor.py @@ -44,8 +44,8 @@ async def async_setup_entry(hass, config_entry, async_add_entities): else: _LOGGER.debug("Found No ESS Data") - if PVS_DEVICE_TYPE not in sunpower_data: - _LOGGER.error("Cannot find PVS Entry") + if PVS_DEVICE_TYPE not in sunpower_data or not sunpower_data[PVS_DEVICE_TYPE]: + _LOGGER.error("Cannot find PVS Entry or PVS data is empty") else: entities = [] @@ -176,6 +176,13 @@ def unique_id(self): @property def native_value(self): """Get the current value""" + # Check if device type and device exist in coordinator data + if ( + self._device_type not in self.coordinator.data + or self.base_unique_id not in self.coordinator.data[self._device_type] + ): + return None + if self._my_device_class == SensorDeviceClass.POWER_FACTOR: try: value = float( diff --git a/custom_components/sunpower/strings.json b/custom_components/sunpower/strings.json index c57b419..411f37e 100644 --- a/custom_components/sunpower/strings.json +++ b/custom_components/sunpower/strings.json @@ -8,11 +8,12 @@ "use_descriptive_names": "Use descriptive entity names (recommended)", "use_product_names": "Use products in entity names (not recommended)" }, - "description": "Hostname or IP of PVS (usually 172.27.153.1)" + "description": "Hostname or IP of PVS (usually 172.27.153.1). Serial suffix will be auto-detected from the PVS." } }, "error": { "cannot_connect": "Cannot Connect", + "invalid_host": "Invalid IP address or hostname format", "unknown": "Unknown Error" }, "abort": { diff --git a/custom_components/sunpower/sunpower.py b/custom_components/sunpower/sunpower.py index 7db42bc..9081903 100644 --- a/custom_components/sunpower/sunpower.py +++ b/custom_components/sunpower/sunpower.py @@ -1,7 +1,19 @@ -""" Basic Sunpower PVS Tool """ +"""SunPower PVS client with automatic LocalAPI/Legacy CGI fallback.""" + +import logging +import traceback import requests import simplejson +from urllib.parse import urlencode + +try: + # Suppress TLS warnings when verify=False + import urllib3 + + urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) +except Exception: # pragma: no cover - optional + pass class ConnectionException(Exception): @@ -13,42 +25,539 @@ class ParseException(Exception): class SunPowerMonitor: - """Basic Class to talk to sunpower pvs 5/6 via the management interface 'API'. - This is not a public API so it might fail at any time. - if you find this useful please complain to sunpower and your sunpower dealer that they - do not have a public API""" + """Client for SunPower PVS with automatic LocalAPI/Legacy CGI fallback. + + Automatically detects firmware version and uses: + - LocalAPI (Varserver FCGI) for firmware build >= 61840 + - Legacy CGI endpoints for older firmware + """ + + # Minimum firmware build number that supports LocalAPI + MIN_LOCALAPI_BUILD = 61840 + + def __init__(self, host, serial_suffix: str | None = None): + """Initialize PVS client with automatic API detection. - def __init__(self, host): - """Initialize.""" + - host: IP or hostname of the PVS + - serial_suffix: last 5 characters of the PVS serial (password for ssm_owner, only needed for LocalAPI) + """ self.host = host - self.command_url = "http://{0}/cgi-bin/dl_cgi?Command=".format(host) + self.base = "http://{0}".format(host) + self.session = requests.Session() + self.timeout = 30 + self.use_localapi = False + self._session_token = None + self._cache_initialized = False + self._last_fetch_time = 0 + self._min_fetch_interval = 1.0 # Minimum 1 second between fetches + + # Check firmware version to determine which API to use + support_check = self.check_localapi_support(host, self.timeout) + + if support_check["supported"]: + # Use LocalAPI for newer firmware + self.use_localapi = True + + # Try to auto-fetch serial suffix from supervisor/info if not provided + if not serial_suffix or not serial_suffix.strip(): + serial_suffix = self._fetch_serial_suffix() + + # Use the serial suffix (auto-fetched or provided) + resolved = (serial_suffix or "").strip() + + if not resolved: + raise ConnectionException( + "Missing serial suffix for LocalAPI. Auto-detection failed. " + "Unable to retrieve serial number from PVS." + ) + + self.serial_suffix = resolved + self._login() + else: + # Use legacy CGI for older firmware + self.use_localapi = False + self.command_url = "http://{0}/cgi-bin/dl_cgi?Command=".format(host) + + def _fetch_serial_suffix(self) -> str: + """Attempt to fetch serial number from supervisor/info endpoint. + + Returns last 5 characters of serial, or empty string if fetch fails. + """ + try: + resp = self.session.get( + "{0}/cgi-bin/dl_cgi/supervisor/info".format(self.base), + timeout=self.timeout + ) + if resp.status_code == 200: + data = resp.json() + if "supervisor" in data and "SERIAL" in data["supervisor"]: + serial = data["supervisor"]["SERIAL"] + if len(serial) >= 5: + return serial[-5:] + except Exception: + pass # Silently fail, will use fallback + return "" + + @staticmethod + def check_localapi_support(host: str, timeout: int = 30) -> dict: + """Check if PVS supports LocalAPI by actually testing the endpoint. + + Returns dict with: + - supported: bool + - build: int or None + - version: str or None + - serial: str or None + - error: str or None + """ + result = { + "supported": False, + "build": None, + "version": None, + "serial": None, + "error": None + } + + try: + resp = requests.get( + "http://{0}/cgi-bin/dl_cgi/supervisor/info".format(host), + timeout=timeout + ) + + if resp.status_code != 200: + result["error"] = "HTTP {0}".format(resp.status_code) + return result + + data = resp.json() + if "supervisor" not in data: + result["error"] = "Invalid response format" + return result + + supervisor = data["supervisor"] + build = supervisor.get("BUILD") + version = supervisor.get("SWVER") + serial = supervisor.get("SERIAL") + + result["build"] = build + result["version"] = version + result["serial"] = serial + + # Actually test if LocalAPI endpoint exists (not just build number) + # Test the /auth endpoint which is the actual LocalAPI path used + if build and build >= SunPowerMonitor.MIN_LOCALAPI_BUILD: + try: + test_resp = requests.get( + "http://{0}/auth".format(host), + timeout=5 + ) + # If we get anything other than 404, LocalAPI exists + # (401/403 means auth required, which is expected) + if test_resp.status_code != 404: + result["supported"] = True + else: + result["error"] = "Build {0} but LocalAPI endpoints not found (404)".format(build) + except Exception: + result["error"] = "Build {0} but LocalAPI endpoint test failed".format(build) + else: + result["error"] = "Firmware build {0} is too old. LocalAPI requires build {1}+".format(build, SunPowerMonitor.MIN_LOCALAPI_BUILD) + + return result + + except requests.exceptions.RequestException as e: + result["error"] = "Connection failed: {0}".format(e) + return result + except Exception as e: + result["error"] = "Unexpected error: {0}".format(e) + return result + + def _login(self): + """Authenticate to LocalAPI, storing session token.""" + import base64 + + # Build Basic auth header (lowercase "basic") + token = base64.b64encode("ssm_owner:{0}".format(self.serial_suffix).encode("utf-8")).decode("ascii") + auth_header = "basic {0}".format(token) + + try: + resp = self.session.get( + "{0}/auth?login".format(self.base), + headers={"Authorization": auth_header}, + timeout=self.timeout + ) + resp.raise_for_status() + data = resp.json() + + # Extract session token and store it for subsequent requests + session_token = data.get("session") + if not session_token: + raise ParseException("Authentication failed: no session token received") + + # Store session token in session headers for all future requests + self.session.headers.update({"Cookie": "session={0}".format(session_token)}) + self._session_token = session_token + + except requests.exceptions.HTTPError as error: + if error.response.status_code == 401: + raise ConnectionException("Authentication failed: invalid credentials") + raise ConnectionException("Authentication failed: HTTP {0}".format(error.response.status_code)) + except requests.exceptions.RequestException as error: + raise ConnectionException("Authentication failed: network error") + except simplejson.errors.JSONDecodeError as error: + raise ParseException("Authentication failed: invalid response format") - def generic_command(self, command): - """All 'commands' to the PVS module use this url pattern and return json - The PVS system can take a very long time to respond so timeout is at 2 minutes""" + def _vars(self, *, names=None, match=None, cache=None, fmt_obj=True, retry_count=0): + """Query /vars endpoint with retry logic. + + names: list of exact variable names + match: substring match + cache: cache id to create or query + fmt_obj: if True, request fmt=obj to get object mapping + retry_count: internal retry counter + """ + params = {} + if names: + params["name"] = ",".join(names) + if match: + params["match"] = match + if cache: + params["cache"] = cache + if fmt_obj: + params["fmt"] = "obj" + + max_retries = 2 + + try: + resp = self.session.get("{0}/vars".format(self.base), params=params, timeout=self.timeout) + + # Handle session expiration + if resp.status_code == 401 or resp.status_code == 403: + if retry_count < max_retries: + # Re-authenticate and retry + self._login() + return self._vars(names=names, match=match, cache=cache, fmt_obj=fmt_obj, retry_count=retry_count + 1) + else: + raise ConnectionException("Authentication failed after retries") + + # Handle 400 Bad Request - might indicate LocalAPI not fully supported + if resp.status_code == 400: + # Log the response body for debugging + try: + error_body = resp.text + logger = logging.getLogger(__name__) + logger.debug(f"400 Bad Request response body: {error_body}") + except Exception: + pass + raise ConnectionException(f"Bad Request (400) - LocalAPI endpoint may not support these parameters. URL: {self.base}/vars, params: {params}") + + resp.raise_for_status() + data = resp.json() + return data + except requests.exceptions.Timeout as error: + if retry_count < max_retries: + # Retry on timeout + return self._vars(names=names, match=match, cache=cache, fmt_obj=fmt_obj, retry_count=retry_count + 1) + raise ConnectionException(f"Request timeout after retries: {error}") + except requests.exceptions.HTTPError as error: + # Catch HTTPError before generic RequestException + raise ConnectionException(f"HTTP {error.response.status_code}: {error}. URL: {self.base}/vars, params: {params}") + except requests.exceptions.RequestException as error: + raise ConnectionException(f"Failed to query device variables: {error}. URL: {self.base}/vars, params: {params}") + except (simplejson.errors.JSONDecodeError, ValueError) as error: + raise ParseException(f"Failed to parse device response: {error}") + + def _fetch_meters(self, use_cache=True): + """Fetch all meter variables and group by device index. + + use_cache: if True and cache exists, use cached data; if False, refresh cache + """ + # Note: LocalAPI supports cache parameter but we don't use it for simplicity. + # With 120s polling intervals, the performance benefit is negligible and + # avoiding cache state management makes the integration more reliable. + data = self._vars(match="meter", fmt_obj=True) + + # Group by meter index (e.g., /sys/devices/meter/0/field -> meter 0) + meters = {} + for var_path, value in data.items(): + if "/sys/devices/meter/" in var_path: + parts = var_path.split("/") + if len(parts) >= 5: + meter_idx = parts[4] # e.g., "0", "1" + field = parts[5] if len(parts) > 5 else None + if field: + meter_key = "/sys/devices/meter/{0}".format(meter_idx) + if meter_key not in meters: + meters[meter_key] = {} + meters[meter_key][field] = value + return meters + + def _fetch_inverters(self, use_cache=True): + """Fetch all inverter variables and group by device index. + + use_cache: if True and cache exists, use cached data; if False, refresh cache + """ + # Note: LocalAPI supports cache parameter but we don't use it for simplicity. + # With 120s polling intervals, the performance benefit is negligible and + # avoiding cache state management makes the integration more reliable. + data = self._vars(match="inverter", fmt_obj=True) + + inverters = {} + for var_path, value in data.items(): + if "/sys/devices/inverter/" in var_path: + parts = var_path.split("/") + if len(parts) >= 5: + inv_idx = parts[4] + field = parts[5] if len(parts) > 5 else None + if field: + inv_key = "/sys/devices/inverter/{0}".format(inv_idx) + if inv_key not in inverters: + inverters[inv_key] = {} + inverters[inv_key][field] = value + return inverters + + def _fetch_sysinfo(self, use_cache=True): + """Fetch system info variables. + + use_cache: if True and cache exists, use cached data; if False, refresh cache + """ + # Note: LocalAPI supports cache parameter but we don't use it for simplicity. + # With 120s polling intervals, the performance benefit is negligible and + # avoiding cache state management makes the integration more reliable. + data = self._vars(match="info", fmt_obj=True) + return data + + @staticmethod + def _key(obj, old_key, new_key, transform=None): + if old_key in obj: + val = obj[old_key] + obj[new_key] = transform(val) if transform else val + + def _legacy_generic_command(self, command): + """Legacy CGI command for older firmware. + + All 'commands' to the PVS module use this url pattern and return json. + The PVS system can take a very long time to respond so timeout is at 2 minutes. + """ try: return requests.get(self.command_url + command, timeout=120).json() except requests.exceptions.RequestException as error: - raise ConnectionException from error + raise ConnectionException("Failed to execute legacy command") except simplejson.errors.JSONDecodeError as error: - raise ParseException from error + raise ParseException("Failed to parse legacy response") def device_list(self): - """Get a list of all devices connected to the PVS""" - return self.generic_command("DeviceList") + """Return DeviceList using LocalAPI (new) or legacy CGI (old). + + Structure: {"devices": [ {DEVICE_TYPE, SERIAL, MODEL, TYPE, DESCR, STATE, ...fields} ]} + """ + if not self.use_localapi: + # Use legacy CGI endpoint for older firmware + return self._legacy_generic_command("DeviceList") + + # Use LocalAPI for newer firmware + logger = logging.getLogger(__name__) + devices = [] + + # Determine if we should use cached data (after first successful fetch) + use_cache = self._cache_initialized + + # PVS device (minimal info) - always add, even if fetch fails + pvs_serial = "PVS-{0}".format(self.host) + pvs_model = "PVS" + pvs_sw_version = "Unknown" + + try: + sysinfo = self._fetch_sysinfo(use_cache=use_cache) + # Use actual serial number from PVS if available + pvs_serial = sysinfo.get("/sys/info/serialnum", pvs_serial) + pvs_model = sysinfo.get("/sys/info/model", pvs_model) + pvs_sw_version = sysinfo.get("/sys/info/sw_rev", pvs_sw_version) + except Exception as e: + # If sysinfo fails, log but use defaults + logger.warning("Failed to fetch PVS info, using defaults: {0}".format(e)) + + # Always add PVS device to devices list + devices.append( + { + "DEVICE_TYPE": "PVS", + "SERIAL": pvs_serial, + "MODEL": pvs_model, + "TYPE": "PVS", + "DESCR": "{0} {1}".format(pvs_model, pvs_serial), + "STATE": "working", + "sw_ver": pvs_sw_version, + # Legacy dl_* diagnostics unavailable via this minimal sysinfo; omit + } + ) + + # Meter devices - with error handling + try: + meters = self._fetch_meters(use_cache=use_cache) + except Exception as e: + logger.warning("Failed to fetch meters: {0}".format(e)) + meters = {} + + for path, m in meters.items(): + dev = { + "DEVICE_TYPE": "Power Meter", + "SERIAL": m.get("sn", "Unknown"), + "MODEL": m.get("prodMdlNm", "Unknown"), + "TYPE": "PVS-METER", + "DESCR": "Power Meter {0}".format(m.get('sn', '')), + "STATE": "working", + } + # Field mappings + dev["net_ltea_3phsum_kwh"] = m.get("netLtea3phsumKwh") + dev["p_3phsum_kw"] = m.get("p3phsumKw") + dev["q_3phsum_kvar"] = m.get("q3phsumKvar") + dev["s_3phsum_kva"] = m.get("s3phsumKva") + dev["tot_pf_rto"] = m.get("totPfRto") + dev["v12_v"] = m.get("v12V") + dev["v1n_v"] = m.get("v1nV") + dev["v2n_v"] = m.get("v2nV") + dev["freq_hz"] = m.get("freqHz") + + # Leg-specific fields + if "i1A" in m: + dev["i1_a"] = m.get("i1A") + if "i2A" in m: + dev["i2_a"] = m.get("i2A") + if "p1Kw" in m: + dev["p1_kw"] = m.get("p1Kw") + if "p2Kw" in m: + dev["p2_kw"] = m.get("p2Kw") + + # Grid/Home energy tracking (to_grid = negative, to_home = positive) + if "negLtea3phsumKwh" in m: + dev["neg_ltea_3phsum_kwh"] = m.get("negLtea3phsumKwh") + if "posLtea3phsumKwh" in m: + dev["pos_ltea_3phsum_kwh"] = m.get("posLtea3phsumKwh") + + devices.append(dev) + + # Inverter devices - with error handling + try: + inverters = self._fetch_inverters(use_cache=use_cache) + except Exception as e: + logger.error("Failed to fetch inverters: {0}".format(e)) + logger.debug("Traceback: {0}".format(traceback.format_exc())) + inverters = {} + + for path, inv in inverters.items(): + dev = { + "DEVICE_TYPE": "Inverter", + "SERIAL": inv.get("sn", "Unknown"), + "MODEL": inv.get("prodMdlNm", "Unknown"), + "TYPE": "MICRO-INVERTER", + "DESCR": "Inverter {0}".format(inv.get('sn', '')), + "STATE": "working", + } + # Field name mapping: LocalAPI uses camelCase (e.g., ltea3phsumKwh), + # but we convert to snake_case (e.g., ltea_3phsum_kwh) to match + # legacy CGI format. This ensures identical data structures for + # backwards compatibility with all downstream code (sensors, entities). + + # Energy + dev["ltea_3phsum_kwh"] = inv.get("ltea3phsumKwh") + + # Power - AC and DC + dev["p_3phsum_kw"] = inv.get("p3phsumKw") # AC power (more accurate) + dev["p_mppt1_kw"] = inv.get("pMppt1Kw") # DC power + + # Voltage - AC and DC + dev["vln_3phavg_v"] = inv.get("vln3phavgV") # AC voltage + dev["v_mppt1_v"] = inv.get("vMppt1V") # DC voltage + + # Current - AC and DC + dev["i_3phsum_a"] = inv.get("i3phsumA") # AC current (actual output) + dev["i_mppt1_a"] = inv.get("iMppt1A") # DC current + + # Temperature and frequency + dev["t_htsnk_degc"] = inv.get("tHtsnkDegc") + dev["freq_hz"] = inv.get("freqHz") + + # Optional MPPT sum if present + if "pMpptsumKw" in inv: + dev["p_mpptsum_kw"] = inv.get("pMpptsumKw") + + devices.append(dev) + + # Mark cache as initialized after first successful fetch + if not self._cache_initialized and (meters or inverters): + self._cache_initialized = True + + # Log summary of what was fetched + logger.info("LocalAPI device_list: PVS={0}, Meters={1}, Inverters={2}".format( + pvs_serial, len(meters), len(inverters) + )) + + return {"devices": devices} def energy_storage_system_status(self): - """Get the status of the energy storage system""" + """Return ESS status using LocalAPI (new) or legacy CGI (old). + + Structure expected by callers: + { "ess_report": { "battery_status": [...], "ess_status": [...], "hub_plus_status": {...} } } + If detailed vars are not available, return empty lists/dicts and let callers handle gracefully. + """ + if not self.use_localapi: + # Use legacy CGI endpoint for older firmware + try: + return requests.get( + "http://{0}/cgi-bin/dl_cgi/energy-storage-system/status".format(self.host), + timeout=120, + ).json() + except requests.exceptions.RequestException as error: + raise ConnectionException("Failed to get ESS status") + except simplejson.errors.JSONDecodeError as error: + raise ParseException("Failed to parse ESS response") + + # Use LocalAPI for newer firmware try: - return requests.get( - "http://{0}/cgi-bin/dl_cgi/energy-storage-system/status".format(self.host), - timeout=120, - ).json() - except requests.exceptions.RequestException as error: - raise ConnectionException from error - except simplejson.errors.JSONDecodeError as error: - raise ParseException from error + livedata = self._vars(match="livedata", cache="ldata", fmt_obj=True) + except Exception: + livedata = {} + + report = { + "battery_status": [], + "ess_status": [], + "hub_plus_status": {}, + } + + # Populate minimal aggregate values if present + if livedata: + soc = livedata.get("/sys/livedata/soc") + ess_p = livedata.get("/sys/livedata/ess_p") + if soc is not None or ess_p is not None: + report["ess_status"].append( + { + "serial_number": "ESS-AGG", + "ess_meter_reading": { + "agg_power": {"value": float(ess_p) if ess_p is not None else 0.0}, + "meter_a": {"reading": {"current": {"value": 0}, "power": {"value": 0}, "voltage": {"value": 0}}}, + "meter_b": {"reading": {"current": {"value": 0}, "power": {"value": 0}, "voltage": {"value": 0}}}, + }, + "enclosure_humidity": {"value": 0}, + "enclosure_temperature": {"value": 0}, + } + ) + report["hub_plus_status"] = { + "serial_number": "HUBPLUS-AGG", + "grid_phase1_voltage": {"value": 0}, + "grid_phase2_voltage": {"value": 0}, + "hub_humidity": {"value": 0}, + "hub_temperature": {"value": 0}, + "inverter_connection_voltage": {"value": 0}, + "load_phase1_voltage": {"value": 0}, + "load_phase2_voltage": {"value": 0}, + } + + return {"ess_report": report} def network_status(self): - """Get a list of network interfaces on the PVS""" - return self.generic_command("Get_Comm") + """Return network/system info using LocalAPI (new) or legacy CGI (old).""" + if not self.use_localapi: + # Use legacy CGI endpoint for older firmware + return self._legacy_generic_command("Get_Comm") + + # Use LocalAPI for newer firmware + info = self._fetch_sysinfo() + return info diff --git a/custom_components/sunpower/translations/en.json b/custom_components/sunpower/translations/en.json index 0eb6620..725ce7c 100644 --- a/custom_components/sunpower/translations/en.json +++ b/custom_components/sunpower/translations/en.json @@ -5,18 +5,19 @@ }, "error": { "cannot_connect": "Cannot Connect", + "invalid_host": "Invalid IP address or hostname format", "unknown": "Unknown Error" }, "step": { "user": { - "data": { - "host": "Host", - "use_descriptive_names": "Use descriptive entity names (recommended)", - "use_product_names": "Use products in entity names (not recommended)" - }, - "description": "Hostname or IP of PVS (usually 172.27.153.1)" + "data": { + "host": "Host", + "use_descriptive_names": "Use descriptive entity names (recommended)", + "use_product_names": "Use products in entity names (not recommended)" + }, + "description": "Hostname or IP of PVS (usually 172.27.153.1). Serial suffix will be auto-detected from the PVS." + } } - } }, "options":{ "step": {