Analyzer response must be a real number.

[arthursloan]

I know this is eventually going to be addressed as part of implementing support for multiple objectives, but right now if the analyzer returns a list of values, ARES OS throws the following error:

Failed to analyze experiment result: Call to analyzer has failed: Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Error in custom analysis logic: must be real number, not list") at Ares.Core.Analyzing.RemoteAnalyzer.Analyze(AnalysisRequest request, CancellationToken cancellationToken) in /Users/runner/work/ARES/ARES/Ares.Core/Analyzing/RemoteAnalyzer.cs:line 35 at Ares.Core.Analyzing.AnalysisHelper.Analyze(ExperimentTemplate template, ExperimentExecutionSummary experimentSummary, ExperimentExecutionSummary startupSummary, RequestMetadata metadata, CancellationToken cancellationToken) in /Users/runner/work/ARES/ARES/Ares.Core/Analyzing/AnalysisHelper.cs:line 53

How necessary is this check that the response in a real number? in the short term, allowing the response to be a list or a struct could allow people to jerry rig a multi-objective analyzer.

[nkleiner]

Hey Arthur, looking at this it doesn't seem like this is a PyAres/ARES error. The "Error in custom analysis logic: " is an output that comes specifically from the try-except being tripped when running whatever analysis logic is tied to your analyzer. The math library in Python has a couple of places that throw this exception, perhaps it was one of those methods? If not I'm happy to investigate further.

[nkleiner]

Hey @arthursloan was there ever any additional discovery on this?

[arthursloan]

So I tested this again and I don't think its and issue with a particular analyzer. The analyzer successfully passes of an AnalysisResponse object to underlying analyzer service. The service gets at least as far as printing that the response is being sent and I can see that it still contains the List. Whatever check is causing the error seems to be happening in on the Protobuf side of the service, rather than the python side.