Python: Unclear how to detect errors during `invoke_stream`

[bastbu]

On version python-0.9.0.beta1, the KernelContext was removed. On KernelContext, there was a last_exception property that allowed us to get an error if one occurred. So far, what I can see with FunctionResult is that on error there is a metadata["error"] field. However, I could not see from the examples what is the supported way to check (and reraise) errors. Is there a recommended way to do this?

[moonbox3]

Hi @bastbu, thanks for your question around this. Let's take the following example from the kernel's invoke method (this is similar for invoke_stream:

exception = None
try:
    function_result = await func.invoke(self, arguments)
except Exception as exc:
    logger.error(
        "Something went wrong in function invocation. During function invocation:"
        f" '{func.plugin_name}.{func.name}'. Error description: '{str(exc)}'"
    )
    exception = exc

# this allows a hook to alter the results before adding.
function_invoked_args = self.on_function_invoked(func.metadata, arguments, function_result, exception)
results.append(function_invoked_args.function_result)

if function_invoked_args.exception:
    raise KernelInvokeException(
        f"Error occurred while invoking function: '{func.plugin_name}.{func.name}'"
    ) from function_invoked_args.exception

If an exception is encountered, we're raising the KernelInvokeException from the function_invoked_args.exception so the original exception is not lost, instead it is chained to the KernelInvokeException. You can access the original exception via the __cause__ attribute on the KernelInvokeException as shown in this sample:

try:
    <some code>
except KernelInvokeException as ke:
    original_exception = ke.__cause__
    # You can re-raise it or handle it as needed
    raise original_exception  # Re-raise the original exception

After catching the original exception as original_exception, you can choose to re-raise it, handle it based on its type, log it, or take any other appropriate action depending on your application's needs.

[bastbu]

Hi @moonbox3 & thanks for your answer! I'm not sure I understand your suggestion - the problem that I'm facing is that KernelFunction.invoke (I'll use the non-streaming version in the following) hides exceptions and returns them as a FunctionResult instead of raising them:

semantic-kernel/python/semantic_kernel/functions/kernel_function.py

Lines 167 to 171 in 4fb351d

	try:
	return await self._invoke_internal(kernel, arguments)
	except Exception as exc:
	logger.error(f"Error occurred while invoking function {self.name}: {exc}")
	return FunctionResult(function=self.metadata, value=None, metadata={"error": exc, "arguments": arguments})

The code you reference seems to come from here:

semantic-kernel/python/semantic_kernel/kernel.py

Lines 326 to 333 in 4fb351d

	try:
	function_result = await func.invoke(self, arguments)
	except Exception as exc:
	logger.error(
	"Something went wrong in function invocation. During function invocation:"
	f" '{func.plugin_name}.{func.name}'. Error description: '{str(exc)}'"
	)
	exception = exc

As the exception is returned instead of raised, the exception = exc assignment is never executed.

Hence the try/catch suggestion does not work, and for now I needed to manually parse the error entry in the metadata on the FunctionResult. The same behavior I can observe as well for KernelFunction.invoke_stream.

Am I understanding correctly that the recommended way is to set up the on_function_invoked handler on the Kernel and that this is the recommended way to access all exceptions? This seems a bit inconvenient for scenarios where I just want to invoke a single function and observe any exception on the invocation.

[moonbox3]

Hi @bastbu, thanks for the clarification. The kernel function exception handling seems to be a miss. I would expect the exception to be chained, and thrown so it can be handled. Let me look into this. Tagging @eavanvalkenburg here for visibility.

[moonbox3]

@bastbu I'm looking at this again. I see the issue now. Firstly, I understand we're consuming the exception as part of the returned FunctionResult. That exception is getting added to the metadata with a key called error, and in the kernel invoke_stream, we're trying to get the exception with the key exception instead of error.

For the kernel invoke method (non-streaming), there is a default method on_function_invoked that attempts to set the exception as part of the function invoked args. But as you pointed out, the exception is included in the FunctionResult, and we wouldn't set the exception by getting into the "except" block from that wraps the kernel invoke.

I am not saying you need to set up an on_function_invoked handler. We should be surfacing the exception properly, that was previously contained in the FunctionResult. Let me work on fixing this.

[bastbu]

@moonbox3 Thanks for the fix! I can confirm that the non-streaming case now works as expected. However, for the streaming case I still run into the same issue. Could it be that there's a typo here?

semantic-kernel/python/semantic_kernel/kernel.py

Lines 213 to 216 in 555a7c8

	if isinstance(stream_message, FunctionResult):
	exception = stream_message.metadata.get("error", None)
	if exception:
	break

It still references error instead of exception.

[moonbox3]

@moonbox3 Thanks for the fix! I can confirm that the non-streaming case now works as expected. However, for the streaming case I still run into the same issue. Could it be that there's a typo here?

semantic-kernel/python/semantic_kernel/kernel.py

Lines 213 to 216 in 555a7c8

	if isinstance(stream_message, FunctionResult):
	exception = stream_message.metadata.get("error", None)
	if exception:
	break

It still references error instead of exception.

Yeah, that was a miss. Apologies. I thought I got that one! Will fix shortly.