VertexAI with Nemoguardrails

[fedewj]

I am trying to integrate nemoguardrails in my RAG(built using Langchain Framework). This RAG can use both GPT and VertexAI models(gemini). With GPT all works perfectly, no error. However, when I try to use gemini I get this error:

**Error in ask**: LLM Call Exception: Task <Task pending name='Task-2953' coro=<BaseChatModel._agenerate_with_cache() running at /app/.venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:715> cb=[gather.<locals>._done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:720]> got Future <Task pending name='Task-2954' coro=<UnaryUnaryCall._invoke() running at /app/.venv/lib/python3.10/site-packages/grpc/aio/_call.py:577>> attached to a different loop

These are my config:

1. config.yml:

models:
 - type: main
   engine: vertexai
   model: gemini-1.0-pro

rails:
  input:
    flows:
      - self check input

nemoguardrails=0.9.1.1
google-cloud-aiplatform=1.73.0
langchain = 0.1.13 
langchain-community= 0.0.38 
langchain-core = 0.1.53
langchain-google-vertexai = 0.1.2 

2. RAG:
   I've followed this documentation (Add Guardrails to a Chain): https://docs.nvidia.com/nemo/guardrails/user_guides/langchain/langchain-integration.html
   .....
   self.guardrails_config = RailsConfig.from_path("./data/configs_guardrails")
   .....
   return RunnableRails(self.guardrails_config, llm, input_key="question",
   output_key="answer", runnable=chain)
   .....

[Pouyanpi]

Hi @jingwangfe can you try out the most recent version (0.11.0) and see if you can reproduce this issue, and try to upgrade your dependencies.

[fedewj]

I have upgraded to the most recent version of nemoguardrails, but it did not resolve my problem.

However, I’ve implemented a workaround by modifying the invoke method of the RunnableRails class as follows (even though I received this warning : LangChainDeprecationWarning: Importing ZapierNLAWrapper from langchain.utilities is deprecated. Please replace deprecated imports:)

`    async def invoke(
        self,
        input: Input,
        config: Optional[RunnableConfig] = None,
        **kwargs: Optional[Any],
    ) -> Output:
        """Invoke this runnable synchronously."""
        input_messages = self._transform_input_to_rails_format(input)
        res = await self.rails.generate_async(
            messages=input_messages, options=GenerationOptions(output_vars=True)
        )
        context = res.output_data
        result = res.response`

Do you think there could be any issues with this modification? Do you think it's possible to solve this by upgrading all the libraries, rather than just upgrading nemoguardrails? Thank you in advance @Pouyanpi

[Pouyanpi]

This is resolved by #1085 and will be available in v0.14.0 version of nemoguardrails.

If you'd like you can try it on develop branch

Looking for an easy way to find the correct provider name, see:

https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user-guides/cli.md#providers
https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user-guides/configuration-guide.md#exploring-available-providers