RAG 101: Session Resources

[multispark]

This is the resources and discussion thread for this live stream: https://aka.ms/raghack/101
It will be updated with links after the session.

Recording: https://www.youtube.com/watch?v=6fgTh9a0YvI
Slides:
PDF- https://speakerdeck.com/pamelafox/raghack-kickoff-and-rag-101
PPTX- https://github.com/microsoft/RAG_Hack/blob/main/slides/RAGHack-KickoffEnglish.pptx

Please ask any follow-up questions here, and the speakers will see them.

[uquad]

Can Azure Document Intelligence parse/understand images if so then what is the pros/cons vs using GPT4v?

[pamelafox]

Azure Document Intelligence can do OCR on images, meaning that it can extract text from images. The reason for using GPT4 is to be able to answer questions about non-text in images as well, so you can do things like ask it to analyze a graph.

[satrox28]

Is the AI services matured enough to perform Image/Video Analysis?.

[pamelafox]

What sort of image/video analysis are you doing? For RAG-with-vision, I use gpt-4o for answering questions about images, and it works fairly well. But it depends on what analysis you're imagining.

[satrox28]

thanks for the answer, like if am attending online exams and it would need an examiner to watch us what we are doing, instead based on the real time video can it track the facial expression when attempting bad things and send some alerts/notification o a pre-recorded video to be analyzed for any copyright infringement or sensitive content before adding to a blog post.

[uquad]

Is it possible to use fine-tuned open source models on Azure?

[pamelafox]

I believe you can do that with Azure ML Studio, this blog post discusses a similar flow:
https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/fine-tune-flux-1-lora-with-your-own-images-and-deploy-using/ba-p/4235626#:~:text=Step%205:%20Create%20the%20Compute.%20In%20order%20to

Is that what you're looking for?

[pamelafox]

You can also fine-tune Llama3.1 more easily than a completely custom model:

https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-three%2Cchatcompletion

[pamelafox]

Cedric will be showing how to fine-tune Llama in his RAFT session next week.

[uquad]

I believe you can do that with Azure ML Studio, this blog post discusses a similar flow: https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/fine-tune-flux-1-lora-with-your-own-images-and-deploy-using/ba-p/4235626#:~:text=Step%205:%20Create%20the%20Compute.%20In%20order%20to

Is that what you're looking for?

Thanks! In this case it fine-tuned Flux on Azure ML but lets say I've locally fine-tuned an open source LLM like lama 3.1 8B ,would I be able upload it to Azure and use it with rest of the other Azure stack/components (Context: https://www.youtube.com/live/6fgTh9a0YvI?si=VIh_pPsoNOHg4D8o&t=642)

[uquad]

Cedric will be showing how to fine-tune Llama in his RAFT session next week.

great! looking forward to it

[pamelafox]

For those of you interested in trying the @Azure agent for GitHub copilot, request access here: https://aka.ms/GHAzCopilot-RAGHack

[sugnicksen009]

How can we evaluate the answers generated from LLM's and ensure I score for evaluation ?

[pamelafox]

I'll have a session on that next week:
https://reactor.microsoft.com/reactor/events/23414/

But if you're keen to evaluate now, this blog post gives an overview of the tools I use:
https://blog.pamelafox.org/2024/01/evaluating-rag-chat-app-approach-sdks.html

[sugnicksen009]

Thanks @pamelafox

[Yash-Kavaiya]

How to evaluate model performance and logging and monitoring is there any tools in Azure?

Like Langsmith , Mlflow

[pamelafox]

Good question! You can use OpenTelemetry via the openai distribution for Azure Monitor support, that's what we do in azure-search-openai-demo:
https://github.com/Azure-Samples/azure-search-openai-demo/?tab=readme-ov-file#monitoring-with-application-insights
I've also deployed Langfuse as I like their UI, I have an Azure deployable version here:
https://github.com/Azure-Samples/azure-search-openai-demo/?tab=readme-ov-file#monitoring-with-application-insights
Then you have to hook up your app code to log to the Langfuse server.