The Llama Index integration enables to monitor your RAG pipelines with a single line of code.

You should create a new instance of the callback handler for each invocation.

Multiple calls in a single thread

You can combine the Llama Index callback handler with the concept of Thread to monitor multiple calls in a single thread.