Different optimization traces & optim on a shared code

[doxav]

Is it possible to support different traces & optimization on shared functions/nodes using decorators ? What would be the best practice ?

In my case, I need to do different traces & optimization on shared code files/functions:

• within each of my LLM agents.
• in a workflow involving all my different agents
• in a workflow involving 2 of my agents (overlapping the one above and one below)
• in a workflow involving 3 of my agents (overlapping the two above)

Thanks a lot

[chinganc]

Good question! In Trace, you can have multiple optimizers (each attends to separate parameters). The paths where those parameter are used can be shared. After calling backward, each optimizer will receive their respective minimal subgraph.

Is this what you meant? It will be helpful if you can help clarify your scenario. E.g.,

Do you mean using different optimizers for different parameters?
Do you mean optimizing them in different frequencies?
Do different traces touch on shared parameters/

I think it will be most clear if you can write down a toy example of such a case, so I can better help :)