20 toks per second is more than enough for agents.

You’re comparing cloud chat speed to agents and finding it lacking.

It’s not slow.

Agents work differently, and force you to work differently. You simply set a goal and let them work and come back later.

Their tool calls take up a lot of time in lag that has nothing to do with tok speed.

20 toks per second are more than enough.


0 Comments

Leave a Reply

//z-na.amazon-adsystem.com/widgets/onejs?MarketPlace=US&adInstanceId=fcd278e1-9042-4686-b67a-366f43de1d63