Bank of America and Capital One route AI work to smaller models to cut costs

Bank of America sends AI queries through Orchestra, a routing layer that uses a small classifier model to pick the most cost-effective model for each query, and chief technology and information officer Hari Gopalkrishnan called routing to lower-cost models an important cost tactic, CIO Dive reported from the Evident AI Symposium in New York on Oct. 22. Capital One distilled its fine-tuned open-weight models into smaller ones for Chat Concierge, its car-buying assistant, which chief enterprise AI officer Prem Natarajan said cut latency fivefold and raised customer engagement for dealerships by 50%. Gopalkrishnan also said Bank of America plans to double its AI budget next year.