Vercel AI SDK has updated its integration with Baseten, specifically in the @ai-sdk/baseten@2.1.0 package. This update makes the native performance client for embeddings an opt-in feature. Previously, the @basetenlabs/performance-client was a default dependency, which presented challenges for developers.
The change addresses issues where the native client, a NAPI addon comprising 16 platform binary packages and 5-16 MB in size, could not load in edge runtimes. Bundlers also struggled to resolve its platform binaries. Because it was imported at the module top level, all consumers incurred its overhead, even if they only used embeddings.
Key Points
- The native performance client for embeddings is now opt-in in @ai-sdk/baseten@2.1.0.
- @basetenlabs/performance-client is no longer a default dependency.
- The native client is a NAPI addon, consisting of 16 platform binary packages.
- It added 5-16 MB to installations and could not load in edge runtimes.
- Bundlers previously could not resolve its platform binaries.
- Embeddings now use plain HTTP to the deployment's OpenAI-compatible endpoint.
- Developers can still use the native client's batching and request hedging by installing it manually.
Context
According to the Vercel AI SDK Changelog, this change was implemented to improve flexibility and reduce overhead for developers. The previous setup meant that every consumer paid for the performance client, even if only embeddings utilized it. The client's nature as a NAPI addon created compatibility issues with edge runtimes and bundlers.
Why It Matters
This update offers builders greater control over their dependency footprint and runtime environment. By making the performance client opt-in, it removes a significant hurdle for deployments in edge environments and reduces unnecessary package size for projects not requiring its specific features, potentially improving deployment efficiency and reducing build times.
What To Do
- Note that @basetenlabs/performance-client is no longer a default dependency.
- If client-side batching and request hedging are needed, install @basetenlabs/performance-client manually.
- Review your project's dependencies to assess the impact of this change on package size.
- Test deployments in edge runtimes to confirm improved compatibility.