Vespa.ai

AI/ML initiatives need a multifaceted platform that delivers many data types to many models. The AI database can help. My new BARC report, sponsored by Vespa.ai, defines what the AI database is, why it matters, and how it supports popular use cases. Read the full report here and excerpts below. Will this type of platform meet your requirements? https://lnkd.in/geGNZYje This emerging type of platform manages objects such as tables and documents, as well as vectors that assign numerical values to chunks of unstructured data. These chunks might be groups of words, images, audio clips, or video segments. The AI database can apply one or more various AI models to each piece of data, whatever the format, and combine multiple signals to create more accurate AI outputs. For example, a GenAI chatbot might calibrate its text outputs based on the results of a machine learning or natural language processing model. By consolidating models and data types in one multi-purpose platform, the AI database improves computing efficiency and scalability compared with disparate single-purpose platforms. Three categories of vendors are converging on the AI database market: lake houses, data warehouses, and vector databases. Lake houses and data warehouses are adding vector and text capabilities, while vector databases are adding support for text and tables. For example, Vespa.ai combines text, table and vector database capabilities into one platform. The AI database organizes and searches this multi structured data as described below. > Organize The AI database organizes its data, for example by placing similar vectors close to another based on assessments of their metadata. It also governs data with role-based access controls, encryption, masking of personally identifiable information (PII), and audit logs that support compliance reporting. This improves accuracy and reduces risk. > Search The AI database also searches tables, text, and vectors. It queries tables (still the most popular AI input) to find or derive specific values; finds document passages that match keywords; and runs similarity searches on vectors. It then selects and makes inferences in the data using various AI models. AI models AI databases support three primary types of AI models: machine learning, natural language processing, and GenAI. These various AI/ML models learn patterns, make inferences, and create outputs based on what they receive from the AI database – in fact, the AI database often hosts and runs the models. In some cases, the models are embedded within applications. Tim Young Jon Bratseth Shawn Rogers Timm Grosser Jacqueline Bloemen Florian Bigelmaier Eckerson Group Vignesh N Subra Gupta M. Andreas Kretz Myles Suer Alena Godin Gilles Simler Dr. Phil Hendrix Bill Schmarzo Paulina Rios Maya Michael Lodge Shaurya Rohatgi #data #ai #database

Learn how Farfetch uses real-time personalized recommendations, developed and running in Vespa.ai, to promote sales and drive customer engagement to over 4 million active customers. #VespaAI #Farfetch

The latest Vespa newsletter is out - highlights: 👉 RAG Vespa now provides LLM inference support, you can now implement a retrieval-augmented generation (RAG) application entirely as a Vespa application. We have added a new sample application demonstrating RAG end-to-end on Vespa: - Generation using an external LLM like OpenAI - Running an LLM locally inside the Vespa application on CPU - Running an LLM inside the Vespa application on Vespa Cloud on GPU 👉 Fuzzy Search with Prefix Match A prefix search will match “Edvard Grieg” and “Edvard Gr”. A fuzzy search matches “Edvard Grieg” with “Edward Grieg”. From Vespa 8.337 you can combine the two to match “Edvard Grieg” and “Edward Gr”. Very powerful for query completion! 👉 Pyvespa Lots of new features, including a notebook that demonstrates how the mixedbread.ai rerank models (cross encoders) can be used for global phase reranking in Vespa. 👉 Vector search performance Up to 9x faster distance calculations! Improvements for euclidean, angular, hamming and dotproduct as well as for HNSW indexing. 👉 Embeddings Since Vespa 8.329, embed the data _once_ for multiple resolutions. Store low-res in memory, hi-res on disk to optimize for cost - then use two-phase ranking for low-latency search with high precision. Get started using Vespa with LlamaIndex! Check out the new Vespa Vector Store demo notebook. Deep dive into this and more features like 10x faster data migration in the latest Vespa Newsletter:

Ensuring the reliability and effectiveness of AI systems remains a significant challenge.💡 🤖 Generative AI must be combined with access to your company data in most use cases, a process called retrieval-augmented generation. Achieving good quality requires a combination of multiple vectors with text and structured data, using machine learning to make final decisions. 💥 Vespa.ai enables solutions that do this while keeping latencies suitable for end users, at any scale. 🎙 In this episode of the EM360 Podcast, Kevin Petrie, VP of research at BARC US speaks to Jon Bratseth, CEO of Vespa.ai, to discuss:  👉The opportunity for Generative AI in business 👉Why you need more than vectors to achieve high quality in real systems 👉How to create high-quality GenerativeAI solutions at an enterprise scale 🎧 Listen now: https://lnkd.in/e-eeQXJd #AI #ArtificialIntelligence #Data

Ensuring the reliability and effectiveness of AI systems remains a significant challenge.💡 🤖 Generative AI must be combined with access to your company data in most use cases, a process called retrieval-augmented generation. Achieving good quality requires a combination of multiple vectors with text and structured data, using machine learning to make final decisions. 💥 Vespa.ai enables solutions that do this while keeping latencies suitable for end users, at any scale. 🎙 In this episode of the EM360 Podcast, Kevin Petrie, VP of research at BARC US speaks to Jon Bratseth, CEO of Vespa.ai, to discuss:  👉The opportunity for Generative AI in business 👉Why you need more than vectors to achieve high quality in real systems 👉How to create high-quality GenerativeAI solutions at an enterprise scale 🎧 Listen now: https://lnkd.in/e-eeQXJd #AI #ArtificialIntelligence #Data

If you want to get up to date on IR for search and RAG, this book is the book you're looking for. https://lnkd.in/dTvaY3sq

This will unlock vector search for the 10-100x number of use cases where it has been too expensive until now. A must read if you're using embeddings.

Recommender systems need to multiply very large sparse matrices. e-commerce platform FARFETCH leverages Vespa's support for sparse and dense tensors + vector search to do this online in less then 100 milliseconds. To scale this you need a platform that provides all the elements of the solution as an integrated whole where computation and data is organized to minimize latency and scale to any size and load: "One of our major challenges is to do it without increasing our infrastructure indefinitely. To tackle this scalability challenge, we choose to use a vector database and Vespa matches our necessity because of all the available features: - Tensor Operations: Dense and sparse vector operations (e.g. map, reduce, slice, etc.). - Complex Ranking: Compute different signals (e.g. bm25, dot-product, cosine, etc.) and use them freely to rank document matches. - Keyword Search: You can use Vespa like Elasticsearch. - LTR: Native model service that can be used for ranking (e.g. LTR). - Filtering: Pre and post-filtering that you can tune to fit your use cases. - Performance: C++ code optimised to use SIMD operations when possible. With this, we intend to serve all recommendations for all platforms using Vespa and have no scalability issues when a new online retailer joins our services."

The vector database Marqo announces that they have chosen to build on Vespa after extensive benchmarking: "For Marqo 2, we looked at a number of open source and proprietary vector databases, including Milvus, Vespa, OpenSearch (AWS managed), Weaviate, Redis, and Qdrant." Vespa came out as the clear winner: "Our internal benchmarks revealed that Vespa excelled as the optimal choice, satisfying all the criteria mentioned above, including being highly performant. For example, with 50M vectors, Vespa had a P50 latency of 16ms vs 140ms for Milvus for an infrastructure identical in cost." and compared to their previous OpenSearch backed version: "this configurability was invaluable in reducing Marqo’s latency by more than half and increasing its throughput by up to 2x, compared to Marqo 1." The post contains some great insights into what people overlook when benchmarking: "Published benchmarks frequently fail to answer many questions that are critical in choosing a vector database for high throughput production vector search at very large scales. For instance: - In production use cases, it is common to encounter tens or even hundreds of millions of vectors. The performance of vector databases for such large numbers of vectors is a critical consideration. Most benchmarks, however, tend to focus on relatively small datasets. - How does the vector database perform under concurrent indexing and search, as well as ongoing mutation of indexed documents? Excellent search performance despite high throughput indexing and updates is a requirement for business-critical use cases. - How do different vector databases compare in terms of space complexity and memory efficiency? - What does it take to have a highly available deployment that can meet strict SLAs despite node failures?" Benchmarking is hard, but when done properly, Vespa tends to come out on top.

Migrating search and recommendation systems from Elastic to Vespa has become very popular. Here the leading Japanese jobs site Stanby details their experience: https://lnkd.in/dVG9DNqX