Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: add diskann index type #9677

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SelinaTang21
Copy link

Motivation and Context

Add a new vector index type diskann to Azure Cosmos DB Mongo vCore vector store.
Issue #9676

Why is this change required? This change enables MongoDB vCore to support the new diskann vector index as a preview feature for vector search. The diskann index, as described in the DiskANN paper, offers an additional option for fast nearest neighbor searches.

Description

Contribution Checklist

@SelinaTang21 SelinaTang21 requested a review from a team as a code owner November 13, 2024 09:53
@markwallace-microsoft markwallace-microsoft added python Pull requests for the Python Semantic Kernel memory labels Nov 13, 2024
@github-actions github-actions bot changed the title add diskann index type Python: add diskann index type Nov 13, 2024
@markwallace-microsoft
Copy link
Member

Python Unit Test Overview

Tests Skipped Failures Errors Time
2869 4 💤 0 ❌ 0 🔥 1m 15s ⏱️

@eavanvalkenburg
Copy link
Member

Hi @SelinaTang21 thanks for this, however we are no longer making updates to the existing memory stores, we are moving to the new model with a VectorStore and VectorStoreRecordCollection, this has not been done yet for MongoDB version of Cosmos, the NoSQL version was just added to main this week, we would love contributions to the new model and we already have DISK_ANN in there (see data/const.py), so for that one this can be adopted directly. We still need to create samples showing how to go from the old to the new stores but in essence that is quite easy since with the new model you can just create your own data model, including one that matches the old one, off course that does depend on the new implementation of the Mongo DB store and collection! Implementing the new one shouldn't be too difficult, it takes only a couple of functions to implement but it is currently not at the top of our list, so have a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
memory python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants