Quickstart with LanceDB Cloud with Alph
🚀 If you haven't signed up for LanceDB Cloud yet, click here to get started!
Welcome to LanceDB Cloud!
In this notebook, we show you how to get started with LanceDB Cloud. Specifically, we will:
- Connect to LanceDB Cloud
- Ingest data
- Create a vector index
- Run queries
Step 1: Install LanceDB
Step 2: Load environment variables and connect to LanceDB Cloud
We use python-dotenv to load credentials from a .env file. Your .env should contain:
LANCEDB_API_KEY=sk_...
LANCEDB_URI=db://your-db-uri
- Get the
db uri
db uri starts with db://, which can be obtained from the project page on the dashboard.
- Get the
api key
api key starts with sk_, which can be obtained from the dashboard by navigating to the API Keys page.
Step 3: Ingest Data
We use the ag_news dataset from HuggingFace, which includes 768-dimensional precomputed embeddings. To optimize performance, we extract the first 3,000 rows from the test split for this example.
ℹ️ There are various ways to specify the table schema. More details can be found in our documentation.
Step 4: Create a vector index
We will create a vector index on the keywords_embeddings column.
⚠️ WARNING: create_index is asynchonous so it returns when indexing is in progress. We provide the list_indices and index_stats APIs to check index status. The index name is formed by appending “_idx” to the column name. Note that list_indices will not return any information until the index has fully ingested and indexed all available data.
⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ⏳ Waiting for keywords_embeddings_idx to be ready... ✅ keywords_embeddings_idx is ready! IndexStatistics(num_indexed_rows=3000, num_unindexed_rows=0, index_type='IVF_PQ', distance_type='cosine', num_indices=1, loss=None)
Step 5: Query
Let's perform a search. Note here that only the text, keywords and label columns will be returned
toyota, profit, carmaker
Let's perform another search to filter by the label column
toyota, profit, carmaker
🎲 Try it - powered by the LanceDB Cloud vector index you just created.
Search the index hosted in LanceDB cloud yourself!
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading weights: 0%| | 0/199 [00:00<?, ?it/s]
MPNetModel LOAD REPORT from: sentence-transformers/all-mpnet-base-v2 Key | Status | | ------------------------+------------+--+- embeddings.position_ids | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
🔍 Search query: "artificial intelligence robots" 📊 Settings: top-5 | distance ≤ 1.5 | filter: All ✅ Found 5 results within distance ≤ 1.5
Step 6: Cleanup
We can now delete the table.
🎉 Congrats! You just built your first semantic search application with LanceDB Cloud!