Weaviate Similarity Search MmBERT Transformers

Similarity Search MmBERT Transformers

vector-searchvector-databaseretrieval-augmented-generationllm-frameworksweaviate-featuresfunction-callingtransformersweaviate-recipesmodel-providersPythongenerative-ai

alph-notebooks/weaviate-recipes / similarity_search_mmBERT_transformers.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

Similarity search with Multilingual ModernBERT (mmBERT) via Hugging Face and Weaviate

Read more on the blog
Check out the model: mmBERT-small

Dependencies

[1]

Connect to Weaviate

This Notebook uses the text2vec-transformers module, which is only available through Weaviate open-source via Docker or Kubernetes. This integration is not available for Weaviate Cloud (WCD) serverless instances, as it requires spinning up a container with the Hugging Face model.

This notebook uses a pre-built transformers model container. For this, create a docker-compose.yml file with the following contents:

---
services:
  weaviate:
    command:
    - --host
    - 0.0.0.0
    - --port
    - '8080'
    - --scheme
    - http
    image: cr.weaviate.io/semitechnologies/weaviate:1.32.2
    ports:
    - 8080:8080
    - 50051:50051
    restart: on-failure:0
    environment:
      TRANSFORMERS_INFERENCE_API: 'http://t2v-transformers:8080'
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-transformers'
      ENABLE_MODULES: 'text2vec-transformers'
      CLUSTER_HOSTNAME: 'weaviate-0'
  t2v-transformers:
    image: cr.weaviate.io/semitechnologies/transformers-inference:jhu-clsp-mmBERT-small
    environment:
      ENABLE_CUDA: '0'
...

and start up Docker container with

docker-compose up -d

[2]

True

[3]

{'text2vec-transformers': {'model': {'_name_or_path': 'jhu-clsp/mmBERT-small',
,   'add_cross_attention': False,
,   'architectures': ['ModernBertForMaskedLM'],
,   'attention_bias': False,
,   'attention_dropout': 0,
,   'bad_words_ids': None,
,   'begin_suppress_tokens': None,
,   'bos_token_id': 2,
,   'chunk_size_feed_forward': 0,
,   'classifier_activation': 'gelu',
,   'classifier_bias': False,
,   'classifier_dropout': 0,
,   'classifier_pooling': 'mean',
,   'cls_token_id': 1,
,   'cross_attention_hidden_size': None,
,   'decoder_bias': True,
,   'decoder_start_token_id': None,
,   'deterministic_flash_attn': False,
,   'diversity_penalty': 0,
,   'do_sample': False,
,   'dtype': 'float32',
,   'early_stopping': False,
,   'embedding_dropout': 0,
,   'encoder_no_repeat_ngram_size': 0,
,   'eos_token_id': 1,
,   'exponential_decay_length_penalty': None,
,   'finetuning_task': None,
,   'forced_bos_token_id': None,
,   'forced_eos_token_id': None,
,   'global_attn_every_n_layers': 3,
,   'global_rope_theta': 160000,
,   'gradient_checkpointing': False,
,   'hidden_activation': 'gelu',
,   'hidden_size': 384,
,   'id2label': {'0': 'LABEL_0', '1': 'LABEL_1'},
,   'initializer_cutoff_factor': 2,
,   'initializer_range': 0.02,
,   'intermediate_size': 1152,
,   'is_decoder': False,
,   'is_encoder_decoder': False,
,   'label2id': {'LABEL_0': 0, 'LABEL_1': 1},
,   'layer_norm_eps': 1e-05,
,   'length_penalty': 1,
,   'local_attention': 128,
,   'local_rope_theta': 160000,
,   'mask_token_id': 4,
,   'max_length': 20,
,   'max_position_embeddings': 8192,
,   'min_length': 0,
,   'mlp_bias': False,
,   'mlp_dropout': 0,
,   'model_type': 'modernbert',
,   'no_repeat_ngram_size': 0,
,   'norm_bias': False,
,   'norm_eps': 1e-05,
,   'num_attention_heads': 6,
,   'num_beam_groups': 1,
,   'num_beams': 1,
,   'num_hidden_layers': 22,
,   'num_return_sequences': 1,
,   'output_attentions': False,
,   'output_hidden_states': False,
,   'output_scores': False,
,   'pad_token_id': 0,
,   'position_embedding_type': 'sans_pos',
,   'prefix': None,
,   'problem_type': None,
,   'pruned_heads': {},
,   'remove_invalid_values': False,
,   'repad_logits_with_grad': False,
,   'repetition_penalty': 1,
,   'return_dict': True,
,   'return_dict_in_generate': False,
,   'sep_token_id': 1,
,   'sparse_pred_ignore_index': -100,
,   'sparse_prediction': False,
,   'suppress_tokens': None,
,   'task_specific_params': None,
,   'temperature': 1,
,   'tf_legacy_loss': False,
,   'tie_encoder_decoder': False,
,   'tie_word_embeddings': True,
,   'tokenizer_class': None,
,   'top_k': 50,
,   'top_p': 1,
,   'torchscript': False,
,   'transformers_version': '4.56.1',
,   'typical_p': 1,
,   'use_bfloat16': False,
,   'vocab_size': 256000},
,  'model_path': './models/model'}}

Create a collection

Collection stores your data and vector embeddings.

[5]

Successfully created collection: MyCollection.

Import the Data

[6]

Insert complete.

Quick check to see if all objects are in.

[7]

Query Weaviate: Similarity Search (Text objects)

Similarity search options for text objects in Weaviate:

nearText Example

Find a object in MyCollection closest to the query "What's the capital of Germany?". Limit it to only 4 responses.

[8]

ID: 126432e1-23d2-4709-aac5-6ccd19552ab1
Data: {
  "text": "The capital of Germany is Berlin."
}
Vector: [-0.002085368847474456, 0.01193431206047535, 0.0018889668863266706] ...
Distance: 0.030832409858703613 

ID: 127ed77a-9508-40c4-962c-04476d21143c
Data: {
  "text": "Beautiful weather today, perfect for a walk."
}
Vector: [-0.0008074266952462494, -0.002204334130510688, 0.007476783357560635] ...
Distance: 0.05185425281524658 

ID: 74862999-97c9-434e-8df7-f2c241a41550
Data: {
  "text": "I love to read books in my free time."
}
Vector: [0.00960230827331543, 0.005805480759590864, -0.0019345288164913654] ...
Distance: 0.052883267402648926

[10]

nearObject Example

Search through the JeopardyQuestion class to find the top 4 objects closest to id (The id was taken from the query above)

[11]

74862999-97c9-434e-8df7-f2c241a41550
ID: 74862999-97c9-434e-8df7-f2c241a41550
Data: {'text': 'I love to read books in my free time.'} 

ID: 127ed77a-9508-40c4-962c-04476d21143c
Data: {'text': 'Beautiful weather today, perfect for a walk.'} 

ID: 1f6493c3-21fa-4893-8093-8101e8f7a616
Data: {'text': 'The quick brown fox jumps over the lazy dog.'} 

ID: f0c7c042-e0ab-4ce3-b725-13a252ac2eda
Data: {'text': 'Ich lese gerne Bücher in meiner Freizeit.'}

nearVector Example

Search through the MyCollection class to find the top 2 objects closest to the query vector.

[12]

[0.00960230827331543, 0.005805480759590864, -0.0019345288164913654, -0.007073240354657173, -0.018555009737610817]
ID: 74862999-97c9-434e-8df7-f2c241a41550
Data: {'text': 'I love to read books in my free time.'} 

ID: 127ed77a-9508-40c4-962c-04476d21143c
Data: {'text': 'Beautiful weather today, perfect for a walk.'} 

ID: 1f6493c3-21fa-4893-8093-8101e8f7a616
Data: {'text': 'The quick brown fox jumps over the lazy dog.'} 

ID: f0c7c042-e0ab-4ce3-b725-13a252ac2eda
Data: {'text': 'Ich lese gerne Bücher in meiner Freizeit.'}