Insert Data
Insert data into an ZBSearch database.
Whenever we create a database with ZBSearch, we must specify a schema, which
represents the entry we are going to index.
Let's say our database and schema look like this:
import { create, insert } from "zbsearch";
const movieDB = create({
schema: {
title: "string",
director: "string",
plot: "string",
year: "number",
isFavorite: "boolean",
},
});(Read more about database creation on the create page)
Insert
Data insertion in ZBSearch is quick and intuitive:
const thePrestigeId = insert(movieDB, {
title: "The prestige",
director: "Christopher Nolan",
plot: "Two friends and fellow magicians become bitter enemies after a sudden tragedy. As they devote themselves to this rivalry, they make sacrifices that bring them fame but with terrible consequences.",
year: 2006,
isFavorite: true,
});
const bigFishId = insert(movieDB, {
title: "Big Fish",
director: "Tim Burton",
plot: "Will Bloom returns home to care for his dying father, who had a penchant for telling unbelievable stories. After he passes away, Will tries to find out if his tales were really true.",
year: 2004,
isFavorite: true,
});
const harryPotterId = insert(movieDB, {
title: "Harry Potter and the Philosopher's Stone",
director: "Chris Columbus",
plot: "Harry Potter, an eleven-year-old orphan, discovers that he is a wizard and is invited to study at Hogwarts. Even as he escapes a dreary life and enters a world of magic, he finds trouble awaiting him.",
year: 2001,
isFavorite: false,
});If you have a lot of records, we suggest using the insertMultiple function as following:
const docs = [
{
title: "The prestige",
director: "Christopher Nolan",
plot: "Two friends and fellow magicians become bitter enemies after a sudden tragedy. As they devote themselves to this rivalry, they make sacrifices that bring them fame but with terrible consequences.",
year: 2006,
isFavorite: true,
},
{
title: "Big Fish",
director: "Tim Burton",
plot: "Will Bloom returns home to care for his dying father, who had a penchant for telling unbelievable stories. After he passes away, Will tries to find out if his tales were really true.",
year: 2004,
isFavorite: true,
},
{
title: "Harry Potter and the Philosopher's Stone",
director: "Chris Columbus",
plot: "Harry Potter, an eleven-year-old orphan, discovers that he is a wizard and is invited to study at Hogwarts. Even as he escapes a dreary life and enters a world of magic, he finds trouble awaiting him.",
year: 2001,
isFavorite: false,
},
];
insertMultiple(movieDB, docs);Inserting a large number of documents in a loop could potentially block the event loop.
Instead insertMultiple handles this case better.
You can pass a third, optional, parameter to change the batch size (default:
1000). We recommend keeping this number as low as possible to avoid blocking
the event loop. The batchSize refers to the maximum number of insert
operations to perform before yielding the event loop.
insertMultiple(movieDB, docs, 500);Validation rules
Defining the schema at database creation time, ZBSearch validates all the inserted documents following those rules:
- throw an error if a field has an unexpected type
- allow missing fields or fields set to
undefined - allow extra fields ignoring them
So the following document will be accepted:
import { create, insert } from "zbsearch";
const movieDB = create({
schema: {
title: "string",
year: "number",
},
});
insert(movieDB, {
title: "The prestige",
// `year` field is missing but it's ok
// year: 2006,
// Extra fields `director` and `isFavorite` will not be indexed
director: "Christopher Nolan",
isFavorite: true,
});Custom document IDs
If the id field is not found, ZBSearch will generate a random id for the document.
To provide a custom ID for a document, see the components page.
ZBSearch automatically uses the id field of the document, if found.
That means that given the following document and schema:
import { create, search } from "zbsearch";
const db = create({
schema: {
id: "string",
author: "string",
quote: "string",
},
});
insert(db, {
id: "73cbcc79-2203-49b8-bb52-60d8e9a66c5f",
author: "Fernando Pessoa",
quote: "I wasn't meant for reality, but life came and found me",
});the document will be indexed with the following id: 73cbcc79-2203-49b8-bb52-60d8e9a66c5f.
Beware
If you try to insert two documents with the same ID, ZBSearch will throw an error.
Remote document storing
By default ZBSearch keeps a copy of the inserted document in memory (and in the serialized data) to speed up search performance.
If this is not acceptable, you can provide a custom documentsStore component which will be responsible to store
and fetch documents from another location (local or remote).
The code example below is an example that implements a proxy: when a document is requested, the code finds it on a location of the filesystem.
We assume each document has an id field which disable ZBSearch random ID generation.
You can replace the file related operations with your custom code.
import { readFile, readdir, writeFile, rm } from "node:fs/promises";
import { resolve } from "node:path";
import { create } from "zbsearch";
const ROOT_LOCATION = "/var/db/orama-example";
async function getDocument(id) {
return JSON.parse(
await readFile(resolve(ROOT_LOCATION, `${id}.json`), "utf-8")
);
}
async function listDocuments() {
const allFiles = await readdir(ROOT_LOCATION);
return allFiles.filter((id) => id.endsWith(".json"));
}
const database = create({
schema: {
title: "string",
director: "string",
},
components: {
// override partially the default documents store
documentsStore: {
create() {
return {};
},
load(raw) {
return {};
},
save(store) {
return {};
},
get(_, id) {
return getDocument(id);
},
getMultiple(_, ids) {
return Promise.all(
ids.map(async (id) => {
return JSON.parse(await getDocument(id));
})
);
},
async getAll() {
const docs = await listDocuments();
return Promise.all(
docs.map(async (id) => {
return JSON.parse(await getDocument(id));
})
);
},
store() {
// No-op
},
remove() {
// No-op
},
async count() {
const docs = await listDocuments();
return docs.count;
},
},
},
});