About interface-datastore
The npm package "interface-datastore" serves as a vital component for developers working with various storage solutions, particularly within the IPFS (InterPlanetary File System) ecosystem. This Node.js module establishes a consistent interface for datastore implementations, ensuring that developers can write adaptable and modular code that interacts seamlessly with different types of backend storage systems. By using "interface-datastore," developers benefit from a standardized approach that simplifies the integration of new storage solutions without the need to overhaul existing applications. This promotes greater code reusability and system interoperability, crucial aspects for scalable and maintainable software development.
To incorporate "interface-datastore" into your Node.js project, you can easily add it using the npm command `npm install interface-datastore`. This command simplifies the installation process, allowing developers to quickly integrate the interface into their projects and start leveraging its functionalities. Once installed, "interface-datastore" provides a set of common API specifications that any datastore implementation must adhere to, such as methods for put, get, has, delete, and more. These APIs are designed to work with asynchronous operations, supporting both callback and promise-based programming styles, which are essential for handling I/O operations efficiently in modern web applications.
"Interface-datastore" is not only a tool for uniformity but also enhances the robustness of applications handling data across various storage backends. It supports a broad range of datastore types, from in-memory and file-based systems to more complex implementations like databases or custom storage solutions. This flexibility makes it an indispensable tool for developers looking to build applications that are both resilient and versatile, capable of operating across different environments and scaling as needed. As part of the IPFS project, "interface-datastore" also underscores the commitment to a decentralized web, where applications can run smoothly regardless of the underlying data storage technology.
Dependencies
Core dependencies of this npm package and its dev dependencies.
aegir, npm-run-all
Documentation
A README file for the interface-datastore code repository. View Code
interface-datastore
datastore interface
About
A Datastore is a key/value database that lets store/retrieve binary blobs using namespaced Keys.
It is used by IPFS to store/retrieve arbitrary metadata needed to run the node - DHT provider records, signed peer records, etc.
Backed Implementations
- File System:
datastore-fs - IndexedDB:
datastore-idb - level:
datastore-level(supports any levelup compatible backend) - Memory:
datastore-core/memory - S3:
datastore-s3
Wrapper Implementations
- Keytransform:
datstore-core/src/keytransform - Mount:
datastore-core/src/mount - Namespace:
datastore-core/src/namespace - Sharding:
datastore-core/src/sharding - Tiered:
datstore-core/src/tiered
If you want the same functionality as go-ds-flatfs, use sharding with fs.
Example
import FsStore from 'datastore-fs'
import { ShardingDataStore, shard } from 'datastore-core'
const fs = new FsStore('path/to/store')
// flatfs now works like go-flatfs
const flatfs = await ShardingStore.createOrOpen(fs, new shard.NextToLast(2))
Test suite
Available via the interface-datastore-tests module
import { interfaceDatastoreTests } from 'interface-datastore-tests'
describe('mystore', () => {
interfaceDatastoreTests({
async setup () {
return instanceOfMyStore
},
async teardown () {
// cleanup resources
}
})
})
Aborting requests
Most API methods accept an [AbortSignal][] as part of an options object. Implementations may listen for an abort event emitted by this object, or test the signal.aborted property. When received implementations should tear down any long-lived requests or resources created.
Concurrency
The streaming (put|get|delete)Many methods are intended to be used with modules such as it-parallel-batch to allow calling code to control levels of parallelisation. The batching method ensures results are returned in the correct order, but interface implementations should be thread safe.
import batch from 'it-parallel-batch'
const source = [{
key: ..,
value: ..
}]
// put values into the datastore concurrently, max 10 at a time
for await (const { key, data } of batch(store.putMany(source), 10)) {
console.info(`Put ${key}`)
}
Keys
To allow a better abstraction on how to address values, there is a Key class which is used as identifier. It's easy to create a key from a Uint8Array or a string.
const a = new Key('a')
const b = new Key(new Uint8Array([0, 1, 2, 3]))
The key scheme is inspired by file systems and Google App Engine key model. Keys are meant to be unique across a system. They are typically hierarchical, incorporating more and more specific namespaces. Thus keys can be deemed 'children' or 'ancestors' of other keys:
new Key('/Comedy')new Key('/Comedy/MontyPython')
Also, every namespace can be parameterized to embed relevant object information. For example, the Key name (most specific namespace) could include the object type:
new Key('/Comedy/MontyPython/Actor:JohnCleese')new Key('/Comedy/MontyPython/Sketch:CheeseShop')new Key('/Comedy/MontyPython/Sketch:CheeseShop/Character:Mousebender')
Install
$ npm i interface-datastore
Browser <script> tag
Loading this module through a script tag will make it's exports available as InterfaceDatastore in the global namespace.
<script src="https://unpkg.com/interface-datastore/dist/index.min.js"></script>
API Docs
License
Licensed under either of
- Apache 2.0, (LICENSE-APACHE / http://www.apache.org/licenses/LICENSE-2.0)
- MIT (LICENSE-MIT / http://opensource.org/licenses/MIT)
Contribute
Contributions welcome! Please check out the issues.
Also see our contributing document for more information on how we work, and about contributing in general.
Please be aware that all interactions related to this repo are subject to the IPFS Code of Conduct.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
