Skip to main content
Module

x/earthstar/src/entries/universal.ts

Storage for private, distributed, offline-first applications.
earthstar-project/earthstar
Go to Latest
import * as earthstar from "https://deno.land/x/earthstar@v10.0.2/src/entries/universal.ts";

Earthstar is a small and resilient distributed storage protocol designed with a strong focus on simplicity and versatility, with the social realities of peer-to-peer computing kept in mind.

This is a reference implementation written in Typescript. You can use it to add Earthstar functionality to applications running on servers, browsers, the command line, or anywhere else JavaScript can be run.

Example usage

import { Replica, ReplicaDriverMemory, Crypto, Peer } from "earthstar";

const shareKeypair = await Crypto.generateShareKeypair("gardening");

const replica = new Replica({
	 driver: ReplicaDriverMemory(shareKeypair.shareAddress),
	 shareSecret: shareKeypair.secret,
});

const authorKeypair = await Crypto.generateAuthorKeypair("suzy");

await replica.set(authorKeypair, {
	 path: "/my-note",
	 text: "Saw seven magpies today",
});

const allDocs = await replica.getAllDocs();

const peer = new Peer();

peer.addReplica(replica);

peer.sync("https://my.server")

This module also exposes server APIs for for building always-online peers. The below example reads some on-disk JSON to initiate some share replicas, and stores their data using the filesystem.

import {
ExtensionKnownShares,
ExtensionSyncWeb,
Server,
} from "https://deno.land/x/earthstar/mod.ts";

const server = new Server([
new ExtensionKnownShares({
	 knownSharesPath: "./known_shares.json",
	 onCreateReplica: (shareAddress) => {
		 return new Earthstar.Replica({
			 driver: new ReplicaDriverFs(shareAddress, "./share_data"),
		 });
	 },
}),
new ExtensionSyncWebsocket(),
]);

Classes

An attachment driver which persists attachments in memory. Works everywhere.

A server won't accept writes

An in-memory replica driver. Its contents will be lost when it is closed. Works everywhere.

Generic top-level error class that other Earthstar errors inherit from.

A replica holding a share's data, used to read, write, and synchronise data to.

A server URL is bad or the network is down

A syncing partner to be used with local instances of IPeer. Works everywhere.

A syncing partner created from an inbound HTTP connection (i.e. a web client).

Holds many shares' replicas and manages their synchronisation with other peers. Recommended as the point of contact between your application and Earthstar shares.

A replica holding a share's data, used to read, write, and synchronise data to.

A cached, synchronous interface to a replica, useful for reactive abstractions. Always returns results from its cache, and proxies the query to the backing replica in case of a cache miss.

An IReplica or IReplicaDriver was used after close() was called on it.

A replica driver which stores data in memory. All data is lost when the replica is closed.

An IReplica or IReplicaDriver was used after close() was called on it.

Get and set values from a common pool of settings for Earthstar clients, such as an author, shares, share secrets, and servers.

Syncs the data of a Peer's replicas with that of another peer.

Validation failed on a document, share address, author address, etc.

Variables

Lowercase alphabetical characters.

Uppercase alphabetical characters.

All characters allowed in an identity's public address.

All characters allowed in an identity's pub key.

All characters allowed in an identity's short name.

All characters allowed in base32.

Higher-level crypto functions. Mostly used for generating new author and share keypairs.

A version of the ICryptoDriver interface backed by noble/ed25519. The slowest crypto driver available, but works everywhere.

All digits.

Validator for the 'es.4' format. Checks if documents are spec-compliant before ingesting, and signs them according to spec.

Format for 'es.5' documents. Supports attachments and share keypairs.

Helper for querying Earthstar docs using a glob-style query string.

A helper used by globToQueryAndRegex -- see that function for details.

All characters permitted in a document's path.

All special characters permitted in a document's path.

All characters allowed in a share's address.

All charaters allowed in a share's key.

All characters allowed in a share's name.

Functions

Put a short name and pub key together into an identity address.

Put a share name and encoded pub key together into a share address.

Encode uint8array bytes to base32 string

Decode base32 string to a uint8array of bytes. Throw a ValidationError if the string is bad.

Check that an identity address is valid.

Check that a share address is valid.

example usage: myArrayOfArrays.sort(arrayCompare)

Convert a keypair (author / share) back into a raw KeypairBytes for use in crypto operations.

Return whether a document is expired or not

Combine a shortname with a raw KeypairBytes to make an AuthorKeypair

Combine a name with a raw KeypairBytes to make an ShareKeypair

Returns a valid share address generated using a given name.

Given an array of format names, and an array of IFormat, returns an array of IFormat restricted to those with matching names.

Returns an object with format names as keys, and corresponding IFormat as values.

Returns the default formats if no formats are given.

Returns the default format if no formats are given.

Check if any value is a subclass of EarthstarError (return true) or not (return false)

Check that a string contains only printable ASCII

Check if any value is a subclass of EarthstarError (return false) or not (return true)

Check that a string only contains character from a string of allowed characters.

Parse an author address into its parts.

Parse a share or author address into its parts

Parse a share address into its parts.

This is a low-level helper for the template matching code; probably don't use it directly.

Find documents whose path matches the glob string. See documentation for globToQueryAndRegex for details on glob strings.

Given a template string like "/posts/{postId}.json", query the replica for docs with matching paths.

Set the crypto driver used for all cryptographic operations.

Returns a promise which is fulfilled after a given number of milliseconds.

Interfaces

A keypair used by individual entities to sign documents.

The core properties all documents must implement, regardless of format.

Contains data written and signed by an identity.

Contains data written and signed by an identity.

A partial es.4 doc that is about to get written. The rest of the properties will be computed automatically.

A partial es.5 doc that is about to get written. The rest of the properties will be computed automatically.

Higher-level crypto functions. Not used directly for the most part, but useful for generating new keypairs.

A crypto driver provides low-level access to an implementation providing ed25519 cryptography, e.g. Chloride, noble/ed25519, Node crypto.

Formats are each responsible for one document format such as "es.4". They are used for signing and validating documents, as well as generating new documents from a given input.

Holds many shares' replicas and manages their synchronisation with other peers. Recommended as the point of contact between your application and Earthstar shares.

An attachment driver provides low-level access to a replica's attachments. ReplicaAttachmentDrivers are not meant to be used directly by users; let the Replica talk to it for you.

A document driver provides low-level access to a replica's documents. ReplicaDocDrivers are not meant to be used directly by users; let the Replica talk to it for you.

A replica driver provides low-level access to a replica's documents and attachments. ReplicaDrivers are not meant to be used directly by users; let the Replica talk to it for you.

Provides a syncer with the means to connect the peer being synced with (the partner).

Options for configuring a new replica.

  • validators: Validators for the kinds of documents this replica will replicate, e.g. FormatValidatorEs4.
  • driver: A driver the replica will use to read and persist documents.

Describes a query for fetching documents from a replica.

Filters a query by document attributes.

Options to initialise a Syncer with.

Type Aliases

An identity's public address.

The human-identifiable portion of an identity's public address, e.g. suzy.

An attachment associated with a document.

A short string with a timestamp and hash of the document's path and author.

A document with it's attachment merged onto a new attachment property.

Verifies a given type is of type IFormat

Extracts a IFormat's config type

Extracts a IFormat's document type, e.g. DocEs5

Extracts a IFormat's input type, used to generate a new document.

Extracts a IFormat's name, e.g. es.5

Verifies a given type is an array of IFormat

Represents fetching all historical versions of a document by authors, or just the latest versions.

  • IngestEventSuccess — a new doc was written
  • IngestEventFailure — refused an invalid doc
  • IngestEventNothingHappened — ingested an obsolete or duplicate doc

The path of a document, e.g. /images/teapot.png.

An event representing when a QuerySource has processed all existing documents.

  • ExpireEvent - An ephemeral document has expired
  • IngestEvent — the result of a replica ingesting a document
  • DocAlreadyExists — processing an old doc as you catch up

A mode representing what kind of docs are desired from a query stream.

  • existing - Only pre-existing documents.
  • new - Only documents written after the stream is initiated
  • everything - Both pre-existing and incoming documents.
  • DocAlreadyExists — processing an old doc as you catch up
  • IngestEvent — the result of a replica ingesting a document
  • ExpireEvent - An ephemeral document has expired
  • AttachmentIngestEvent - A new attachment has been ingested
  • AttachmentPruneEvent - An attachment without a corresponding document has been pruned.
  • ReplicaEventWillClose — the replica is about to close
  • ReplicaEventDidClose — the replica has closed

A share's public address.

A keypair used to write to a specific share

The human-identifiable portion of a share's address, e.g. gardening.

An event with an Earthstar document and corresponding ID.

A type of message one SyncAgent can send to another.

An event sent when a SyncAgent doesn't want anything anymore, though it'll still serve HAVE requests.

A special event for the implementation of a PlumTree. Asks the recipient to begin lazily messaging us.

An event to be passed on to a RangeMessenger

The current status of a SyncAgent.

Signals that a SyncAgent wants a document/documents from another SyncAgent

An 'appetite' which determines when the syncer will stop syncing.

  • once - The syncer will only attempt to reconcile existing docs and then stop.
  • continuous - Indefinite syncing, including existing docs and new ones as they are ingested into the replica.

An event for disclosing which shares a Peer has without actually revealing them. Another peer can use the salt to hash their own shares' addresses and see if they match.

An event a Syncer can send or receive.

A map of sync statuses by the share address they're associated with.

A SyncAgentEvent addressed to a specific share address.

A UNIX timestamp in microseconds.