- data is named, not content addressed (to support overwrite & deletion), so the primitive data type is associative arrays of names to documents instead of sets of messages
- documents are grouped into "shares" and you have to know the "share address" before the server will acknowledge that it indeed has documents in that share.
Seems like an interesting approach: part way private, part way open. It think projects like this which pick an extreme on that axis end up suffering for it--which is most of them.
I bet it would succeed at keeping the spammers and scammers out, but I'd be leery of using it with a large group against a motivated adversary--if a single user exposes the share address (which is in a gray area between public and private) it seems like all of the users of that share are exposed. I'd prefer to see a web of trust so that an adversary has to compromise each hop on the trust graph before they find their target--but that's a lot of complexity for users to handle so I can see why keeping it simple might be worth tolerating the lack of granularity.
I certainly hope that something like this succeeds. It feels like we've lost the web and there are too many of us to coordinate en masse without something web-like--and at a time where that coordination is pretty important.
nine_k 3 days ago [-]
Hmm, a proper capability-based design would, of course, give each user a separate share address, which could be disabled at any time.
Share addresses should be handled much like ssh keys.
__MatrixMan__ 1 days ago [-]
I haven't used this, just explored the docs a bit, so maybe I'm misunderstanding, but my feeling is that one-share-per-user is too small--you could only use that to chat with yourself.
Havoc 4 days ago [-]
Not entirely following what this is?
A document and media database of some sort with lots of JavaScript involved somehow? Aimed at something vaguely self-host-ish? With mastodon-ish replication to peer nodes?
Seems like there is an ambitious grand vision happening here of sorts but not quite seeing it.
danjl 3 days ago [-]
The distributed part of Earthstar seems like the selling point? Otherwise, it certainly does not have the range of database capabilities of either PGlite or SQLite, which also work offline, store data locally, and work in the browser. What do you get from the distributed nature of Earthstar? Backup? That's pretty simple with both SQLite and PGlite too, so I'm not sure I see a big benefit for most db users. The win seems to come from people who want to store data on their machines AND share that data with other individuals who also get local copies. Maybe this is for pirating content?
dpflug 3 days ago [-]
It could be useful for, for instance, sharing photos with family, meeting notes, etc. Doesn't have to be piracy.
danjl 3 days ago [-]
So, for people who are paranoid about using the services that are designed for these sorts of sharing, like Dropbox, GDrive, GPhotos, and a million others? These theoretical people would suffer the frustrations of setting up and managing their own db for the added privacy? I wonder what they have in those photos and meeting notes that is so sensitive?
doodlebugging 3 days ago [-]
I don't use any of those services, though I used to use Dropbox for sharing family reunion photos with older relatives who didn't have the technical skills to do much more than click links to download photos. I am also not a potential user of Earthstar's product.
>I wonder what they have in those photos and meeting notes that is so sensitive?
As someone who is actively working on documenting my family history using photos, original copies and reproductions of official documents, old paper letters and copies of letters, I can say that the dominant reason that I do not use any of those "services that are designed for these sorts of sharing, like Dropbox, GDrive, GPhotos, and a million others" because none of the things that I am working with are any of anyone else's business.
Using many of those "services" grants perpetual ownership and license to use anywhere, forever to the service operator and should that service collapse for any reason their only asset is the contact info and personal data that their customers have chosen to entrust them with.
Frankly speaking here, I have been using computers for a long time and the most dominant recent trend among tech-bros has been to conjure an app or service to con people into paying SaaS fees even if they stop using the app or service. There are fewer apps available that can be bought as a pay once use it forever app.
I'm not interesting in playing anyone's con games and the software industry is full of them as many here note whenever someone does a new "Show HN" post. Oftentimes some of the first comments note for other interested parties that there is a requirement to give an email address to test the app or to register an account just to be able to see for ourselves what it does and whether it is truly useful.
I'm not interested in helping fund someone's second or third or fourth income stream when it requires handing control of something I own outright to a third party who will make every attempt to monetize every part of the transactional relationship that develops when a user registers an account.
I do not trust companies or individuals to have my best interests in mind when I am using their products or services.
Since the materials that my family members have collected and preserved for generations can't be replaced and we are not trying to write books or shoot films about any of the individuals who may have led interesting lives it is none of anyone else's business what the photos depict or what the documents tell us about how each branch of the tree connects to an individual's trunk.
The archive of the work I'm doing will be available to my family members. Some of them unfortunately use social media to keep in touch with friends and family. I can't control what they do with the results of my work but I can control everything up to the point where I distribute it and it's my decision to keep it all local on storage that I own and control.
The tone of your last sentence is accusatory and, like this last sentence of mine is out of place.
upghost 3 days ago [-]
Damn, preach. Epic defense of privacy, doodlebugging. Thank you for giving voice to my thoughts.
doodlebugging 3 days ago [-]
I really appreciate you taking the time to read all of that and I appreciate your supportive reply.
Preserving one's privacy to the fullest extent should be one of the main characteristics of a well-informed citizenry and not an excuse to demonize people who exercise their rights to withhold private information from entities that have no business obtaining that private information.
It's too easy today for information to be collected, profiles built, and those profiles used to silently discriminate against a person or a group of people who have something in common. There is limited or no accountability in the system since many of these capabilities evolved much faster than any regulatory framework could. Once the money started flowing in the data collection space, it acted to restrict the pace of regulatory development since some of those monies were spent to lobby individuals or groups who had the ability and the responsibility to protect privacy rights.
Those of us normal people out here who all contributed to the widening of the data stream by signing up as users of these predatory apps and services are the ultimate losers, having effectively ceded control of our own privacy rights to the extent that we can all be easily profiled, monitored, and predictive tools can be configured to allow any new business relationship to be an adversarial relationship where a profile of the new customer informs the service provider about how they should treat the new customer in order to maximize profits from that individual.
It's wrong. It's predatory. It's an unfortunate consequence of the selfish greed that has become an acceptable lifestyle choice for many in the software industry.
But, here I sit. I'm old and I ramble and at the end of the day my words are weak signal in a high background noise stream that looks exciting to the adventurous ones too inexperienced to understand the risks they ask us all to accept.
escapecharacter 4 days ago [-]
An equivalent term is "local first", where there seems to be more activity.
erikerikson 4 days ago [-]
I came to comment this.
What's the use case?
How would I use it?
Why would I want to use it?
Definitely some interesting features but the vision seemed lost in their listing.
brendanyounger 4 days ago [-]
I believe this is superseded by the Willow protocol [0] which was posted here a while back.
I tried to read about Willow, but could never make heads or tails of it [1]. It seems to be a hierarchical key value store with a bunch of extra fields for timestamps and cryptographic tokens, but it's woefully short on working examples. Even the recent Rust library [2] is effectively a bunch of abstract classes with implementations that just throw errors.
I could make a quip about the state of European tech sophistication, but I'll let it pass with a polite smile.
> All of these above features are enabled by the new Willow General Purpose Sync Protocol we've been working on. It's a new protocol inspired by Earthstar, refining and building upon its concepts. We will be publishing a new website with complete specifications for this protocol.
> If Earthstar feels very similar to Willow, then that is no coincidence. Willow started out as a reimagining of Earthstar, future Earthstar versions will build upon Willow, and the core maintainer of Earthstar is one of the two Willow authors.
jazzyjackson 4 days ago [-]
I was going to say what the sibling said, no need for snark, NLnet funds a ton of cool projects. If I were to judge the quality of software written in silicon valley on the vc funded projects that don't pan out, god help me
culi 3 days ago [-]
The website literally says "Earthstar has a new specification powered by Willow." in a banner at the very top
omnimus 4 days ago [-]
Yeah one person open-source project says everything about the state of European tech sophistocation.
PaulRobinson 4 days ago [-]
Just like how Linux and Python started out and crashed and burned and nobody ever uses them any more lol
Mate, what are you actually on about?
saranshsharma 4 days ago [-]
[dead]
qwertox 4 days ago [-]
Does a Python client exist? The only thing I see in the docs which is related to Python is
I would recommend instead jellyfin, immich, and maybe just plain sftp for all the rest. You can sync (err, become distributed) by way of git, syncthing, or rsync.
What I haven't seen yet is a good self hosted identity manager, SSO sort of thing. I believe bluesky is working on one but I dont know if it will be self hostable (Freudian autocorrect: self hostage)
Another thing I haven't figured out yet with my homelab/private cloud is how to be my own CA and manage access to my machine without teaching my counterparties ssh. I want something like NordVPN offers in their meshnet: anyone logged into the family plan can see each other's machines as if they were local network. Can wireguard accomplish this? I think this is what yggdrasil is too but haven't dug into it.
globular-toast 4 days ago [-]
Isn't this a library for building your own applications? I don't see how jellyfin etc is relevant here.
By "good" identity management I guess you mean something other than OpenLDAP or 389? Not sure how well these work with Windoze if you care about that.
Being your own CA is easy. Installing that CA in all the machines you want to is the sucky bit. It all works if you use let's encrypt.
Wireguard can let you join two LANs together over the internet but you'll probably want two wireguard capable routers on each end (like pfsense or something). Otherwise you'll have to install special routing configs on to each client machine. Forwarding avahi (for auto-discovery) over the link is something I haven't figured out yet.
jazzyjackson 3 days ago [-]
I see what you mean, I was just going off the byline Works offline.Store music, photos, video.Actually delete stuff.Temporary documents.Live syncing.Use one or many identities.Sneakernets.Always self-hosted.
I’ve been struggling creating clientside certs for mTLS, seems like it will work for most browsers that let you install certs, but not for an app (like the immich and jellyfin apps) so I’ll probably end up with a VPN solution, easy enough to tell my sister to open this app and hit the button before trying to connect.
dudus 4 days ago [-]
I'll suggest maybe tailscale is what you are looking for to share with family a VLAN
idle_zealot 4 days ago [-]
Adding on to this: you can host your own Tailscale-compatible server with Headscale, if you want to be totally independent.
jazzyjackson 3 days ago [-]
I’m going to dig into Headscale with Authelia for OIDC, that’s pretty close to what I was imagining, found this tutorial [0] (wow posted just 4 days ago). Thanks ya’ll.
It's not the same. It's not a massive binary, but it's not tiny either.
wongarsu 4 days ago [-]
Compressed less than 800kB including some tests and demo code. I wouldn't want to include that on the homepage, but as part of a webapp it would be completely fine.
rkagerer 4 days ago [-]
What did they sunset?
TJSomething 4 days ago [-]
Web SQL, which was Chrome's implemented proposal to add SQLite to the JavaScript standard library. Chrome said they scrapped it because SQLite wasn't really intended to be used in a security environment as hostile as the web.
Mozilla did not implement it because it's not any kind of standard subset of SQL and no such subset is agreed upon while also useful. They didn't want to tie web standards, supported features in browser releases, and SQLite's release lifecycle to each other.
hinkley 3 days ago [-]
I heard that since there’s only one SQLite vendor and nothing else comparable to allow for multiple options, they wouldn’t support it.
I would like to hear more about the hostile environment comment. I thought they sandboxed every page. Cross origin data theft?
randomcatuser 4 days ago [-]
I think this will really benefit from blog posts!! (ie, why did we design it like this?/what were the alternatives?)
At a first glance, it sounds interesting though! identities being keypairs, files, content that is accessible via name...
canadiantim 4 days ago [-]
Any possibility of implementation in different languages like Python? I like everything about this except that it's Javascript and I'm still cautious about using javascript for something like this
__MatrixMan__ 4 days ago [-]
Applications like this are why I like languages like nim and gleam, which compile to both javascript and something else. This lets my app run on a raspberry pi or an esp32 or in a browser tab.
Unfortunately neither are especially popular, so we end up asking implementations in multiple languages, which must be a drag for maintainers of projects like this. Alas, here we are. If not one of those, then I too would be interested in a python client library.
mrbluecoat 4 days ago [-]
> Always self-hosted. Servers optional. No blockchain. No tokens. Free forever, in every sense.
Best thesis statement ever.
ofrzeta 4 days ago [-]
[wrong thread, sorry]
CalRobert 4 days ago [-]
Wrong place?
jlkuester7 4 days ago [-]
Honestly it kinda feels like they just reimplemented CouchDB/PouchDB....
Not that it would be a bad thing to do that, but really, for small scale stuff Couch works pretty okay. My first impressions of Earthstar do not lead me to believe they have a particular emphasis on performance at scale.
kapad 3 days ago [-]
JAWA = Just Another Web3 App
gwbas1c 4 days ago [-]
What applications run on top of this?
Without some seed applications, it's hard to know if this actually works, or is just mental gymnastics.
sunshine-o 4 days ago [-]
[dead]
Rendered at 05:23:05 GMT+0000 (Coordinated Universal Time) with Vercel.
- data is named, not content addressed (to support overwrite & deletion), so the primitive data type is associative arrays of names to documents instead of sets of messages
- documents are grouped into "shares" and you have to know the "share address" before the server will acknowledge that it indeed has documents in that share.
Seems like an interesting approach: part way private, part way open. It think projects like this which pick an extreme on that axis end up suffering for it--which is most of them.
I bet it would succeed at keeping the spammers and scammers out, but I'd be leery of using it with a large group against a motivated adversary--if a single user exposes the share address (which is in a gray area between public and private) it seems like all of the users of that share are exposed. I'd prefer to see a web of trust so that an adversary has to compromise each hop on the trust graph before they find their target--but that's a lot of complexity for users to handle so I can see why keeping it simple might be worth tolerating the lack of granularity.
I certainly hope that something like this succeeds. It feels like we've lost the web and there are too many of us to coordinate en masse without something web-like--and at a time where that coordination is pretty important.
Share addresses should be handled much like ssh keys.
A document and media database of some sort with lots of JavaScript involved somehow? Aimed at something vaguely self-host-ish? With mastodon-ish replication to peer nodes?
Seems like there is an ambitious grand vision happening here of sorts but not quite seeing it.
>I wonder what they have in those photos and meeting notes that is so sensitive?
As someone who is actively working on documenting my family history using photos, original copies and reproductions of official documents, old paper letters and copies of letters, I can say that the dominant reason that I do not use any of those "services that are designed for these sorts of sharing, like Dropbox, GDrive, GPhotos, and a million others" because none of the things that I am working with are any of anyone else's business.
Using many of those "services" grants perpetual ownership and license to use anywhere, forever to the service operator and should that service collapse for any reason their only asset is the contact info and personal data that their customers have chosen to entrust them with.
Frankly speaking here, I have been using computers for a long time and the most dominant recent trend among tech-bros has been to conjure an app or service to con people into paying SaaS fees even if they stop using the app or service. There are fewer apps available that can be bought as a pay once use it forever app.
I'm not interesting in playing anyone's con games and the software industry is full of them as many here note whenever someone does a new "Show HN" post. Oftentimes some of the first comments note for other interested parties that there is a requirement to give an email address to test the app or to register an account just to be able to see for ourselves what it does and whether it is truly useful.
I'm not interested in helping fund someone's second or third or fourth income stream when it requires handing control of something I own outright to a third party who will make every attempt to monetize every part of the transactional relationship that develops when a user registers an account.
I do not trust companies or individuals to have my best interests in mind when I am using their products or services.
Since the materials that my family members have collected and preserved for generations can't be replaced and we are not trying to write books or shoot films about any of the individuals who may have led interesting lives it is none of anyone else's business what the photos depict or what the documents tell us about how each branch of the tree connects to an individual's trunk.
The archive of the work I'm doing will be available to my family members. Some of them unfortunately use social media to keep in touch with friends and family. I can't control what they do with the results of my work but I can control everything up to the point where I distribute it and it's my decision to keep it all local on storage that I own and control.
The tone of your last sentence is accusatory and, like this last sentence of mine is out of place.
Preserving one's privacy to the fullest extent should be one of the main characteristics of a well-informed citizenry and not an excuse to demonize people who exercise their rights to withhold private information from entities that have no business obtaining that private information.
It's too easy today for information to be collected, profiles built, and those profiles used to silently discriminate against a person or a group of people who have something in common. There is limited or no accountability in the system since many of these capabilities evolved much faster than any regulatory framework could. Once the money started flowing in the data collection space, it acted to restrict the pace of regulatory development since some of those monies were spent to lobby individuals or groups who had the ability and the responsibility to protect privacy rights.
Those of us normal people out here who all contributed to the widening of the data stream by signing up as users of these predatory apps and services are the ultimate losers, having effectively ceded control of our own privacy rights to the extent that we can all be easily profiled, monitored, and predictive tools can be configured to allow any new business relationship to be an adversarial relationship where a profile of the new customer informs the service provider about how they should treat the new customer in order to maximize profits from that individual.
It's wrong. It's predatory. It's an unfortunate consequence of the selfish greed that has become an acceptable lifestyle choice for many in the software industry.
But, here I sit. I'm old and I ramble and at the end of the day my words are weak signal in a high background noise stream that looks exciting to the adventurous ones too inexperienced to understand the risks they ask us all to accept.
What's the use case?
How would I use it?
Why would I want to use it?
Definitely some interesting features but the vision seemed lost in their listing.
I tried to read about Willow, but could never make heads or tails of it [1]. It seems to be a hierarchical key value store with a bunch of extra fields for timestamps and cryptographic tokens, but it's woefully short on working examples. Even the recent Rust library [2] is effectively a bunch of abstract classes with implementations that just throw errors.
I could make a quip about the state of European tech sophistication, but I'll let it pass with a polite smile.
[0] https://willowprotocol.org/earthstar/spec/
[1] https://willowprotocol.org/specs/data-model/index.html#data_...
[2] https://github.com/earthstar-project/willow-rs/tree/main
[1] https://nlnet.nl/project/current.html
[2] https://nlnet.nl/funding.html
> All of these above features are enabled by the new Willow General Purpose Sync Protocol we've been working on. It's a new protocol inspired by Earthstar, refining and building upon its concepts. We will be publishing a new website with complete specifications for this protocol.
---
https://willowprotocol.org/more/compare/index.html#compare_e...
> If Earthstar feels very similar to Willow, then that is no coincidence. Willow started out as a reimagining of Earthstar, future Earthstar versions will build upon Willow, and the core maintainer of Earthstar is one of the two Willow authors.
Mate, what are you actually on about?
What I haven't seen yet is a good self hosted identity manager, SSO sort of thing. I believe bluesky is working on one but I dont know if it will be self hostable (Freudian autocorrect: self hostage)
Another thing I haven't figured out yet with my homelab/private cloud is how to be my own CA and manage access to my machine without teaching my counterparties ssh. I want something like NordVPN offers in their meshnet: anyone logged into the family plan can see each other's machines as if they were local network. Can wireguard accomplish this? I think this is what yggdrasil is too but haven't dug into it.
By "good" identity management I guess you mean something other than OpenLDAP or 389? Not sure how well these work with Windoze if you care about that.
Being your own CA is easy. Installing that CA in all the machines you want to is the sucky bit. It all works if you use let's encrypt.
Wireguard can let you join two LANs together over the internet but you'll probably want two wireguard capable routers on each end (like pfsense or something). Otherwise you'll have to install special routing configs on to each client machine. Forwarding avahi (for auto-discovery) over the link is something I haven't figured out yet.
I’ve been struggling creating clientside certs for mTLS, seems like it will work for most browsers that let you install certs, but not for an app (like the immich and jellyfin apps) so I’ll probably end up with a VPN solution, easy enough to tell my sister to open this app and hit the button before trying to connect.
[0] https://www.reddit.com/r/selfhosted/comments/1ic1w4q/headsca...
Mozilla did not implement it because it's not any kind of standard subset of SQL and no such subset is agreed upon while also useful. They didn't want to tie web standards, supported features in browser releases, and SQLite's release lifecycle to each other.
I would like to hear more about the hostile environment comment. I thought they sandboxed every page. Cross origin data theft?
At a first glance, it sounds interesting though! identities being keypairs, files, content that is accessible via name...
Unfortunately neither are especially popular, so we end up asking implementations in multiple languages, which must be a drag for maintainers of projects like this. Alas, here we are. If not one of those, then I too would be interested in a python client library.
Best thesis statement ever.
Not that it would be a bad thing to do that, but really, for small scale stuff Couch works pretty okay. My first impressions of Earthstar do not lead me to believe they have a particular emphasis on performance at scale.
Without some seed applications, it's hard to know if this actually works, or is just mental gymnastics.