NIP-29 groups states

[dtonon]

It's a work in progress, I'm sharing to receive feedbacks.

[fiatjaf]

I like that you organized everything in nice tables and made everything clearer.

I also like the new group properties/capabilities.

But I really don't want everybody to rewrite everything (and I would have to rewrite it in 3 or 4 places -- and worse: keep two ways of parsing the events!, the horror!).

I think this PR should be changed to:

• adding a table that organizes all the tags/properties of a a group clearly by what they do, like you have here already.
• for the new behaviors that you introduced, turn them into new standalone tags, or secondary values to some of the existing tags (like ["closed", "approval"], for example, I don't know).

Just because all of this is arbitrary (the tags could be ["blxuswoz"] and the spec should say that means the group is private, for example), there is no point in trying to make the tags themselves look nicer. I am for having them look nicer, but given that so much software is already written then it's better to just amend the spec.

[dtonon]

Yes, the fallback to manage the old (current) flags is a mess, I reluctantly added the note. But the change is so significant that we need to offer a way to migrate.

for the new behaviors that you introduced, turn them into new standalone tags,

They (read/write/admission/lock) already are new tags.
Relays that support NIP-29 (and there aren't many, I think) can simply support legacy flags with the proposed mapping until the new spec is widely used by clients, and then drop them.

[fiatjaf]

You didn't address my feedback at all, so I will repeat in different terms:

There is zero benefit to changing the current schema.

[staab]

I will take a closer look tomorrow, but I think this is mostly unnecessary spec churn, and the new functionality can be additive. The current 4 tags cover 4 axes of access, which is actually a very clean model. I am not sure what it would mean for a "member" to be unable to post, although I understand you might want to control read/write access independently. It might make sense to build out more complete role based access control, see #2316 for a relay-level version of this (I am already planning for a group level extension of tge same concept).

[dtonon]

Actually, I should have expressed my criticism of the current NIP, in order to better contextualize the proposal.
I did not want to change things just for taste. I started from real problems I found while working with groups. Let me list them, so my goal is clearer:

1. hidden is not well defined. It says relays should hide the metadata from non-members, but it says nothing about the content. So it is unclear if a non-member can read the messages of a hidden group.

2. This creates a weak state. Today a group can be hidden from the listing but still readable by anyone who knows the id. The id cannot be changed, so if it leaks, the group is public forever. It looks like access control, but it is not.

3. closed mixes two things: "no open join" and "join only with an invite code". There is no way to say "anyone can ask to join, and an admin approves or rejects by hand". This manual approval is a very common case.

4. No emergency state. If something bad is posted, an admin or the relay owner cannot quickly freeze and hide a group while keeping the content for later inspection. The only option is to delete.

5. No fine control over writing. Today writing is only on or off (restricted or not). You cannot make a group where only admins open threads and members can only reply (a curated group). And you cannot make a group where members can only read and not post at all. A read-only group is useful as an announcement channel (many old forums work like this, and also some Telegram/WhatsApp channels), and also to temporarily block a group when there are problems, without hiding it.

6. The write default is open. Without restricted, anyone, even non-members, can write. For a "group" this is surprising.
   Side note: I think that for a group the membership should be mandatory, even if with an "auto accept" mode.

7. Adding flags later is not safe. The flags are additive: present means "more restricted", absent means "open". So if a new flag is added in the future, an old client that does not know it will ignore it and treat the group as more open than it really is. For access control, failing open is the wrong direction.

About the cost: you are right, and I agree we should avoid useless churn. Many of these points can be solved in an additive way, for example a second value on closed like ["closed", "approval"], a new locked tag, and a clarification of hidden in the text. I am happy to go this way. My goal is to fix the problems above, not to replace the schema for its own sake.

One more note for @staab: these group states are not the same as per-user roles (#2316). Roles say what a single user can do; these states say how the whole group behaves. I think both are useful and can live together.

If you agree, I can rewrite the proposal as an additive change: keep the current tags, and only add what is missing.

[fiatjaf]

1. hidden is not well defined. It says relays should hide the metadata from non-members, but it says nothing about the content. So it is unclear if a non-member can read the messages of a hidden group.

Yes, it doesn't say anything because it isn't supposed to have that property. private is the one that hides content.

2. This creates a weak state. Today a group can be hidden from the listing but still readable by anyone who knows the id. The id cannot be changed, so if it leaks, the group is public forever. It looks like access control, but it is not.

If it wasn't private then it was already public, it just doesn't show in group listings.

3. closed mixes two things: "no open join" and "join only with an invite code". There is no way to say "anyone can ask to join, and an admin approves or rejects by hand". This manual approval is a very common case.

I agree the closed state could have different submodes. But actually manual approval isn't really a thing either on Telegram or Discord as far as I know. I can't think of any platform that supports it (but I do want NIP-29 to support, I've seen people who wanted such a mode on Telegram more than once and it wasn't available).

4. No emergency state. If something bad is posted, an admin or the relay owner cannot quickly freeze and hide a group while keeping the content for later inspection. The only option is to delete.

This is another good-to-have feature, but I have never seen it used anywhere else, so it's weird that you act as if this was a total oversight.

5. No fine control over writing. Today writing is only on or off (restricted or not). You cannot make a group where only admins open threads and members can only reply (a curated group). And you cannot make a group where members can only read and not post at all. A read-only group is useful as an announcement channel (many old forums work like this, and also some Telegram/WhatsApp channels), and also to temporarily block a group when there are problems, without hiding it.

Actually we have supported_kinds, which addresses some of this, but I agree would be nice to support all the things you mention.

6. The write default is open. Without restricted, anyone, even non-members, can write. For a "group" this is surprising.

The default is determined by the client that is creating the group, not by the spec. If the client always creates groups with the restricted tag that is the default there.

Side note: I think that for a group the membership should be mandatory, even if with an "auto accept" mode.

Why? I like the idea of unrestricted groups, as they are already naturally restricted by the overall relay policies that already apply to other events (on Pyramid for example only members of the Pyramid ladder can post on unrestricted groups). A relay could also just opt for interpreting unrestricted as only-members too, this is in the scope of general relay policy.

7. Adding flags later is not safe. The flags are additive: present means "more restricted", absent means "open". So if a new flag is added in the future, an old client that does not know it will ignore it and treat the group as more open than it really is. For access control, failing open is the wrong direction.

I understand your sentiment, and maybe it would have been better to have the non-tag mode be "closed" and not "open", but I don't think this justifies changing the spec now. The spec is arbitrary, client developers should not interpret the meaning of tags by just reading them, they should interpret them by reading the spec.

Shortly put, you're just talking about a bug in the client, and bugs have to be fixed.

[dtonon]

But actually manual approval isn't really a thing either on Telegram or Discord as far as I know.

Probably it is one of the most important feature to build curated groups (manual can include every async automation, as well). It's an error to think about groups only taking in consideration Telegram and Discords, they can have broader applications and different requirements. These specs seem failing exactly on this, they have been set up whit a limited chat-only view.

Why? I like the idea of unrestricted groups, as they are already naturally restricted by the overall relay policies that already apply to other events

Unrestricted groups are fine, but for me this means that users can freely join the group, and then post. Here we are talking about posting in group without being a member, that's odd. The problem arises since we are mixing the admission axis with the writing axis.

client developers should not interpret the meaning of tags by just reading them, they should interpret them by reading the spec.
Shortly put, you're just talking about a bug in the client, and bugs have to be fixed.

This specs open a lot of misinterpretations and so, possibile bugs. We should think also about developers UX (sorry, I have to say that), don't just blame them when they fail to interpret specs, because this impacts directly on the existence, the quality and the interoperability of Nostr clients. We can add random flags and write complex rules about how they sum or when they collide and must be avoided, but I don't think is a wise and practical solution in the long term, we are accumulating technical debt.

I'm really trying to add the features I have in mind following the flags structure, but it's hard; it feels unnatural and chaotic.
Anyway, these flags have been introduced just six months ago (#fb18d4c), I understand the importance of taking a conservative approach and ensuring backward compatibility, but are we really sure we cannot rewrite them using a more general and clean paradigm?

[fiatjaf]

But actually manual approval isn't really a thing either on Telegram or Discord as far as I know.

Probably it is one of the most important feature to build curated groups (manual can include every async automation, as well). It's an error to think about groups only taking in consideration Telegram and Discords, they can have broader applications and different requirements. These specs seem failing exactly on this, they have been set up whit a limited chat-only view.

Do you have an example of any platform that has this feature? I'm just curious. Discord has forums, by the way, it's not only chat. The biggest "forum" software out there today is Discourse, and it also doesn't have that. Old school forums didn't have anything either as far as I know.

Unrestricted groups are fine, but for me this means that users can freely join the group, and then post. Here we are talking about posting in group without being a member, that's odd. The problem arises since we are mixing the admission axis with the writing axis.

Users can freely join when the group is ["open"], that's a different property. There is no mixing anywhere. You may think the restricted/unrestricted stuff is useless and you can just not use it, but it is very clear to me what it does.

This specs open a lot of misinterpretations and so, possibile bugs. We should think also about developers UX (sorry, I have to say that), don't just blame them when they fail to interpret specs, because this impacts directly on the existence, the quality and the interoperability of Nostr clients. We can add random flags and write complex rules about how they sum or when they collide and must be avoided, but I don't think is a wise and practical solution in the long term, we are accumulating technical debt.

I agree, I welcome some clarifications to the spec. I specially liked the tables you created for explaining the expected effects each property has on the group.

I'm really trying to add the features I have in mind following the flags structure, but it's hard; it feels unnatural and chaotic.

I think we might be on two separate universes then, because to me everything is very clear and straightforward. If you make a list of the features you need I can try to write my own pull request.

Anyway, these flags have been introduced just six months ago (#fb18d4c), I understand the importance of taking a conservative approach and ensuring backward compatibility, but are we really sure we cannot rewrite them using a more general and clean paradigm?

I really don't want to personally rewrite my 4 implementations of this again for what to me is just cosmetic changes in a JSON array structure; and I also don't want to keep maintaining both the old way and the new way together. I also wouldn't want to ask Flotilla, Nostrord, Wisp Android and Wisp iOS and maybe there are other apps I don't remember out there. Also Chachi and Grimoire had this stuff working too, even though they're kinda abandoned now they can still be revived and used later, but this change would kill them.

[dtonon]

Do you have an example of any platform that has this feature? I'm just curious. Discord has forums, by the way, it's not only chat. The biggest "forum" software out there today is Discourse, and it also doesn't have that. Old school forums didn't have anything either as far as I know.

Telegram has a "Request Admin Approval" setting on invite links
Signal invite links have a "Require admin approval" toggle
Discord offers a "Apply to Join" flow, also with custom questions*
WhatsApp group admins can turn on "Approve new members"
Facebook Groups admins approve member requests, optionally with screening questions*
Reddit restricted/private subreddits require mod approval to join or post
Discourse has "must approve users" flag for closed communities

Old school forums have admission using the email check, that is a least a little barrier agains spammers and bot.
Anyway, the pending queue can also me managed by some automation, it's not necessarily a manual action.

I don't know if others support a similar feature, but I think we should innovate with new possibilities, not just copy others, if we want to attract new users.

• I was thinking to add also this feature; or to let the users interact immediately but leave the first events under pending approval, and unlock them definitively after a while.

Users can freely join when the group is ["open"], that's a different property.

I don't see open nowhere in the specs.

There is no mixing anywhere. You may think the restricted/unrestricted stuff is useless and you can just not use it, but it is very clear to me what it does.

NIP-29 states: "restricted indicates that only members can write messages to the group. Omitting this tag indicates that anyone can send messages to the group".
From this, I gather that anyone can post without being a member of the group. But, for example, Pyramid doesn't work like that and blocks events for not members.

If you make a list of the features you need I can try to write my own pull request.

Thank you for the availability. I will try a little more, If necessary I’d be happy to ask for your help.
In short, I want to control access to groups (basically an admission queue) and how users interact (can write, reply, post, etc).

I also wouldn't want to ask Flotilla, Nostrord, Wisp Android and Wisp iOS
... this change would kill them.

I don't think this is actually so tragic. If specs change the only part that needs immediate update are relays (how many support groups in addition to Khatru/Pyramid?), which should threat the current flags as alias of the new structure when receiving the group requests and generating the kind:39000 events (that should contain both structure in the transaction phase). That's all, all clients continue to work just fine. The old spec can be sunset, and clients that are actually developed and used will perform the update at a time that is convenient for them.

Anyway it seems that the majority of current groups are open, so they don't experiment any limitation for this reason. At the same time, they would be affected only minimally by this hypothetical migration.

[staab]

I agree with fiatjaf's comment on most of this. The current semantics are already pretty good, and renaming things doesn't help. Emergency mode is pretty advanced; I don't see a real need for it, but I don't mind if you want to add it.

you cannot make a group where members can only read and not post at all.

I agree we need this, but I don't think adding ad-hoc tags forever really solves it. Instead, we should add roles (see below).

The write default is open. Without restricted, anyone, even non-members, can write. For a "group" this is surprising.

The default is determined by the client that is creating the group, not by the spec. If the client always creates groups with the restricted tag that is the default there.

The default is also determined by the relay implementation. If you publish a group event to a non-nip-29 relay it will still work, but tags won't be enforced. The defaults should be what a naive relay would do; additional tags are add-on functionality. I think progressive enhancement is the way to make something like this work, and while it might be nicer to start with max restrictions and "grant" additional access, that's not how nostr relays work naively.

if a new flag is added in the future, an old client that does not know it will ignore it and treat the group as more open than it really is. For access control, failing open is the wrong direction.

This is ok, because clients need to handle errors that it doesn't anticipate. The relay is the one doing real access control, if a client doesn't know about a tag it will result in slightly degraded UX, but not a leak of private data.

Unrestricted groups are fine, but for me this means that users can freely join the group, and then post.

This is because you're thinking of groups as standalone things, but Flotilla/nostrord treat nip 29 groups as sub-groups of a relay. You shouldn't have to join every room in a "group". Also, I think there are use-cases where drive-by posting without being a member makes sense, e.g. reddit. Requiring membership to post is very much a group policy thing, not something for the spec to enforce.

I don't see open nowhere in the specs.

Open is the opposite of closed, it used to be there but now it's just the default.

---

The main things I think are needed here are:

• Some indication of how membership requests are processed. closed is bad, maybe we could switch to ["access", "open|invite|never|manual"] for different modes (multiple of which may be specified by including multiple access tags).
• Role-based access control. This is a real feature request from a user of mine, and is something Discord does well. It solves the "admins can write but members can only read" use case, as well as numerous others which vary from community to community. See here for a proposal that covers both cosmetic and functional roles.

[fiatjaf]

Telegram has a "Request Admin Approval" setting on invite links
Signal invite links have a "Require admin approval" toggle
Discord offers a "Apply to Join" flow, also with custom questions*
WhatsApp group admins can turn on "Approve new members"
Facebook Groups admins approve member requests, optionally with screening questions*
Reddit restricted/private subreddits require mod approval to join or post
Discourse has "must approve users" flag for closed communities

ahahah whoa.

[Semisol]

Some indication of how membership requests are processed. closed is bad, maybe we could switch to ["access", "open|invite|never|manual"] for different modes (multiple of which may be specified by including multiple access tags).

I feel like the inevitable end is an ACL-ish system:

• add members
• create invite codes
• bypass member approval (for anyone this person invites)
• read content
• write content
• moderate content (delete / approve members / etc.)
• bypass moderation

And outside of that, hidden and restricted need to stay + some sort of "approve new members".

This allows you to do everything, even lockdown mode, without much relay complexity.

Matrix has a pretty dumb way to do this, which is that permissions are assigned based on a power level. If for example write is set to 25, your power level would need to be >=25.

[staab]

I agree, but I think the power level method is not sufficient. We need venn diagrams, not just euler.

[Semisol]

I think that we should just rework roles, then. Currently they are up to each relay. A role definition event that has a content with {"role name": {... info object ...}} where info object could have permissions, and a display name for the role?

[staab]

Has anyone here taken the time to read my two NIP drafts that defines RBAC on both the group and relay level?

[dtonon]

The current semantics are already pretty good, and renaming things doesn't help.

As I already said current semantics for me are quite confusing, they try to reframe in a linear way a problem that universally has been already solved with ACL on read/write/admission axes, and in the mean time they didn't offer the basic flexibility that many groups need.
Also from the PR that reviewed flags is clear that the interpretation was not easy.
Even if it were just a matter of renaming it, though it’s much more than that, it would greatly improve the UX for developers.

Just one example about the semantic, from NIP-29:

closed indicates that join requests are ignored. Omitting this tag indicates that users can expect join requests to be honored.

And later, under "Adding yourself to a group" (!!?):

If a group is closed, join requests are not honored unless they include an invite code.

So what closed really means? Omitting it means automatic join? Or a group where admission have been blocked? So Why codes should bypass it? This is not only a poorly worded sentence, it highlights a broader issue in how we think about the life cycle of groups and admission flows. The code exception has been added, in the wrong place, to solve an immediate need, without taking in consideration the full picture.

you're thinking of groups as standalone things, but Flotilla/nostrord treat nip 29 groups as sub-groups of a relay. You shouldn't have to join every room in a "group".

Actually, I have the exact same vision, Squalk works like that.

I think there are use-cases where drive-by posting without being a member makes sense, e.g. reddit. Requiring membership to post is very much a group policy thing, not something for the spec to enforce.

We’re overcomplicating simple things. Why on earth should someone who isn’t formally a member of a group be able to post in it? Who benefits? Group owners like to have the full picture of their members; in fact NIP-29 states:

It's expected that the group state (of who is an allowed member or not, who is an admin and with which permission or not, what are the group name and picture etc) can be fully reconstructed from the canonical sequence of these events

All this without talking about the spam implication. We need ways to control and adjust joining/posting, not the inverse. If someone want to send a isolate message let they join, post and leave; this is the exception, not the norm.
Even if this is a relay policy, it makes sense to me to provide guidelines on this matter; otherwise clients will get confused trying to interact without a clear structure.

Has anyone here taken the time to read my two NIP drafts that defines RBAC on both the group and relay level?

I took a quick look at it; I haven't been feeling too well for the past couple of days, but I'll take care of it as soon as possible.

[staab]

Let's not talk about closed. It's clearly bad and we agree on that.

Why on earth should someone who isn’t formally a member of a group be able to post in it?

I already answered this question and gave a hypothetical example of why you might want to drive by post. Here are some more: do I have to be a "member" of a physical bulletin board to post an ad on it, a "member" of a podcast to be a guest, a "member" of a blog to leave a comment? Here's another reason: making users join the group requires them to sign an additional event that may not be necessary if group membership is not gated.

Spam can be managed by membership, but it can also be managed by web of trust, pow, captchas, whatever. Specs should not be prescriptive, they should describe affordances that are unambiguous, and which people can use for different use cases. This PR is well-designed for certain use cases, but couples various axes when it "makes sense", making the spec more complex and less flexible.

I agree with the basic thing you're trying to do here, but I would go the opposite direction in practice: tags on a group are attributes of the group. Which means they are how the group behaves when dealing with a non-member. Membership escalates permissions, and those permissions should be attributes of the membership. Hence my RBAC proposal. This is infinitely more flexible, because you can define any permutation of rules depending on the member (or their role). Here's an example of something weird that might be useful: a write-only group, which non-members can send form submissions to, but which only members can read.

Anyway, in concrete terms, here's how I think we should do this. A group should have the following axes defined, which define how the group works for non-members:

• Can non-members see the group?
• Can non-members read stuff in the group?
• Can non-members post to the group?
• What is the join policy (join requests are ignored, join requests are auto-approved, join requests are manually approved, join requests are auto-approved if they have a valid invite code)?

Admins can see, read, post to, and auto-join the group (or can't leave the group).

Roles can then define additional privileges:

• Members with any role can see the group
• Members with any role can read stuff in the group
• Members with a role with the write permission can post to the group
• Members with a role with the put-user permission can add users to the group (etc for all the other admin things)

We could even use the same terminology for both groups and roles. A group would have a ["can", "put-user"] to allow any non-member to add members to the group. Silly, but symmetric with a role with ["can", "put-user"].

I think the crux of this conversation is that the goal (imo) of designing a spec is not to restrict functionality to stuff we think we need, because we're going to be wrong and end up with a brittle spec and less experimentation/building, which is what nostr is all about. Rather, the goal should be to design clear, decoupled affordances which are simple to interpret and cover all affordances. If they enable things that "no one would want", that's ok. Either someone will actually want that, or relay implementations can make it impossible via policy.

I just drafted a new merge request with a cleaner rbac spec: #2376. I don't really think it's worth breaking everything, but it's getting really hard to fix this spec without breaking compatibility.

[dtonon]

@staab thank you for your detailed response!

do I have to be a "member" of a physical bulletin board to post an ad on it, a "member" of a podcast to be a guest, a "member" of a blog to leave a comment?

These are interesting examples; I don't know how they concretely fit in a group, but the core concept is that for spot participation could not be so necessary to join a group. Noted.

making users join the group requires them to sign an additional event that may not be necessary if group membership is not gated

This is actually the only cons I see. I'm not strictly opposed to posting without joining, actually we also have reading without joining and no one objects to this, but I have a feeling that establishing a clear workflow could make future management much easier for the price of a signature. And we provide group admins with an additional aggregate metric on participation (although it's certainly not perfect).

tags on a group are attributes of the group. Which means they are how the group behaves when dealing with a non-member. Membership escalates permissions, and those permissions should be attributes of the membership.

I like this approach, it is definitely clearer and more organized than the current one. And I like the idea of an evolute role system. But I would prefer to offer it incrementally, without an imposition, to ensure that the system can work with just the kind:39000 in most cases, and in the most common ones, in terms of access and participation. User management is another story.

This could be a backward compatible version using linear flags:

Admission

free => Join requests are automatically accepted
review => Join requests are reviewed, manually or by an automatic workflow
invite => Join using a invite code is enabled
pay => Join require paying a fee (how?)
closed => Join requests are automatically rejected

Visibility

public => Everything is visibile to anyone (current implicit default)
preview => Metadata and root content are visibile, replies (as defined in my PR) are accessibile only to members
private => Metadata are visibile, content is accessible only to members
hidden => Group is hidden, metadata are visible only to members

Participation

writable => Posting is enabled to everyone (current implicit default)
restricted => Posting is restricted to members
replies => Posting is restricted to replies (as defined in my PR)
disabled => Posting is disabled

Special states
locked => Override participation flags and rejects joins and posting (admin excluded)
quarantined => Override visibility flags and hide the group and its metadata (admin excluded)

I kept everything linear and one-word to uniformity, even if it's not the most elegant solution.
Flags within each section override one another in order from bottom to top.
I would like switch to restricted as default, but it breaks backwards compatibility. But even more than that, I would prefer it if the defaults didn't exist at all, and it were necessary to specify a flag.

A role system should overwrite Visibility and Participation areas.

the goal (imo) of designing a spec is not to restrict functionality to stuff we think we need, because we're going to be wrong and end up with a brittle spec and less experimentation/building,

I totally agree.

I just drafted a new merge request with a cleaner rbac spec: #2376. I don't really think it's worth breaking everything, but it's getting really hard to fix this spec without breaking compatibility.

Well, before going on we should decide if we can or not breaking things :)

[dtonon]

Pushed an update that reflects my last comment.

[dtonon]

@fiatjaf @staab do you have any comment about the last update?

[fiatjaf]

Sorry, I've spent all my energy, will need some more time before I can read the latest changes.

[staab]

I also have been busy, partly thinking about this from a different perspective. We'll likely need roles for groups eventually, but I'm going to stick to the relay level for now. Your latest comment makes a lot of sense, but the diff is clearly vibed and way too long (my agent also likes to talk about what things looked like before the change). The spec is already a monster, edit it down and I'll take another read.

[fiatjaf]

I hadn't looked at the diff, it's bad. Please don't do this.

[dtonon]

but the diff is clearly vibed and way too long (my agent also likes to talk about what things looked like before the change)

Yes, I shared my last comment with a LLM asking to format everything into tables, but then I deeply reviewed and edited it to remove some unsolicited parts and the references to the previous version, since the backward compatibility is practically total; I also added the groups/events examples. I will try to review it again and see if I can compress it, but I don't know if it's really possibile without removing useful informations.
I certainly didn't want to give you any slop, sorry!

[dtonon]

I removed the references to the previous version and to backward compatibility.
The examples of events take up space, but I think they're useful.

[staab]

Much better. I think this works fine for me.