Skip to content

Update nip 17 to indicate that wraps should be signed with the conversation key#2396

Closed
staab wants to merge 3 commits into
nostr-protocol:masterfrom
coracle-social:nip17-sign-with-conversation-key
Closed

Update nip 17 to indicate that wraps should be signed with the conversation key#2396
staab wants to merge 3 commits into
nostr-protocol:masterfrom
coracle-social:nip17-sign-with-conversation-key

Conversation

@staab

@staab staab commented Jun 25, 2026

Copy link
Copy Markdown
Member

For discussion. This trades off some metadata hiding (watchers can now identify a single conversation over time) for spam prevention and the ability to target a conversation by counterparty.

In concrete terms, this allows the user to:

  • Only download messages from people they follow or have otherwise whitelisted instead of unknown contact.
  • Download all messages, and lazily decrypt them as they become relevant to user actions.
  • Decrypt a single message per signer in order to identify counterparties without unwrapping all events.
  • Download and encrypt messages from a single counterparty the user actually wants to talk to.

This is backwards compatible in the sense that old clients that don't assign any expectation to the pubkey will continue to work. However, in order to take advantage of the benefits of this change, clients will need to switch to using the conversation key to sign wraps, which will take some time.

@alexgleason

Copy link
Copy Markdown
Member

Holy shit this would completely fix NIP-17 🤯

@vitorpamplona

Copy link
Copy Markdown
Collaborator

This is what NIP-04 always wanted to be. @paulmillr check this out.

The only downside of this is that we can't offer users the option to stay in the ephemeral key mode or allow permanent conversation fingerprints that anyone can see. Because as long as one NIP-17 client still files regular ephemeral keys, clients implementing this new mode only will miss messages.

@vitorpamplona

vitorpamplona commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Important to note that, with this, a leaked nsec can decrypt not only incoming wraps but also all outgoing wraps.

Similarly, the leaked nsec can also delete and rebuild wraps that, until now, were understood as the "property" of the receiver and safe from counterparty nsec leaks, and only the receiver can request deletion.

@staab

staab commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

Great points. I think it's worth it, but it definitely weakens security some.

@vitorpamplona

vitorpamplona commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

What if we created new kinds for these in a new NIP? Would that allow new apps to declare compliance with this new NIP and ignore regular NIP-17 messages? Otherwise, I am not sure we will ever fully transition over... Some folks are still using NIP-04 even after 4 years.. so... apps using this new system might be forced to still load everything from the old system for the next ... 4 years?

@arthurfranca

Copy link
Copy Markdown
Contributor

A conversation key isn't a valid nostr key for signing, so this would need an extra derivation step, which would require a new signer method.

fwiw I'm already implementing a chat spec that does exactly that while fixing other nip-17 drawbacks. within 1 month probably a proof of concept will be up (I'm working on many things simultaneously 😵 ).

@staab

staab commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

What if we created new kinds for these in a new NIP?

I don't think so in this case, since keeping the kinds means clients can immediately start using the new scheme without breaking older clients. This makes the migration much more likely to happen.

A conversation key isn't a valid nostr key for signing

Why? It's a random 256 byte value, seems like it would work fine.

@fiatjaf

fiatjaf commented Jun 25, 2026

Copy link
Copy Markdown
Member

fwiw I'm already implementing a chat spec that does exactly that while fixing other nip-17 drawbacks.

You'll never have a better chance than now to convince everybody about your new changes. If we can fit them here into a subtle NIP-17 change everybody wins.

@arthurfranca

Copy link
Copy Markdown
Contributor

Why? It's a random 256 byte value, seems like it would work fine.

It is related to it not being a scalar.

@fiatjaf I'm just saying that changing NIP-17 isn't the right move because imo NIP-17 would need many patches to be an optimal DM spec, though this PR points to the right direction, atleast for 1 on 1 chat.

Things that I've been working on that aren't addressed on NIP-17:

  1. Rotating the conversation-key-derived (this PR-ish) private key (I call this key a private channel; though for group chat - a future addition - it wouldn't be based on the converstation key)
  2. A message envelope that batch sends messages to many peers at once, to avoid being rate-limited by relays
  3. Chunking for allowing to send arbitrarily large events as messages (e.g. send a one-million member list for example)
  4. Plausible deniability by not signing messages
  5. Using an extra private key (I call it content key) besides the identity/main one so that leaking your identity alone isn't enough to read the messages (need both keys to encrypt/decrypt)
  6. Content key must be synced to other "devices"/instances
  7. Migrate to nip44v3 from @Semisol to help with better and more fine-grained permission prompts
  8. More stuff though the list is already big.

Some of the above features require new signer methods. Even this PR as-is would require signers exposing something to be able to sign with the conversation key.

@fiatjaf

fiatjaf commented Jun 26, 2026

Copy link
Copy Markdown
Member

Why? It's a random 256 byte value, seems like it would work fine.

It is related to it not being a scalar.

Anything is a scalar (or can be interpreted as such).

Specifically anything between 1 and FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE BAAEDCE6 AF48A03B BFD25E8C D0364141 is a valid private key for secp256k1 (so yes, there is a range of numbers that aren't valid keys, specifically 0.000000000000000000000000000000000000373445534504013400567088856307011153030212200776955239720900516111703432179328684584042213922584763086121029118658043444156646728515625% of the numbers aren't valid private keys).

@fiatjaf

fiatjaf commented Jun 26, 2026

Copy link
Copy Markdown
Member
  1. Rotating the conversation-key-derived (this PR-ish) private key (I call this key a private channel; though for group chat - a future addition - it wouldn't be based on the converstation key)

I believe NIP-17 is compatible with this if we do #2361, but there are arguments against it. Basically everything that rotates keys comes with many problems.

Still I think it remains an optional and is basically compatible with NIP-17 as it exists today.

  1. A message envelope that batch sends messages to many peers at once, to avoid being rate-limited by relays

Sounds like a minor issue to me, I think NIP-17 shouldn't be used by more than 3 people anyway, and even that is a stretch.

  1. Chunking for allowing to send arbitrarily large events as messages (e.g. send a one-million member list for example)

No, that is bad, stop it!

  1. Plausible deniability by not signing messages

NIP-17 rumors aren't signed already (but the seal is signed, we can't escape that).

  1. Using an extra private key (I call it content key) besides the identity/main one so that leaking your identity alone isn't enough to read the messages (need both keys to encrypt/decrypt)

Same: #2361

  1. Content key must be synced to other "devices"/instances

Same: #2361

  1. Migrate to nip44v3 from @Semisol to help with better and more fine-grained permission prompts

I don't know what this is. Is it that thing about asking the signer to decrypt the event and not the ciphertext directly? Looks good, but it's orthogonal to the messaging protocol, no?

  1. More stuff though the list is already big.

It looks like you are very set on going your own way, but I recommend against that.

If you really want to do your own messaging protocol that may end up being good, but please don't do it "on Nostr". Do something different, more specialized and more efficient, reuse Nostr identities, use Nostr for signaling, I don't know, but don't sent the actual messages as Nostr events to relays, that is ugly.

Some of the above features require new signer methods. Even this PR as-is would require signers exposing something to be able to sign with the conversation key.

Good point, @staab.

But I think it is fine, we should make an allowance for that since it will fix NIP-17 a lot anyway.

@staab

staab commented Jun 26, 2026

Copy link
Copy Markdown
Member Author

Just pushed an update which derives a separate key for signing and normalizes it to a proper secp256k1 key. The clanker gave me advice and helped me write the example, so let me know if I did something wrong.

@arthurfranca

Copy link
Copy Markdown
Contributor

I'm going to appeal to an authority: #945 (comment)
the scalar thing is lost in some other comment somewhere

@arthurfranca

Copy link
Copy Markdown
Contributor

If you really want to do your own messaging protocol that may end up being good, but please don't do it "on Nostr".

@fiatjaf Nostr is perfect for this, if we use relays this way. I mean, when dealing with relays, I just have to be aware of their limitations. Publishing rate limits, expecting just temporary free storage and all the other necessary limits to be able to offer relays for free to unknown users. Building a relay that runs on less than 10$ made me want to build apps that never rely on big relays/blossom servers keeping data around foreverish for free (Damus/Primal/Ditto I'm looking at you).

That messaging protocol includes just the minimum that was needed to get over those limitations. It is just for private comms over Nostr relays, it doesn't replace what is already in place for public events.

Sorry for being a bit off-topic.

@fiatjaf

fiatjaf commented Jun 26, 2026

Copy link
Copy Markdown
Member

You can define whatever custom server interface you want and make it very cheap to run, even cheaper than Nostr relays. But it looks like you're just abusing free infrastructure run by other people, not a great long-term strategy.

@arthurfranca

arthurfranca commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

But it looks like you're just abusing free infrastructure run by other people

Aren't gift wraps doing the same thing? Users of NIP-17 in general aren't paying for DM relays. Edit: I got what you mean, you dislike chunking etc, but I meant the protocol tries to respect the limits not get over, my English is not the best. The current implementation even adds an expiration tag of 2 days by default on events. Edit 2: It seems natural to use the peer's read relays to send messages meant to them.

@staab staab force-pushed the nip17-sign-with-conversation-key branch from b324a99 to 065ded7 Compare June 26, 2026 22:37
@staab

staab commented Jun 26, 2026

Copy link
Copy Markdown
Member Author

Just added a mechanism that adds a counter to the salt to avoid out of range private keys. I think this addresses Mike's comment about invalid keys as well as Paul's comment about key length.

@paulmillr

Copy link
Copy Markdown
Contributor

@vitorpamplona I never implemented nip 17 myself, but that seems like a nice simplification

@reyakov

reyakov commented Jun 27, 2026

Copy link
Copy Markdown

I'm working on this NIP for Coop (https://git.reya.su/reya/coop-mobile)

@Letdown2491

Copy link
Copy Markdown

So I am definitely the smallest brain in the room but I'll chime in. The thing that bugs me about this is that it solves filtering by making the addressing handle permanent and public, and that one decision is what drags in all three downsides people are flagging (linkable fingerprint, per-conversation censorship, rewritable history).

Correct me if I'm wrong, but filtering doesn't need a permanent public handle, just one both parties can derive and nobody else can. Why not keep the random one-time signing key on the kind:1059 wrap so it stays unforgeable and the recipient still owns it, and instead of a static p-tag, address it with a rotating tag derived from the ECDH secret both sides already compute. This way, both parties req the current epoch's tag, observers can't link across epochs or tie it to an identity, and since the signing key is still random the history-rewrite problem just doesn't exist. A stranger can't derive your tag either, so established conversations are spam-proof for free, and there's no new signer method since it's addressing, not signing.

From my read of the conversation, his is similar to what Nymchat. You rotate the recipient key and advertise the next one inside each encrypted message. The difference is that keeps the chain in-band, so the address only exists if you received the previous message. Deriving the tag from the ECDH secret plus an epoch is stateless instead, so either side can compute the current address on its own without having seen prior messages. It doesn't try to add forward secrecy the way Nymchat optionally can, but it's no worse than NIP-17 there today.

@fiatjaf

fiatjaf commented Jun 27, 2026

Copy link
Copy Markdown
Member

@Letdown2491 good idea.

I tried to write it here: #2397

@alexgleason

Copy link
Copy Markdown
Member

I'm still a bigger fan of this approach versus the one in #2397 due to the simplicity of it.

@alexgleason

alexgleason commented Jun 28, 2026

Copy link
Copy Markdown
Member

A weird side-effect of this (and #2397 - both proposals) is that the counterparty can delete your events. Is that good? Maybe it's okay. But I've been thinking a lot today about how to prevent events from being deleted from relays when you and another person share a key. Either party can sign and publish a kind 5 tagging any of the events in your conversation.

EDIT: Reading NIP-59 more carefully, I realize that's already the case:

Since signing keys are random, relays SHOULD delete kind:1059 events whose p-tag matches the signer of NIP-09 deletions or NIP-62 vanish requests.

@fiatjaf

fiatjaf commented Jun 28, 2026

Copy link
Copy Markdown
Member

I was going to say that relays would have to pick a policy to implement whether conversations can be deleted or not and by whom.

For example: if A is registered with the relay and uses it as its DM relay but B is not and is just sending messages to it, then the relay could allow just A to delete.

The problem is that if the relay sees an event from X (X being the ECDH of A and B) then it can't know if that event "belongs" to A unless A somehow proves it is part of X. There is probably a way to do that but I don't know how.

Now I realize that the paragraphs above are useless, because the relay can just check the "p" tag of the event to determine who is the receiver, and just let that person delete it but not anyone else (not X, the event author, either).

@alexgleason

Copy link
Copy Markdown
Member

just let that person delete it but not anyone else (not X, the event author, either).

We should do that. That would fix the problem @vitorpamplona mentioned above about either party wiping the history, among other things, and it makes sense semantically.

@alexgleason

Copy link
Copy Markdown
Member

@fiatjaf

fiatjaf commented Jun 29, 2026

Copy link
Copy Markdown
Member

I don't think it is a problem, I think it can be a feature. If you don't want that you have to take steps to prevent it by using a relay that doesn't allow it.

@staab

staab commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

I like @Letdown2491's idea. Also, I don't think one-off keys prevents rewriting history; clients could just retain the one-off keys and execute the attack.

Here's another idea: instead of time-based epochs we could just statically define some arbitrary number of keys that might be used for a conversation. More keys would be harder to fingerprint. Example:

number_of_distinct_keys_per_conversation = 100
salt = 'nip59-signing-key-' + randomInt(0, number_of_distinct_keys_per_conversation)

Then you could derive all 100 keys and use those as the authors filter. Of course, relays could instantly de-anonymize.

@fiatjaf

fiatjaf commented Jun 30, 2026

Copy link
Copy Markdown
Member

This is all so ugly I want to vomit.

But fine by me either way.

I like the original proposal in this PR more now, because of the reasons I stated in the other PR:

With this we can make a DM relay that whitelists static conversation keys and accepts messages from those without requiring AUTH.

Requiring AUTH deanonymizes the sender (to the relay) immediately, but if the receiver contacts the relay somehow and preauthorizes some people then the relay can just accept such messages without knowing who is sending them.

Now if the signing keypair changes every few days or if it can be 100 different ones that just makes everything harder.

@vitorpamplona

Copy link
Copy Markdown
Collaborator

I thought a lot about this PR (and its variants) in the last few days.. I think the current NIP-17 is better, simpler, and nicer to work with than any of these variations. To me, nothing here delivers significant gains over a simple interactively-paginated feed. It's all bikeshedding.

@fiatjaf

fiatjaf commented Jun 30, 2026

Copy link
Copy Markdown
Member

interactively-paginated feed

You can't paginate one conversation without paginating all the other conversations at the same time.

@vitorpamplona

Copy link
Copy Markdown
Collaborator

You can't paginate one conversation without paginating all the other conversations at the same time.

Yep. But good interfaces don't need to worry about it.

@alexgleason

alexgleason commented Jul 6, 2026

Copy link
Copy Markdown
Member

If person A and B both reply to each other, they become linked by the p-tag. The p-tag value should therefore be a random pubkey.

EDIT: Never mind, because you can't prove that the conversation key is either participant. It could be a random third-party.

@staab

staab commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Closing to let someone else open a PR they want to sponsor. This isn't high on my list right now and I don't think this particular incarnation is the best.

@staab staab closed this Jul 7, 2026
@alexgleason

Copy link
Copy Markdown
Member

I still care a lot. I'm going to try to implement it just for an invite system at first. If that works I'll try to push it through signers but I need @vitorpamplona to care enough to accept my Amber pull request if I do.

@vitorpamplona

Copy link
Copy Markdown
Collaborator

You need @greenart7c3 to care... :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants