Taking "Taking Things Seriously" Seriously (aka Julia Enhancement Protocol Protocol)

[oscardssmith]

Many programming languages have very formal processes for implementing language enhancements (such as the following)

1. https://github.com/JuliaLang/Juleps
2. https://openjdk.org/jeps/1
3. https://peps.python.org/pep-0001/
4. https://isocpp.org/std/submit-a-proposal

Coming from a Julia perspective of "just make a PR", these all are incredibly formal (especially for a relatively small language with a tightly knit culture), but especially since Julia as a language has matured fairly significantly over the past few years, we do likely want to increase the amount of process and effort for adding new features to Base (both to improve the quality of features that get added and to discourage adding to Base by default).

It's worth noting that Julia already has a process for this https://github.com/JuliaLang/Juleps, but it is very optional (at time of writing there only appear to be 9 entries), likely due to the added inconvenience of requiring a pull request to a separate repository. I think the idea and use of the Julep process for the changes that have used it has worked well, but I think the organization has meant that we do not use them as frequently as we should.

As such, I think we should likely adopt a process less formal than that of other languages, but more formal than what we have now. As a rough draft of one (stealing lots here from python's PEP1 since it seems pretty good overall).

Each Pull request introducing new functionality should include (either directly or by link to some external document such as a Julep, github issue, or HackMD document):

1. Abstract – a brief description of the technical issue being addressed.
2. Motivation – It should clearly explain why the existing language specification is inadequate to address the problem that the JEP solves. This can include collecting documented support for the JEP from important projects in the Julia ecosystem.
3. Rationale – The rationale fleshes out the specification by describing why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion.
4. Specification – The technical specification should describe the syntax and semantics of any new language feature.
5. Backwards Compatibility – All JEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The JEP must explain how the author proposes to deal with these incompatibilities.
6. Security Implications – If there are security concerns in relation to the JEP, those concerns should be explicitly written out to make sure reviewers of the JEP are aware of them.
7. Reference Implementation – If possible, the code should be implemented in a (possibly unregistered) package. While there is merit to the approach of reaching consensus on the specification and rationale before writing code, the principle of “rough consensus and running code” is still useful when it comes to resolving many discussions of API details. The final implementation must include test code, documentation, and news entries as appropriate.
8. Rejected Ideas – Throughout the discussion of a JEP, various ideas will be proposed which are not accepted. Those rejected ideas should be recorded along with the reasoning as to why they were rejected. This both helps record the thought process behind the final version of the JEP as well as preventing people from bringing up the same rejected idea again in subsequent discussions. In a way this section can be thought of as a breakout section of the Rationale section that is focused specifically on why certain ideas were not ultimately pursued.
9. Open Issues – While a JEP is in draft, ideas can come up which warrant further discussion. Those ideas should be recorded so people know that they are being thought about but do not have a concrete resolution. This helps make sure all issues required for the JEP to be ready for consideration are complete and reduces people duplicating prior discussion.

This is just a draft, but when finalized, I think it should go into the contributing guidelines and possibly be part of a template for feature PRs

[mkitti]

Do you mean we should take https://github.com/JuliaLang/Juleps seriously?

[StefanKarpinski]

These are some of the reasons I think that the previous Julep process never really took off

1. Overlap with issues/PRs on JuliaLang/julia and lack of clarity about which to do—if we don't offer clear guidance about when a Julep/TTS report is required, then opening an issue/PR is easier and will tend to be favored.
2. The separate repo reduces visibility and means that changes which should get more visibility instead get less visibility.
3. A Julep should be a document that people comment and collaborate on, which evolves over time and lets you see changes that have been made since the last time you looked at it. Neither GitHub Issues nor the documents in the Julep repo are particularly good for this.
   • GitHub issues are not great because they are designed for back-and-forth discussion rather than creating and collaborating on a document. Yes, you can edit posts, but it's just not what issues are designed for.
   • The Julep repo was an attempt to address that by using Markdown documents in a git repo, but that is still quite clunky for the use case.

The most successful approach we've had so far is using HackMD documents and sharing links to them for people to comment and collaborate on. Google docs would probably work well too, especially now that it apparently supports Markdown. There are probably other platforms that would work, but we may want to limit what we accept. So how about this approach...

When a Julep is required:

• If a change significantly changes or expands the API of language then it requires a Julep.
• Bug fixes and other changes that do not affect the public API do not require a Julep.

To file a Julep:

1. Create your document with the appropriate sections as above that people can view, comment on, and optionally edit
2. Create an issue on JuliaLang/julia which links to the Julep documents

There will still be a bit of ambiguity about whether comments should go on the document or the GitHub issue, but I'm not sure what to do about that—there are probably different kinds of comments/questions that should go in each place and it's hard to enforce anything rigid there. In general I think that comments/questions about the proposal itself should go on the docuement itself, unless there's no clear place to attach them. Meta comments like "I've updated the document in response to feeback, please review again." should go on the GitHub issue discussion.

[adienes]

The separate repo reduces visibility

at least for my perspective this is a big one. to the extent that drive-by feedback from casuals like me is desired, I would strongly suggest the link to HackMD or google doc go in an issue in the regular julia issue tracker rather than a separate repo

I only have enough muscle-memory-addiction to alt-tab-refresh so many feeds 😅. even the "Discussions" page in this very repo I hardly read.

[JeffBezanson]

I basically agree with Stefan here.

To me, it is just obvious that any proposed change requires an adequate explanation and justification, but there is nothing wrong with writing that down explicitly and adding some detail about how to do it.

The proposed document sections are good, but I would describe them as "suggested things to consider" rather than a required format. Also perhaps this is obvious but I don't think we should ever respond to a perfectly good PR writeup with "this needs to go in a hackmd document".

[oscardssmith]

Also perhaps this is obvious but I don't think we should ever respond to a perfectly good PR writeup with "this needs to go in a hackmd document".

I attempted to make it clear from the language that a PR writeup was one of the acceptable locations in which to write one of these. Is there a better way I could phrase it?

[IanButterworth]

I was just trying to find if there was an issue/PR here discussing introducing a Path type in Base and it took me a frustratingly long time to find #26

Julep findability is definitely an issue. Should that closed PR have a julia issue that's also closed, given the conclusion on that thread?

[ViralBShah]

Should this get moved to the juleps repo too?

[JeffBezanson]

I don't think the separate Juleps repo is working out very well. From what I can see the best list of proposals is https://github.com/JuliaLang/julia/labels/julep. From a quick browse it looks like the Juleps repo has been adopted by Pkg only 😄 . Shouldn't all those be Pkg issues?

The major barrier I see is that there is a continuous spectrum from a simple fix to an overhaul of the language. It is difficult to decide where the threshold is for a Julep / major proposal. Anyway I'm thinking of trying a github project on top of the julep label, where we can run more of a process and mark the status of each, then we just need to drop links to that project in prominent places.

[topolarity]

It needs some amount of community / contributor "push" if it's going to succeed IMO

For example, I'd have been more than happy to follow the Julep process for JuliaLang/julia#59774 but I was never pushed to do so. Instead I published to the main julialang repository, because that seemed to be the best place for visibility.

[JeffBezanson]

That's the thing; I think PRs (or even just issues) are perfectly fine, the only problem is visibility for people who want to see the high-level picture and major changes, AFAICT.

[ViralBShah]

Seems like the kind of thing AI should be able to help with - summarize all the ongoing PRs in the last 3 months.