Display of diagnostic messages

[camilogarciabotero]

Hi,

This is not really an issue, but a couple of projects I found could be helpful and inspire features. I came across this https://github.com/zesterer/chumsky and this https://github.com/zesterer/ariadne Rust projects for parsing compiler errors and got nice diagnostic renderings, I don’t really know if there is an overlap with the JuliaSyntax.jl project, but thought might spark others ideas 😃

PD: I was encouraged to post this from a Slack channel.

[c42f]

Wow what a beautiful error message!

If someone wants to work on beautifying the error message rendering that would be great. I will probably get to it eventually :-)

[FedeClaudi]

Hello everyone, I'm the developer of Term.jl. I just wanted to say, this issue inspired me to add this kind of "annotations" in the image above to Term as a new feature (see: FedeClaudi/Term.jl#185). This is what it looks like:

Annotation(
     highlight("var::Int = 2x + \"a\""), 
    "2x + \"a\"", 
     "This seems like a bad idea"; 
     style="bright_yellow"
) |> print   

If it's something you'd find useful, let me know!

---

Edit: @c42f perhaps there's a way to integrate Term's stacktraces to help beautify messages? I've spent a lot of time on the error message system in Term and, while not perfect, I think it might help. This is what an error message currently looks like

[c42f]

@FedeClaudi this looks beautiful ❤️ (Sorry didn't mean to ignore you, I just missed your message.)

If you'd like to help develop code in JuliaSyntax for rendering error messages that would be amazing! The key thing we need in this package is highlighting and annotating byte ranges within a file. You can see how this currently works here:

JuliaSyntax.jl/src/diagnostics.jl

Line 44 in 45d8431

function show_diagnostic(io::IO, diagnostic::Diagnostic, source::SourceFile)

---

Dependencies are a tricky thing for JuliaSyntax.jl as we need to vendor it into Base and eventually make it part of bootstrap somehow. So I wonder how we'd deal with that. Depending on Term.jl could be a problem if we have to vendor Term and all its dependencies into Base as well.

[FedeClaudi]

Dependencies are a tricky thing for JuliaSyntax.jl as we need to vendor it into Base and eventually make it part of bootstrap somehow. So I wonder how we'd deal with that. Depending on Term.jl could be a problem if we have to vendor Term and all its dependencies into Base as well.

Got it, integration with Term.jl would not be ideal then as it does have a few dependencies. Thought it was worth a shot!

Keep up the good work,
Federico

[c42f]

Thought it was worth a shot!

Maybe we could have hooks so that Term could be installed as an optional dependency? I'd like to make it easy to experiment with pretty printing.

[FedeClaudi]

That could be a nice way to go about it!

[kdheepak]

I'd be interested in contributing to making better diagnostics possible! Do we have a plan for how to tackle this?

Btw, I thought this article on the rust blog was instructive: https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html

What are your thoughts on conducting a informal survey first to see what styles people would prefer and why? I seem to remember that some folks from the rust team did a survey after implementing diagnostics a number of different ways, and I seem to remember reading about it in a blog post or article but I can't seem to find that now.

[c42f]

I'd be interested in contributing to making better diagnostics possible

Great!

Do we have a plan for how to tackle this?

There's two aspects to this:

1. Semantics: reducing double-reporting of the same error and improving how the compiler detects errors. This is the hard part, and there's some discussion of a few ideas I have for this in Diagnostics as pattern matching #93. To summarize, I currently feel that forcing a lot of error recovery code into the main parser code is somewhat unnatural and I'd like to explore alternatives which are more data driven.
2. Presentation: improving how messages are formatted - roughly what we're discussing in this issue.

article on the rust blog

I like this article, I think I've read it before. The key point is that errors should focus on the code you wrote and this is something I've tried to do already in this package: errors are always presented with a chunk of source code highlighted for context. In contrast to the rust blog, JuliaSyntax doesn't mix the error message with the code - I'm trying preserve the visual shape of the code to visually orient the user in the minimum amount of time. So the error message is separate for now.

A problem here is the limitations of ansi color highlighting

• We probably need to opt for more compatible ansi codes by default and do more cross platform testing, see Error message in plain text discards visual location information #61
• ANSI highlighting doesn't survive copy and pasting from a terminal to the web, or whatever. May need ascii art mode for this, or maybe an ability to emit the error in various markup for pasting (eg, md, html?) I don't know what's best. (Ooh. Could we add a marked up version to an ANSI link which the user can follow to preserve formatting?)

Unicode makes the ascii art options difficult to do in full generality. I like this blog post from the Fennel language discussing why ANSI highlighting is good when dealing with unicode text in diverse languages http://technomancy.us/198

By the way, I just discovered that some terminals like kitty support ANSI underline styles. Cool!

println("\e[58:5:9m\e[4:3masdf_asdf")

What are your thoughts on conducting a informal survey first to see what styles people would prefer and why

I think it would be best to spend some time researching what other well-regarded compilers are doing first and build up a set of options. There's been a lot of work put into this by other people and we should learn from that. Also if we can find that existing survey it would be very useful and much easier than doing our own.

[kdheepak]

Thanks for the detailed comment! I'll respond when I get a chance to go through the links but just wanted to reply to one thing:

Also if we can find that existing survey it would be very useful and much easier than doing our own.

I was searching high and low for the article, and it turns out it was a YouTube video from just last month! I must have watched it and promptly forgot that it was a video. Link here: https://www.youtube.com/watch?v=Iv3UuGdB0TM

It is not exactly a survey but the video is very informative, and I recommend watching it. The video is by Jonathan Turner, the same person that wrote the article from the comment.

I'm wondering if it worth pinging @jntrnr to see if they could share their goals document, along with some process ideas etc, and maybe even if they'd be willing to consult or mentor during this process.

[c42f]

Could we add a marked up version to an ANSI link which the user can follow to preserve formatting

Answering this question - yes! Here's a demo of one way to implement an "open in browser" functionality for error messages:

Here's a webpage which we will target:

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8" >
	<meta name="viewport" content="initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0, user-scalable=no, width=device-width">

	<title>Ansi Demo</title>

	<link rel="stylesheet" href="https://muffinman.io/skyblue/css/skyblue.css">

    <style>
    body {
        background: #fdfdfd;
        padding-bottom: 100px;
    }

    .content {
        position: relative;
        margin: 0 auto;
        padding: 20px;
        max-width: 840px;
    }

    #source {
        margin: 20px auto;
        background: black;
    }

    </style>
    <script src="https://cdn.jsdelivr.net/gh/drudru/ansi_up/ansi_up.js"></script>
</head>

<body>

  <div class='content margin-y-40'>

      <h1>Highlighted</h1>

      <pre id='source'>
some code
or other
      </pre>
      <button onclick="copy_html_to_clipboard()">Copy html to clipboard</button>

      <script>
          const query = window.location.search
          console.log(query)
          const url_params = new URLSearchParams(query)
          const ansi_str = url_params.get('ansi')
          console.log(ansi_str)
          const ansi_up = new AnsiUp
          const src_div = document.getElementById("source")
          const html = ansi_up.ansi_to_html(ansi_str)
          console.log(html)
          src_div.innerHTML = html

          function copy_html_to_clipboard() {
              navigator.clipboard.writeText(src_div.outerHTML)
          }
      </script>
</body>
</html>

Here's some Julia code which catches a parser error and creates a link which can be followed from the terminal, allowing for richer formatting and whatever browser goodness. (In this demo, it's not actually richer because I just sent along the ANSI-formatted error string. But we could send html or all sorts of things in the query payload.)

julia> using JuliaSyntax, URIs

julia> try
           JuliaSyntax.parse(JuliaSyntax.SyntaxNode, "function fx()\n  (x + \nend")
       catch exc
           iob = IOBuffer()
           showerror(IOContext(iob, :color=>true), exc)
           str = String(take!(iob))
           println(stdout, "ERROR:")
           println(stdout, str)
           println(stdout, "\e]8;;$(URI(scheme="file", path="/home/c42f/error_test.html", query=Dict("ansi"=>str)))\e\\<open error in browser>\e]8;;\e\\\n")
       end

[kdheepak]

Nice!

I'm sure it would even be possible to open that in VSCode's built-in web view, which could lead to some interesting workflows.

Btw, do you know if there's a way to detect if the terminal supports that ANSI link? When I tried in in the built-in terminal inside VSCode using git-bash it just printed the string but it wasn't clickable.

[kdheepak]

btw, I tested that copy pasting from the HTML example you shared works with Outlook and GMail (i.e. color from syntax highlighting is preserved), but does not work with Slack for example.

What do you think about still using minimal ascii art for conveying some important information? Or maybe ascii art if the line is pure ascii?

[c42f]

detect if the terminal supports that ANSI link

Auto detection is very uneven, unfortunately. Some terminals do support this and I think one of the most sane is kitty: https://sw.kovidgoyal.net/kitty/kittens/query_terminal/

There's the ancient terminfo database, but I very much doubt that has info on hyperlinks which are a recent feature.

There's the TERM environment variable which can sometimes be useful, I don't know?

Generally terminal feature detection is a bit of a disaster area. Some discussion can be found here:
kovidgoyal/kitty#68

IIUC the hyperlink escape sequence is designed to just be ignored by most terminals which don't support it. And it's actually enjoyed quite wide and rapid implementation across terminal emulators, as described in https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

So I think there's a good chance we could turn it on by default. But I'm fairly sure we can't rely on it yet - need several years for wide deployment I suspect.

When I tried in in the built-in terminal inside VSCode using git-bash it just printed the string but it wasn't clickable

IIUC this should probably work according to microsoft/vscode#39278 ? I don't know what's going wrong.

does not work with Slack for example

Yes. And similarly there's no way to get it to work in github markdown. Because after all these years github still doesn't have any way to get colors into markdown. Though... we could solve that with a "copy as image" done on the javascript side. Or similarly "copy as markdown" which could include alt-text for some kind of selectability?