Optimize TypeSpec Validation CI runtime in the spec repo

[v-xuto]

Description:

The TypeSpec Validation CI in the spec repo is taking too long to complete. In many cases, the run time is around an hour or even longer, which significantly slows down development and validation workflows.

Example link: https://github.com/Azure/azure-rest-api-specs/actions/runs/28857085073/job/85586200452?pr=44494

Expected behavior:

TypeSpec Validation CI should finish in a reasonable amount of time, ideally much faster than the current runtime.

Actual behavior:

The CI often takes one hour or more to complete.

Impact:

• Slower feedback loop for spec changes
• Reduced development efficiency
• Longer wait time for validation results

@mikeharder, @qiaozha for notification.

[timotheeguerin]

while I do agree there is perf improvements to do, I am not sure spending time on making typespec validatrion faster is worth it, if you touch every spec which cause this super long pipeline then TSV all pipeline is better and already paralizzed(though as well could be faster)

[qiaozha]

if I remember correctly, the tsv runs against the service not the changed files?

[timotheeguerin]

Typespec Validation job runs only on changed services.

Typespec Validation All runs on all but only when it is triggered by changing dependencies or tsv code

[qiaozha]

I know what you mean, this issue is about TypeSpec Validation, which often take more than an hour to finish.

[qiaozha]

If you take a look at the screenshot, TypeSpec Validation All finishes faster than TypeSpec Validation.

[timotheeguerin]

That's what I said in my first reply, tsv all is parallelized, tsv is meant to run on regular PRs where only 1 service is modified. If you modify a lot it will be slow and I don't know if we want to care about that. Parallelizing it will make it worse for happy path by creating additional jobs and so worse for service teams.
My point is if you touch a lot of services for migration just look at tsv all.

[qiaozha]

Can we make those linter rules run parallelly within TSV runs against a single service? 40 minutes is still a lot and I don't think those linter rule has dependencies?

[timotheeguerin]

There is definietly some performance improvements to do both in the multi run and tsv itself. I had this PR #43183 to runs services in parallel #43183 but with the git checks this require that we migrate every service to structure v2 before we can do that. I think it had to do with knowing if a file was out of the service directory.

[qiaozha]

I actually don't think folder structure validation should be added within TSV before, because it validates against the service structure, whether that services is using swagger or TypeSpec to express its API doesn't matter.