Skip to content

Commit f210913

Browse files
authored
perf(sdk): cut analyzer time in half and add a local build opt-out (#59)
* perf(sdk): cut analyzer time in half and add a local build opt-out Every defect class is now reported by one analyzer rule. About 100 duplicate rules are off, Asyncify is removed, and 12 VS Threading rules that only apply inside Visual Studio extensions are off. Across the 434 projects of headless-framework this halves analyzer CPU time and cuts a full rebuild from 61 s to 46 s. Local builds may set RunAnalyzersDuringBuild=false or RunAnalyzersDuringLiveAnalysis=false. Roslyn reads those only while RunAnalyzers is empty, so local builds now clear it instead of forcing true. CI and AI-agent builds still force analyzers on. Crash and hang dumps now default on only in CI (EnableTestDumps). CA1510-CA1513 are off while the guard-clause ban list is active, because they recommend the helpers that list bans. * refactor(sdk): organize injected analyzer configs and make the editorconfig scaffold editor-only The injected analyzer configs now keep one section per analyzer family, with rules sorted by ID, so every rule has one predictable place and a comment naming why it differs from the package default. The tests overlay drops entries that repeated the base value or re-enabled a duplicate the base turns off (AsyncFixer03). editorconfig.txt no longer repeats severities, code-style preferences, or naming rules. A consumer .editorconfig outranks the injected configs, so a copied severity pinned that repository to the SDK version it was copied from. ReSharper's inspectcode reads the injected configs, which was the other reason for the copy. * perf(sdk): turn off three noisy rules and drop entries that restate defaults MA0045 is migration advice ("this sync method should become async"), not a defect, and VSTHRD002 already reports synchronous task waits; it was the most-suppressed analyzer rule in measured consumers. CA1508 is costly flow analysis with frequent false positives on defensive checks. VSTHRD003's deadlock needs JoinableTaskFactory main-thread affinity, which only Visual Studio-style apps have. The 26 CA entries set to warning only restated what AnalysisMode=All already sets. Third-party entries stay explicit even when they match today's package default, so a package that changes its default in a later version cannot silently change ours. * build: generate the repository .editorconfig from the shipped configs This repository's own projects build with plain Microsoft.NET.Sdk, so they never receive the injected Headless.NET.Sdk.*.editorconfig files. Its root .editorconfig was an old hand-made copy that still carried pre-reorganization severities and the dotnet_analyzer_diagnostic.severity = default line. eng/tools/sync_root_editorconfig.py now generates it from editorconfig.txt plus the injected Analyzers and Tests configs, and root_editorconfig_should_match_the_injected_configs fails when they drift. The integration tests also lose their remaining analyzer notes: two regexes move to [GeneratedRegex] (SYSLIB1045), and the nuspec helpers open and dispose the package archive themselves, so CA2000 no longer loses track of ownership at three call sites.
1 parent 810ee66 commit f210913

33 files changed

Lines changed: 1697 additions & 1448 deletions

‎.editorconfig‎

Lines changed: 742 additions & 355 deletions
Large diffs are not rendered by default.

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
The product is the consumer contract, and the README's **Support contract** section is its source of truth. Read that section before you change anything under `src/`. Two consequences guide most decisions:
88

99
- **Every consumption mode is first-class.** A change must behave the same under PackageReference and MSBuild-SDK consumption, in single- and multi-targeting builds, and for every satellite. Prove it with a consumer-build test, not by reading the targets.
10-
- **Policies are not consumer knobs.** Analyzer infrastructure and quality gates stay on. Add an opt-out only where the README already documents one, such as the banned-symbol lists.
10+
- **Policies are not consumer knobs.** Analyzer infrastructure and quality gates stay on. Add an opt-out only where the README already documents one, such as the banned-symbol lists. The local inner-loop switches `RunAnalyzersDuringBuild` and `RunAnalyzersDuringLiveAnalysis` are honored only outside CI and AI-agent builds, which force them back on.
1111

1212
## Where a change lands
1313

@@ -28,7 +28,7 @@ Use the `make` targets; `make help` lists them.
2828
- **Local packs need a unique version.** The integration fixture refuses a package version that already exists in `~/.nuget/packages`, because the cached copy would shadow the packages under test. The Makefile packs as `0.0.0-local.`. When you pack by hand, pass `-p:MinVerVersionOverride=-local.`.
2929
- **The build fails on unformatted C#.** `CSharpier.MSBuild` reports `Was not formatted` as a build error and does not fix the file. Run `dotnet tool restore`, then `dotnet csharpier format `. Keep the CSharpier version in `dotnet-tools.json` equal to the `CSharpier.MSBuild` version in `Directory.Packages.props`.
3030
- **Agent warnings-as-errors does not apply here.** `SupportDetectLlmContext.props` turns warnings into errors when an agent drives a consumer build. It is product behavior, covered by the integration tests. This repository's own projects use plain `Microsoft.NET.Sdk`, so it does not affect your builds.
31-
- **Severity edits go in two places.** Rider, ReSharper, and `jb inspectcode` read severities only from a real `.editorconfig`, and projects outside the SDK never receive the injected configs. `configurations/editorconfig.txt` repeats the injected severities for them. When you change a severity in `Headless.NET.Sdk.Analyzers.editorconfig` or `Headless.NET.Sdk.Tests.editorconfig`, make the same change in `editorconfig.txt`. `editorconfig_scaffold_severities_should_match_the_injected_configs` fails on drift.
31+
- **Severities live only in the injected configs.** Edit `Headless.NET.Sdk.Analyzers.editorconfig` or `Headless.NET.Sdk.Tests.editorconfig`; each keeps one section per analyzer family with rules sorted by ID. The compiler and ReSharper's `jb inspectcode` both read them. `configurations/editorconfig.txt` is an editor-only scaffold, because a consumer `.editorconfig` outranks the SDK and a copied severity would pin that repository to the SDK version it was copied from. `editorconfig_scaffold_should_not_carry_analyzer_settings` fails if one is added. This repository's own root `.editorconfig` is generated from those files by `python3 eng/tools/sync_root_editorconfig.py`, because its projects use plain `Microsoft.NET.Sdk`; rerun the script after editing a shipped config, or `root_editorconfig_should_match_the_injected_configs` fails.
3232
- **Version pins move together.** A tool or analyzer version appears in `Directory.Packages.props`, in the shipped props that expose it, and in `tests/Headless.NET.Sdk.TestToolVersions.Anchor`. The anchor exists only so that Dependabot opens bump PRs. `VersionConsistencyTests` fails when the three disagree.
3333
- **Test names** use `should_{action}_{expected}_when_{condition}`.
3434

‎Directory.Build.targets‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
<RepositoryCommit Condition="'$(RepositoryCommit)' == ''">$(SourceRevisionId)RepositoryCommit>
2424
<_HeadlessNuspecDescription>$([MSBuild]::Escape('$(Description)'))_HeadlessNuspecDescription>
2525
<_HeadlessNuspecPackageTags>$([MSBuild]::Escape('$(PackageTags)'))_HeadlessNuspecPackageTags>
26-
PackageId=$(PackageId);PackageVersion=$(PackageVersion);Title=$(Title);Authors=$(Authors);Description=$(_HeadlessNuspecDescription);PackageIcon=$(PackageIcon);PackageReadmeFile=$(PackageReadmeFile);PackageProjectUrl=$(PackageProjectUrl);PackageReleaseNotes=$(PackageReleaseNotes);PackageTags=$(_HeadlessNuspecPackageTags);PackageProjectDirectory=$(MSBuildProjectDirectory);BaseSdkDirectory=$(MSBuildThisFileDirectory)src/Headless.NET.Sdk;RepositoryRoot=$(MSBuildThisFileDirectory);RepositoryUrl=$(RepositoryUrl);RepositoryType=$(RepositoryType);RepositoryBranch=$(RepositoryBranch);RepositoryCommit=$(RepositoryCommit);Copyright=$(Copyright);MeziantouAnalyzerVersion=$(_HeadlessMeziantouAnalyzerVersion);BannedApiAnalyzersVersion=$(_HeadlessBannedApiAnalyzersVersion);AsyncFixerVersion=$(_HeadlessAsyncFixerVersion);AsyncifyVersion=$(_HeadlessAsyncifyVersion);VisualStudioThreadingAnalyzersVersion=$(_HeadlessVisualStudioThreadingAnalyzersVersion);MultithreadingAnalyzerVersion=$(_HeadlessMultithreadingAnalyzerVersion);RoslynatorAnalyzersVersion=$(_HeadlessRoslynatorAnalyzersVersion);RoslynatorFormattingAnalyzersVersion=$(_HeadlessRoslynatorFormattingAnalyzersVersion);ReflectionAnalyzersVersion=$(_HeadlessReflectionAnalyzersVersion);ErrorProneAnalyzersVersion=$(_HeadlessErrorProneAnalyzersVersion);SbomTargetsVersion=$(_HeadlessMicrosoftSbomTargetsVersion)
26+
PackageId=$(PackageId);PackageVersion=$(PackageVersion);Title=$(Title);Authors=$(Authors);Description=$(_HeadlessNuspecDescription);PackageIcon=$(PackageIcon);PackageReadmeFile=$(PackageReadmeFile);PackageProjectUrl=$(PackageProjectUrl);PackageReleaseNotes=$(PackageReleaseNotes);PackageTags=$(_HeadlessNuspecPackageTags);PackageProjectDirectory=$(MSBuildProjectDirectory);BaseSdkDirectory=$(MSBuildThisFileDirectory)src/Headless.NET.Sdk;RepositoryRoot=$(MSBuildThisFileDirectory);RepositoryUrl=$(RepositoryUrl);RepositoryType=$(RepositoryType);RepositoryBranch=$(RepositoryBranch);RepositoryCommit=$(RepositoryCommit);Copyright=$(Copyright);MeziantouAnalyzerVersion=$(_HeadlessMeziantouAnalyzerVersion);BannedApiAnalyzersVersion=$(_HeadlessBannedApiAnalyzersVersion);AsyncFixerVersion=$(_HeadlessAsyncFixerVersion);VisualStudioThreadingAnalyzersVersion=$(_HeadlessVisualStudioThreadingAnalyzersVersion);MultithreadingAnalyzerVersion=$(_HeadlessMultithreadingAnalyzerVersion);RoslynatorAnalyzersVersion=$(_HeadlessRoslynatorAnalyzersVersion);RoslynatorFormattingAnalyzersVersion=$(_HeadlessRoslynatorFormattingAnalyzersVersion);ReflectionAnalyzersVersion=$(_HeadlessReflectionAnalyzersVersion);ErrorProneAnalyzersVersion=$(_HeadlessErrorProneAnalyzersVersion);SbomTargetsVersion=$(_HeadlessMicrosoftSbomTargetsVersion)
2727
<NuspecProperties Condition="'$(PackageId)' == 'Headless.NET.Sdk.Test'"
2828
>$(NuspecProperties);MtpCrashDumpVersion=$(_HeadlessMtpCrashDumpVersion);MtpCodeCoverageVersion=$(_HeadlessMtpCodeCoverageVersion);MtpHangDumpVersion=$(_HeadlessMtpHangDumpVersion);MtpHotReloadVersion=$(_HeadlessMtpHotReloadVersion);MtpRetryVersion=$(_HeadlessMtpRetryVersion);MtpTrxReportVersion=$(_HeadlessMtpTrxReportVersion)NuspecProperties>
2929
PropertyGroup>

‎Directory.Packages.props‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,6 @@
3838
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.290" />
3939
<PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.6.0" />
4040
<PackageVersion Include="AsyncFixer" Version="2.1.0" />
41-
<PackageVersion Include="Asyncify" Version="0.9.7" />
4241
<PackageVersion Include="Microsoft.VisualStudio.Threading.Analyzers" Version="18.7.23" />
4342
<PackageVersion Include="SmartAnalyzers.MultithreadingAnalyzer" Version="1.1.31" />
4443
<PackageVersion Include="Roslynator.Analyzers" Version="5.0.0" />

‎README.md‎

Lines changed: 51 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Release notes are maintained in [GitHub Releases](https://github.com/xshaheen/he
1212
is identical; the documented first-clean-restore bootstrap is required for PackageReference mode.
1313
- Package assets apply only to the project that opts in. The packages do not ship `buildTransitive` assets.
1414
- Multi-targeting outer builds remain supported through `buildMultiTargeting`; inner builds receive the normal `build` contract exactly once.
15-
- Named quality policies are authoritative. The analyzer infrastructure and CI quality gates are not consumer opt-outs; the two shipped banned-symbol lists retain the documented whole-policy and per-list opt-outs.
15+
- Named quality policies are authoritative. The analyzer infrastructure and CI quality gates are not consumer opt-outs; the two shipped banned-symbol lists retain the documented whole-policy and per-list opt-outs. A local inner loop may skip analyzers during build or live analysis, but CI and AI-agent builds always run them.
1616

1717
## Package family
1818

@@ -195,7 +195,6 @@ The following analyzer packages are injected as private, implicit dependencies f
195195
- `Meziantou.Analyzer`
196196
- `Microsoft.CodeAnalysis.BannedApiAnalyzers`
197197
- `AsyncFixer`
198-
- `Asyncify`
199198
- `Microsoft.VisualStudio.Threading.Analyzers`
200199
- `SmartAnalyzers.MultithreadingAnalyzer`
201200
- `Roslynator.Analyzers`
@@ -205,7 +204,7 @@ The following analyzer packages are injected as private, implicit dependencies f
205204

206205
The sole self-reference exception is a project whose evaluated `PackageId` is
207206
`Meziantou.Analyzer`; Headless omits that one analyzer reference so the analyzer package can use
208-
the SDK without depending on itself. The other nine analyzer references and all mandatory policy
207+
the SDK without depending on itself. The other eight analyzer references and all mandatory policy
209208
still apply.
210209

211210
`Roslynator.Formatting.Analyzers` complements CSharpier without becoming a second formatter. The
@@ -220,15 +219,27 @@ literals and inline suppression reasons that no line-length warning can fix.
220219

221220
The bundled general, Newtonsoft.Json, and guard-clause banned-symbol lists are enabled by default. The guard-clause list bans the BCL throw helpers `ArgumentNullException.ThrowIfNull`, `ArgumentException.ThrowIfNullOrEmpty` and `ThrowIfNullOrWhiteSpace`, every `ArgumentOutOfRangeException.ThrowIf*`, and `ObjectDisposedException.ThrowIf`, and points to the `Headless.Checks` `Argument.*` and `Ensure.*` guards instead. Consumers can disable the complete banned-symbol policy with `DisableSupportBannedSymbols=true`, or disable any list independently through `IncludeDefaultBannedSymbols=false`, `BannedNewtonsoftJsonSymbols=false`, and `BannedGuardClauseSymbols=false`. The `Microsoft.CodeAnalysis.BannedApiAnalyzers` package remains part of the analyzer infrastructure.
222221

223-
The SDK also ships `vs-threading.SyncMethodsToExcludeFromVSTHRD103.Headless.txt`, which stops
224-
VSTHRD103 from reporting synchronous calls that do no I/O and no blocking wait inside async methods:
225-
EF Core `Add`/`AddRange` and `IDbContextFactory.CreateDbContext`, `MemoryStream`, `StringReader`, and
226-
`StringWriter` operations, `CancellationTokenSource.Cancel`, `Timer.Dispose`, and xUnit/NUnit
227-
assertions over synchronous delegates. The analyzer merges every
222+
Each defect class is reported by one rule. Where packages overlap, the SDK keeps the rule with the
223+
broader trigger set or a code fix and sets the duplicates to `none`. Examples: CA1849 reports blocking
224+
calls in async code, replacing VSTHRD103, AsyncFixer02, and MA0042. MA0001, MA0074, and MA0011
225+
report culture and comparison defaults, replacing CA1304, CA1305, CA1307, CA1309, CA1310, and CA1311.
226+
VSTHRD002 reports synchronous waits as a suggestion, because most are deliberate sync bridges. The Visual Studio extension rules that depend on
227+
JoinableTaskFactory (VSTHRD001, VSTHRD004, VSTHRD010, VSTHRD011, VSTHRD012, VSTHRD102, VSTHRD106,
228+
VSTHRD108, VSTHRD109, VSTHRD112, VSTHRD113, and VSTHRD115) are off. VSTHRD010 alone cost 7% of
229+
analyzer time in a 434-project solution. The comment beside each `none` entry in
230+
`Headless.NET.Sdk.Analyzers.editorconfig` names the rule that replaces it. A consumer `.editorconfig`
231+
can re-enable any of them. While the guard-clause banned-symbol list is active, CA1510 through CA1513
232+
are off, because they recommend the helpers that the list bans.
233+
234+
The SDK also ships `vs-threading.SyncMethodsToExcludeFromVSTHRD103.Headless.txt` for consumers that
235+
re-enable VSTHRD103. It stops VSTHRD103 from reporting synchronous calls that do no I/O and no
236+
blocking wait inside async methods: EF Core `Add`/`AddRange` and `IDbContextFactory.CreateDbContext`,
237+
`MemoryStream`, `StringReader`, and `StringWriter` operations, `CancellationTokenSource.Cancel`,
238+
`Timer.Dispose`, and xUnit/NUnit assertions over synchronous delegates. The analyzer merges every
228239
`vs-threading.SyncMethodsToExcludeFromVSTHRD103*.txt` additional file, so a consumer adds its own
229-
exclusions in a separately named file.
240+
exclusions in a separately named file. CA1849 has no exclusion mechanism.
230241

231-
The SDK owns the versions of all ten implicit analyzer references. Central Package Management
242+
The SDK owns the versions of all nine implicit analyzer references. Central Package Management
232243
consumers must not add `PackageVersion` entries for those analyzer IDs. SDK-form consumption rejects
233244
them as SDK-defined implicit references with NU1009; PackageReference consumption rejects conflicting
234245
central versions against the package family's exact dependency ranges.
@@ -263,6 +274,9 @@ the listed default; explicit values win unless the behavior is identified as man
263274
| `MinimumExpectedTests` | `1` | Sets the MTP minimum-test guard. Set `0` to omit only the SDK-supplied `--minimum-expected-tests` argument; this does not guarantee that a zero-test run succeeds. |
264275
| `EnableXunitEntryPointDisableWarnings` | `true` when a supported xUnit v3 package is directly referenced | xUnit v3 4.0.1 and later wrap their generated entry point and AOT source in `#pragma warning disable` when `XUNIT_GENERATED_DISABLE_WARNINGS` is defined, so analyzer warnings cannot fail a build on code the consumer does not own. Set `false` to prevent the SDK from adding the constant. |
265276
| `OptimizeTestRun` | enabled unless `false` | Set `false` to keep analyzers enabled during MTP's test-build phase. |
277+
| `EnableTestDumps` | `true` on CI, otherwise unset | Adds the MTP `--crashdump` and `--hangdump` (10-minute timeout) arguments when `true`. Each extension relaunches the test host under a controller process, which costs about 0.2 s per local run. |
278+
| `RunAnalyzersDuringBuild` | `true` | Set `false` in a local inner loop to compile without analyzers, about 40% faster; editor live analysis still reports findings. CI and AI-agent builds force it back to `true`. |
279+
| `RunAnalyzersDuringLiveAnalysis` | `true` | Set `false` locally to stop editor background analysis while builds still analyze. CI and AI-agent builds force it back to `true`. |
266280
| `DisableSupportPackageInformation` | `false` | Set `true` to opt out of Headless package metadata and symbol policy. |
267281
| `SearchReadmeFileAbove` | `false` | Searches parent directories for a package README. |
268282
| `DisableReadme` | `false` | Prevents automatic package README discovery and packing. |
@@ -285,10 +299,29 @@ the listed default; explicit values win unless the behavior is identified as man
285299
| `HeadlessCopyGitAttributesToSolutionDir` | master selector | Selects only `.gitattributes`. |
286300
| `HeadlessOverwriteConfigFiles` | `false` | Allows the explicit scaffold target to replace existing files. |
287301

288-
The explicit target framework, ten analyzer packages, analyzer configuration, CI warning gate,
302+
The explicit target framework, nine analyzer packages, analyzer configuration, CI and AI-agent analyzer execution, CI warning gate,
289303
NuGet audit policy, and SDK-owned MTP extension
290304
versions are mandatory policy. Legacy analyzer/configuration opt-out names do not disable them.
291305

306+
### Inner-loop performance
307+
308+
Analyzers cost roughly 40% of compile time on a typical library. A developer who wants faster
309+
local builds can keep analysis in the editor and skip it during build. Set the property in
310+
`Directory.Build.props` for the whole team, or in a user-local props file that it imports:
311+
312+
```xml
313+
<PropertyGroup>
314+
<RunAnalyzersDuringBuild>falseRunAnalyzersDuringBuild>
315+
PropertyGroup>
316+
```
317+
318+
Findings still reach the developer through the editor, and CI and AI-agent builds enforce them.
319+
Do not pass it as a global property (`-p:` or `Directory.Build.rsp`): a global property overrides
320+
the CI and AI-agent re-assertion, so those builds would skip analyzers too.
321+
Keep editor background analysis scoped to open files: that is the default in Visual Studio and in
322+
VS Code (`dotnet.backgroundAnalysis.analyzerDiagnosticsScope`). In Rider, leave solution-wide
323+
analysis off on large solutions.
324+
292325
## CI, restore, and vulnerability policy
293326

294327
Headless detects common CI provider environment variables. Consumers can also activate CI behavior with `ContinuousIntegrationBuild=true`; there is no separate `IsContinuousIntegration` input.
@@ -322,7 +355,7 @@ Unlike CI detection, this signal is consumer-overridable: set `HeadlessIsLlmCont
322355

323356
## Test SDK contract
324357

325-
`Headless.NET.Sdk.Test` is Microsoft Testing Platform only. It defaults test hosts to `OutputType=Exe`, `IsTestProject=true`, `IsPackable=false`, and `IsPublishable=false`, and supplies restore-visible MTP extensions for crash dumps, hang dumps, hot reload, retry, TRX reporting, and coverage. Default execution includes TRX output, crash and hang dumps, and a minimum expected test count; coverage is enabled on CI.
358+
`Headless.NET.Sdk.Test` is Microsoft Testing Platform only. It defaults test hosts to `OutputType=Exe`, `IsTestProject=true`, `IsPackable=false`, and `IsPublishable=false`, and supplies restore-visible MTP extensions for crash dumps, hang dumps, hot reload, retry, TRX reporting, and coverage. Default execution includes TRX output and a minimum expected test count; coverage and crash and hang dumps are enabled on CI.
326359

327360
The test framework remains consumer-selected. For example:
328361

@@ -436,6 +469,12 @@ dotnet build -t:HeadlessScaffoldConfigFiles
436469

437470
Use the `HeadlessCopy*` selectors for individual files and `HeadlessOverwriteConfigFiles=true` only when replacement is intended.
438471

472+
The scaffolded `.editorconfig` carries editor, formatter, and ReSharper settings only. Analyzer
473+
severities, code-style preferences, and naming rules stay in the injected configs, so an SDK upgrade
474+
applies them without editing the consumer's `.editorconfig`. A severity line in a consumer
475+
`.editorconfig` outranks the SDK, so keep only deliberate overrides there. A repository that copied an
476+
earlier scaffold, which repeated every severity, should delete those copied severity lines.
477+
439478
## Building and publishing this repository
440479

441480
The repository uses the .NET SDK pinned by `global.json`. `make verify` runs the CI sequence and writes a test summary under `artifacts/proof/`; `make help` lists the faster targets. The equivalent commands:

0 commit comments

Comments
 (0)