Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions azure-pipelines-PR.yml
Original file line number Diff line number Diff line change
Expand Up @@ -919,3 +919,23 @@ stages:
buildScript: build.sh
displayName: FsharpPlus_Net10_Linux
useVmImage: $(UbuntuMachineQueueName)
- repo: TheAngryByrd/IcedTasks
commit: 863bf91cdee93d8c4c875bb5d321dd92eb20d5a9
buildScript: dotnet build IcedTasks.sln
displayName: IcedTasks_Build
- repo: TheAngryByrd/IcedTasks
commit: 863bf91cdee93d8c4c875bb5d321dd92eb20d5a9
buildScript: dotnet test IcedTasks.sln
displayName: IcedTasks_Test
- repo: demystifyfp/FsToolkit.ErrorHandling
commit: 9cd957e335767df03e2fb0aa2f7b0fed782c5091
buildScript: dotnet build FsToolkit.ErrorHandling.sln
displayName: FsToolkit_ErrorHandling_Build
- repo: demystifyfp/FsToolkit.ErrorHandling
commit: 9cd957e335767df03e2fb0aa2f7b0fed782c5091
buildScript: dotnet test FsToolkit.ErrorHandling.sln
displayName: FsToolkit_ErrorHandling_Test
- repo: opentk/opentk
commit: 60c20cca65a7df6e8335e8d6060d91b30909fbea
buildScript: dotnet build tests/OpenTK.Tests/OpenTK.Tests.fsproj -c Release ;; dotnet build tests/OpenTK.Tests.Integration/OpenTK.Tests.Integration.fsproj -c Release
displayName: OpenTK_FSharp_Build
140 changes: 140 additions & 0 deletions docs/regression-fs0229-bstream-misalignment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Regression: FS0229 B-Stream Misalignment in TypedTreePickle

## Summary

A metadata unpickling regression causes `FS0229` errors when the F# compiler (post-nullness-checking merge) reads metadata from assemblies compiled with `LangVersion < 9.0`. The root cause is a stream alignment bug in `TypedTreePickle.fs` where the secondary metadata stream ("B-stream") gets out of sync between writer and reader.

## Error Manifestation

```
error FS0229: Error reading/writing metadata for assembly '<AssemblyName>':
The data read from the stream is inconsistent, reading past end of resource,
u_ty - 4/B OR u_ty - 1/B, byte = <N>
```

This error occurs when consuming metadata from an assembly where:
1. The assembly was compiled by the current compiler (which writes B-stream data)
2. The compilation used `LangVersion 8.0` or earlier (which disables `langFeatureNullness`)
3. The assembly references BCL types whose type parameters carry `NotSupportsNull` or `AllowsRefStruct` constraints

**Affected real-world library**: [FsToolkit.ErrorHandling](https://github.com/demystifyfp/FsToolkit.ErrorHandling), which uses `<LangVersion>8.0</LangVersion>` for `netstandard2.0`/`netstandard2.1` TFMs.

## Root Cause

### Dual-Stream Metadata Format

F# compiler metadata uses two serialization streams:
- **Stream A** (main): Type tags, type constructor references, type parameter references, etc.
- **Stream B** (secondary): Nullness information per type + newer constraint data (e.g., `NotSupportsNull`, `AllowsRefStruct`)

These streams are written in parallel during pickling and read in parallel during unpickling. The invariant is: **every byte written to stream B by the writer must have a corresponding read in the reader**.

### The Bug

In `p_ty2` (the type pickle function), nullness information is written to stream B **conditionally**:

```fsharp
// BEFORE FIX (buggy)
| TType_app(tc, tinst, nullness) ->
if st.oglobals.langFeatureNullness then
match nullness.Evaluate() with
| NullnessInfo.WithNull -> p_byteB 12 st
| NullnessInfo.WithoutNull -> p_byteB 13 st
| NullnessInfo.AmbivalentToNull -> p_byteB 14 st
// No else branch - B-stream byte skipped when langFeatureNullness = false!
p_byte 2 st
p_tcref "typ" tc st
p_tys tinst st
```

But in `u_ty` (the type unpickle function), the B-stream byte is read **unconditionally**:

```fsharp
| 2 ->
let tagB = u_byteB st // Always reads, regardless of langFeatureNullness at compile time
let tcref = u_tcref st
let tinst = u_tys st
match tagB with
| 0 -> TType_app(tcref, tinst, KnownAmbivalentToNull)
| 12 -> TType_app(tcref, tinst, KnownWithNull)
...
```

This affects type tags 1 (TType_app no args), 2 (TType_app), 3 (TType_fun), and 4 (TType_var).

Meanwhile, `p_tyar_constraints` **unconditionally** writes constraint data to B-stream:

```fsharp
let p_tyar_constraints cxs st =
let cxs1, cxs2 = cxs |> List.partition (function
| TyparConstraint.NotSupportsNull _ | TyparConstraint.AllowsRefStruct _ -> false
| _ -> true)
p_list p_tyar_constraint cxs1 st
p_listB p_tyar_constraintB cxs2 st // Always writes to B, regardless of langFeatureNullness
```

### Misalignment Cascade

When `langFeatureNullness = false`:

1. Writer processes types → skips B-bytes for each type tag 1-4
2. Writer processes type parameter constraints → writes `NotSupportsNull` data to B-stream (value `0x01`)
3. Reader processes types → reads B-stream expecting nullness tags → gets constraint data instead
4. Constraint byte `0x01` is not a valid nullness tag (valid values: 0, 9-20) → `ufailwith "u_ty - 4/B"` or similar

The misalignment cascades: once one byte is read from the wrong position, all subsequent B-stream reads are shifted.

## Fix

Added `else p_byteB 0 st` to all four type cases in `p_ty2`, ensuring a B-byte is always written regardless of `langFeatureNullness`:

```fsharp
// AFTER FIX
| TType_app(tc, tinst, nullness) ->
if st.oglobals.langFeatureNullness then
match nullness.Evaluate() with
| NullnessInfo.WithNull -> p_byteB 12 st
| NullnessInfo.WithoutNull -> p_byteB 13 st
| NullnessInfo.AmbivalentToNull -> p_byteB 14 st
else
p_byteB 0 st // Keep B-stream aligned
p_byte 2 st
p_tcref "typ" tc st
p_tys tinst st
```

Value `0` means "no nullness info / AmbivalentToNull" and is already handled by all reader match cases.

## Timeline

| Date | PR | Change |
|------|-----|--------|
| Jul 2024 | [#15181](https://github.com/dotnet/fsharp/pull/15181) | Nullness checking: introduced B-stream for nullness bytes, conditional write in `p_ty2` |
| Aug 2024 | [#15310](https://github.com/dotnet/fsharp/pull/15310) | Nullness checking applied to codebase |
| Sep 2024 | [#17706](https://github.com/dotnet/fsharp/pull/17706) | `AllowsRefStruct`: added constraint data to B-stream unconditionally via `p_listB` |

The bug was latent from #15181 but only manifested when #17706 added unconditional B-stream writes for constraints. Before #17706, the B-stream was empty when `langFeatureNullness = false`, so the reader's unconditional reads would hit the end-of-stream sentinel (returning 0) harmlessly. After #17706, constraint data appeared in the B-stream even without nullness, causing the misalignment.

## Regression Tests

Two tests added in `tests/FSharp.Compiler.ComponentTests/Import/ImportTests.fs`:

1. **Basic test**: Compiles a library with `LangVersion=8.0` containing generic types with BCL constraints (e.g., `IEquatable<'T>`), then references it from another compilation and verifies no FS0229 error.

2. **Stress test**: Multiple type parameters with various constraint patterns, function types, and nested generics — all compiled at `LangVersion=8.0` and successfully consumed.

## Reproduction

To reproduce the original bug (before fix):

1. Clone [FsToolkit.ErrorHandling](https://github.com/demystifyfp/FsToolkit.ErrorHandling)
2. Inject the pre-fix compiler via `UseLocalCompiler.Directory.Build.props`
3. Build `netstandard2.0` TFM (uses `LangVersion=8.0`)
4. Build `net9.0` TFM that references the `netstandard2.0` output
5. The `net9.0` build fails with `FS0229: u_ty - 4/B`

## Files Changed

- `src/Compiler/TypedTree/TypedTreePickle.fs` — Added `else p_byteB 0 st` to four locations in `p_ty2`
- `tests/FSharp.Compiler.ComponentTests/Import/ImportTests.fs` — Two regression tests
- `tests/FSharp.Compiler.ComponentTests/FSharp.Compiler.ComponentTests.fsproj` — Added `ImportTests.fs` include (was missing since migration)
3 changes: 3 additions & 0 deletions docs/release-notes/.FSharp.Compiler.Service/10.0.300.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
### Fixed

* Fix FS0229 B-stream misalignment when reading metadata from assemblies compiled with LangVersion < 9.0. ([PR #19260](https://github.com/dotnet/fsharp/pull/19260))
* Fix FS3356 false positive for instance extension members with same name on different types. ([PR #19260](https://github.com/dotnet/fsharp/pull/19260))

### Added

### Changed
Expand Down
112 changes: 77 additions & 35 deletions eng/templates/regression-test-jobs.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Template for F# Compiler Regression Tests
# Tests third-party F# projects with the freshly built compiler
#
# buildScript notation:
# - Single command: buildScript: dotnet build MySolution.sln
# - Multiple commands: buildScript: dotnet build A.fsproj ;; dotnet build B.fsproj
# Commands separated by ';;' run sequentially; the job fails on the first non-zero exit code.

parameters:
- name: testMatrix
Expand All @@ -10,7 +15,7 @@ parameters:

jobs:
- ${{ each item in parameters.testMatrix }}:
- job: RegressionTest_${{ replace(item.displayName, '-', '_') }}_${{ replace(replace(item.repo, '/', '_'), '-', '_') }}
- job: RegressionTest_${{ replace(replace(item.displayName, '-', '_'), '.', '_') }}_${{ replace(replace(replace(item.repo, '/', '_'), '-', '_'), '.', '_') }}
displayName: '${{ item.displayName }} Regression Test'
dependsOn: ${{ parameters.dependsOn }}
${{ if item.useVmImage }}:
Expand Down Expand Up @@ -61,18 +66,22 @@ jobs:
Get-ChildItem -Name

$buildScript = '${{ item.buildScript }}'
if ($buildScript -like "dotnet*") {
Write-Host "Build command is a built-in dotnet command: $buildScript"
Write-Host "Skipping file existence check for built-in command"
} else {
Write-Host "Verifying build script exists: $buildScript"
if (Test-Path $buildScript) {
Write-Host "Build script found: $buildScript"
# Support ';;' separator for multiple commands — validate each command's script file
$commands = $buildScript -split ';;' | ForEach-Object { $_.Trim() } | Where-Object { $_ }
foreach ($cmd in $commands) {
if ($cmd -like "dotnet*") {
Write-Host "Built-in dotnet command, skipping file check: $cmd"
} else {
Write-Host "Build script not found: $buildScript"
Write-Host "Available files in root:"
Get-ChildItem
exit 1
$scriptFile = ($cmd -split ' ', 2)[0]
Write-Host "Verifying build script exists: $scriptFile"
if (Test-Path $scriptFile) {
Write-Host "Build script found: $scriptFile"
} else {
Write-Host "Build script not found: $scriptFile"
Write-Host "Available files in root:"
Get-ChildItem
exit 1
}
}
}
displayName: Checkout ${{ item.displayName }} at specific commit
Expand All @@ -95,6 +104,13 @@ jobs:
version: '8.0.x'
installationPath: $(Pipeline.Workspace)/TestRepo/.dotnet

- task: UseDotNet@2
displayName: Install .NET SDK 9.0.x for ${{ item.displayName }}
inputs:
packageType: sdk
version: '9.0.x'
installationPath: $(Pipeline.Workspace)/TestRepo/.dotnet

- task: UseDotNet@2
displayName: Install .NET SDK 10.0.100 for ${{ item.displayName }}
inputs:
Expand Down Expand Up @@ -140,6 +156,18 @@ jobs:
Write-Host "==========================================="
displayName: Report build environment for ${{ item.displayName }}

- pwsh: |
Set-Location $(Pipeline.Workspace)/TestRepo
if (Test-Path ".config/dotnet-tools.json") {
Write-Host "Restoring dotnet tools..."
dotnet tool restore
Write-Host "Tool restore exit code: $LASTEXITCODE"
} else {
Write-Host "No dotnet-tools.json found, skipping tool restore"
}
displayName: Restore dotnet tools for ${{ item.displayName }}
continueOnError: true

- pwsh: |
Set-Location $(Pipeline.Workspace)/TestRepo
Write-Host "============================================"
Expand All @@ -150,7 +178,6 @@ jobs:
Write-Host "============================================"
Write-Host ""

$buildScript = '${{ item.buildScript }}'
$errorLogPath = "$(Pipeline.Workspace)/build-errors.log"
$fullLogPath = "$(Pipeline.Workspace)/build-full.log"

Expand All @@ -168,41 +195,56 @@ jobs:
Add-Content -Path $errorLogPath -Value $line
}
}

if ($buildScript -like "dotnet*") {
Write-Host "Executing built-in command: $buildScript"
if ($IsWindows) {
cmd /c $buildScript 2>&1 | Tee-Object -FilePath $fullLogPath | ForEach-Object {

function Run-Command {
param([string]$cmd)
if ($cmd -like "dotnet*") {
Write-Host "Executing built-in command: $cmd"
if ($IsWindows) {
cmd /c $cmd 2>&1 | Tee-Object -FilePath $fullLogPath -Append | ForEach-Object {
Process-BuildOutput $_
}
} else {
bash -c "$cmd" 2>&1 | Tee-Object -FilePath $fullLogPath -Append | ForEach-Object {
Process-BuildOutput $_
}
}
} elseif ($IsWindows) {
Write-Host "Executing file-based script: $cmd"
cmd /c ".\$cmd" 2>&1 | Tee-Object -FilePath $fullLogPath -Append | ForEach-Object {
Process-BuildOutput $_
}
} else {
bash -c "$buildScript" 2>&1 | Tee-Object -FilePath $fullLogPath | ForEach-Object {
Write-Host "Executing file-based script: $cmd"
$scriptFile = ($cmd -split ' ', 2)[0]
chmod +x "$scriptFile"
bash -c "./$cmd" 2>&1 | Tee-Object -FilePath $fullLogPath -Append | ForEach-Object {
Process-BuildOutput $_
}
}
} elseif ($IsWindows) {
Write-Host "Executing file-based script: $buildScript"
& ".\$buildScript" 2>&1 | Tee-Object -FilePath $fullLogPath | ForEach-Object {
Process-BuildOutput $_
}
} else {
Write-Host "Executing file-based script: $buildScript"
chmod +x "$buildScript"
bash -c "./$buildScript" 2>&1 | Tee-Object -FilePath $fullLogPath | ForEach-Object {
Process-BuildOutput $_
return $LASTEXITCODE
}

# Support ';;' separator for multiple commands
$commands = ('${{ item.buildScript }}' -split ';;') | ForEach-Object { $_.Trim() } | Where-Object { $_ }

foreach ($cmd in $commands) {
$exitCode = Run-Command $cmd
if ($exitCode -ne 0) {
Write-Host ""
Write-Host "============================================"
Write-Host "Command failed: $cmd"
Write-Host "Exit code: $exitCode"
Write-Host "============================================"
exit $exitCode
}
}
$exitCode = $LASTEXITCODE

Write-Host ""
Write-Host "============================================"
Write-Host "Build completed for ${{ item.displayName }}"
Write-Host "Exit code: $exitCode"
Write-Host "Exit code: 0"
Write-Host "============================================"

if ($exitCode -ne 0) {
exit $exitCode
}
displayName: Build ${{ item.displayName }} with local F# compiler
env:
LocalFSharpCompilerPath: $(Pipeline.Workspace)/FSharpCompiler
Expand Down
20 changes: 13 additions & 7 deletions src/Compiler/Checking/PostInferenceChecks.fs
Original file line number Diff line number Diff line change
Expand Up @@ -2680,21 +2680,27 @@ let CheckEntityDefns cenv env tycons =
// check modules
//--------------------------------------------------------------------------

/// Check for duplicate extension member names that would cause IL conflicts.
/// Extension members for types with the same simple name but different fully qualified names
/// will be emitted into the same IL container type, causing a duplicate member error.
/// Check for duplicate static extension member names that would cause IL conflicts.
/// Static extension members for types with the same simple name but different fully qualified names
/// compile to static methods in the same module IL type without a distinguishing first parameter,
/// so they can produce duplicate IL method signatures.
/// Instance extension members are safe because they compile with the extended type as the first
/// parameter, which differentiates the IL signatures.
let CheckForDuplicateExtensionMemberNames (cenv: cenv) (vals: Val seq) =
if cenv.reportErrors then
let extensionMembers =
let staticExtensionMembers =
vals
|> Seq.filter (fun v -> v.IsExtensionMember && v.IsMember)
|> Seq.filter (fun v ->
v.IsExtensionMember
&& v.IsMember
&& not (v.IsInstanceMember))
|> Seq.toList

if not extensionMembers.IsEmpty then
if not staticExtensionMembers.IsEmpty then
// Group by LogicalName which includes generic arity suffix (e.g., Expr`1 for Expr<'T>)
// This matches how types are compiled to IL, so Expr and Expr<'T> are separate groups
let groupedByLogicalName =
extensionMembers
staticExtensionMembers
|> List.groupBy (fun v -> v.MemberApparentEntity.LogicalName)

for (logicalName, members) in groupedByLogicalName do
Expand Down
Loading
Loading