Skip to content

Add regression tests for DAP debugger secret masking - #4579

Merged
rentziass merged 2 commits into
mainfrom
rentziass-dap-masking-regression-tests
Aug 3, 2026
Merged

rentziass merged 2 commits into
mainfrom
rentziass-dap-masking-regression-tests

Conversation

@rentziass

@rentziass rentziass commented Jul 28, 2026 •

Copy link
Copy Markdown
Member

Why

The DAP transport is a secret-carrying channel that bypasses the job log's masking, so every user-visible string a DAP producer relays has to go through the runner's SecretMasker at the point of construction. (The raw protocol JSON deliberately isn't masked - masking it would corrupt envelope fields like type/command/seq if a secret collided with them. See the comment on SendMessageInternal.)

That behavior was already correct, but it was unpinned: several sinks had no test at all, so a future refactor could silently drop a mask call and nothing would go red. The welcome message and DapVariableProvider were covered; the REPL and the rest of DapDebugger were not.

What

Adds L0 regression tests for each previously untested sink:

Producer Sink
DapReplExecutor process stdout
DapReplExecutor process stderr
DapReplExecutor EvaluateResponseBody error path (step host throws)
DapReplExecutor ${{ }} expression expansion
DapDebugger HandleMessageAsync catch-all error response
DapDebugger threads response job label
DapDebugger stopped event step description

Every one of these was verified to fail when its mask call is removed, so none of them are vacuous.

DapReplExecutorL0 gains a FakeStepHost that replays canned stdout/stderr through the same OutputDataReceived/ErrorDataReceived events a real process raises. This lets the full REPL output pipeline run end to end without launching a process. The DapDebugger tests go over a real socket using the harness already in DapDebuggerL0.

The HandleMessageAsync test is the one bit of cleverness worth flagging: it triggers the catch-all by sending a non-integer frameId, because Newtonsoft embeds the offending value in its exception message. That gives a realistic case where attacker-influenced input ends up in an error string.

The one deliberate gap

DapReplExecutor echoes $

Comment on lines +1703 to +1706
var response = await ReadDapMessageAsync(stream, TimeSpan.FromSeconds(5));
Assert.Contains("\"success\":false", response);
Assert.Contains("***", response);
Assert.DoesNotContain(secret, response, StringComparison.Ordinal);
[Fact]
[Trait("Level", "L0")]
[Trait("Category", "Worker")]
public void BuildEnvironment_MasksNothingButExpandsSecretValuesForExecution()
The DAP transport is a secret-carrying channel that bypasses the job
log's masking, so every user-visible string a DAP producer relays is run
through the runner's SecretMasker at the point of construction. That
behavior was correct but unpinned: nothing failed if a sink lost its
mask call.

Add L0 regression tests covering each previously untested sink:

  DapReplExecutor  stdout, stderr, the EvaluateResponseBody error path,
                   and expression expansion
  DapDebugger      the HandleMessageAsync catch-all error response,
                   the threads response job label, and the stopped
                   event step description

Each test was verified to fail when its mask call is removed.

The one REPL sink deliberately left unmasked is the console echo of the
script the user just typed, which only reflects their own input back to
the session that sent it. Cover its truncation behavior instead, so the
gap reads as a decision rather than an oversight.

DapReplExecutorL0 gains a FakeStepHost so the output pipeline can be
exercised without launching a process, replaying canned stdout/stderr
through the same events a real process raises.

Refs github/actions-runtime#5586 (security review finding #13).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e4fd6981-5088-4e99-9cc2-d1b16bf3c44a
@rentziass
rentziass force-pushed the rentziass-dap-masking-regression-tests branch from ffbee36 to 4ad1d89 Compare July 28, 2026 13:51
@rentziass
rentziass enabled auto-merge (squash) August 3, 2026 08:44
@rentziass
rentziass merged commit 35bcfea into main Aug 3, 2026
12 checks passed
@rentziass
rentziass deleted the rentziass-dap-masking-regression-tests branch August 3, 2026 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants