Skip to content

Skip /clr tests - #796

Merged
Stephan T. Lavavej (StephanTLavavej) merged 3 commits into
microsoft:masterfrom
StephanTLavavej:skip_clr
May 7, 2020
Merged

Stephan T. Lavavej (StephanTLavavej) merged 3 commits into
microsoft:masterfrom
StephanTLavavej:skip_clr

Conversation

@StephanTLavavej

@StephanTLavavej Stephan T. Lavavej (StephanTLavavej) commented May 5, 2020 •

Copy link
Copy Markdown
Member

Fixes #788. We don't yet build /clr support libraries, so we shouldn't attempt to test them. (This fails on my machine, for some reason; on other machines it tests a Frankenstein combination of STL OSS and released VS code.)

Additionally, this changes the Python logic to use exact matches for compiler options instead of startswith. This doesn't make a difference given the current ordering (looking for clr:pure first) and our current compiler options, but I believe this is more robust (given the existence of /clr:meow variants). As before, we ignore the first character because MSVC supports -meow and /meow option styles.

This PR resolves an enormous number of test failures on my machine. Now, the test log looks like:

    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0009999275207519531,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:19",
      "output": "Test requires the following unavailable features: clr"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:20",
      "output": "Test requires the following unavailable features: clr"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:21",
      "output": "Test requires the following unavailable features: clr_pure"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:22",
      "output": "Test requires the following unavailable features: clr_pure"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:23",
      "output": "Test requires the following unavailable features: edg"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0009996891021728516,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:24",
      "output": "Test requires the following unavailable features: edg"
    },
    {
      "code": "UNSUPPORTED",
      "elapsed": 0.0,
      "name": "std :: tests/Dev08_496675_iostream_int_reading:25",
      "output": "Test requires the following unavailable features: edg"
    },

I verified that the configuration numbers for /clr, /clr:pure, and /BE correspond to the lines in the test matrix.

I believe this is my first 100% Python PR. 😸

Comment thread tests/utils/stl/test/tests.py Outdated
Comment thread tests/utils/stl/test/tests.py Outdated
@cbezault

Copy link
Copy Markdown
Contributor

UNSUPPORTED isn't a test failure. It just means the prerequisites to run the test are not satisfied.

@StephanTLavavej

Copy link
Copy Markdown
Member Author

UNSUPPORTED isn't a test failure. It just means the prerequisites to run the test are not satisfied.

Yep, I understand. Before this change, /clr outright failed for me, due to missing mscoree.lib. Now it's unsupported, which is good.

@cbezault

Copy link
Copy Markdown
Contributor

Oh woops, I misread the description as complaining that you were getting UNRESOLVEDs as opposed to describing the behavior you were now seeing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Related to test code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: We should skip /clr until we actually build it

3 participants