Skip to content

Flaky test testFindItemsByNamespace #43

@ThugJudy

Description

@ThugJudy
  • I have checked the discussions
  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I have checked the FAQ of this repository and believe that this is not a duplicate.

Describe the bug

The test case com.ctrip.framework.apollo.openapi.client.service.ItemOpenApiServiceTest.testFindItemsByNamespace fails due to the below assertion:

assertEquals(String.format("%s/envs/%s/apps/%s/clusters/%s/namespaces/%s/items?size=%s&page=%s",
someBaseUrl, someEnv, someAppId, someCluster, someNamespace, size, page), get.getURI().toString());

The parameters of the URI if shuffled, can cause flakiness as it is a simple string comparison where the order in which the parameters arrive is not maintained. The flakiness occurs at the following point:
https://github.com/apolloconfig/apollo-java/blob/main/apollo-openapi/src/main/java/com/ctrip/framework/apollo/openapi/client/service/ItemOpenApiService.java#L215-L222

The add params in line 221 and 222, adds the param to a hashMap. As hashmaps do not preserve the order in which the parameters arrive, these params may be shuffled sometimes causing flakiness.

I found and confirmed the flaky behavior using an open-source research tool NonDex, which shuffles implementations of nondeterminism operations.

To Reproduce

The following command can be used to reproduce assertion failures and verify the fix:

mvn -pl main edu.illinois:nondex-maven-plugin:2.1.1:nondex  -Dtest=com.ctrip.framework.apollo.openapi.client.service.ItemOpenApiServiceTest#testFindItemsByNamespace

Expected behavior

The test is expected to pass when the nondex tool runs but it it fails in most cases.

Additional Details & Logs

  • Java version "1.8.0_381"
  • Apache Maven 3.6.3
  • macOS Venture Version 13.4.1 (22F82)

Please let me know if you have any concerns or questions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions