From 7cfbf57d8671fe2b1667af066aa67e0d19b3406e Mon Sep 17 00:00:00 2001 From: James Teh Date: Tue, 3 Jun 2025 14:44:05 +1000 Subject: [PATCH 1/4] Tentative testdriver methods --- rfcs/tentative_testdriver_methods.md | 29 ++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 rfcs/tentative_testdriver_methods.md diff --git a/rfcs/tentative_testdriver_methods.md b/rfcs/tentative_testdriver_methods.md new file mode 100644 index 0000000..c53c7a7 --- /dev/null +++ b/rfcs/tentative_testdriver_methods.md @@ -0,0 +1,29 @@ +# RFC 226: Tentative testdriver methods + +## Summary +Allow tentative methods to be added to testdriver.js that are not yet included in the WebDriver or WebDriver BiDi specifications. +This should only be allowed where there is consensus between multiple browser vendors and where there is clear progress towards a specification. + +## Background +As part of the [Interop 2025 Accessibility Investigation Area](https://github.com/web-platform-tests/interop-accessibility/issues/148), the intention is to extend browsers and WPT to allow additional accessibility properties to be tested by web platform tests. +However, there are open questions regarding the shape of the API for exposing these properties, what properties should be exposed, how the tests should be written, etc. +While the end goal is to extend the WebDriver specification with the required new endpoints, these open questions need to be answered before this is feasible. +To answer these questions, it would be helpful if browser vendors could collaborate on a tentative but working implementation. + +## Details +1. There must be publicly documented (e.g. as part of an Interop Investigation Area) consensus between at least two browser vendors and intent to work towards a specification. +2. Any tentative methods added to testdriver.js should be prefixed with `tentative_`. +3. Any utility methods which call tentative testdriver methods should also be prefixed with `tentative_`. +4. Any tests directly or indirectly calling tentative testdriver methods must be marked tentative. + +## Alternatives considered +1. Avoid any changes to the WPT repository until the specification is finalised. + This makes it very difficult for browser vendors to collaborate and to learn about problems that are much easier to discover and understand "in practice". + In contrast, if the specification were developed without a working implementation, there is a much higher risk of fundamental design problems in the specification which are much harder to fix later. +2. Develop the implementation as vendor specific tests with methods in testdriver-vendor.js. + While this does allow a single vendor to iterate on a working implementation, it is still very difficult for multiple browser vendors to collaborate on this. + At the very least, other vendors may wish to contribute tests which exercise the implementation to ensure it fits their needs before agreeing to a final specification. + +## Risks +Methods could be added which never end up being specified, resulting in cruft and non-standardised functionality. +This is not a significant risk to the web at large because these methods only impact tests and the tests must be marked tentative, preventing them from being considered for Interop scoring, for example. From bbb248e5dcf0d713320945a723ee2a0814d35bdf Mon Sep 17 00:00:00 2001 From: James Teh Date: Thu, 3 Jul 2025 17:10:45 +1000 Subject: [PATCH 2/4] Add additional risks per review. --- rfcs/tentative_testdriver_methods.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/rfcs/tentative_testdriver_methods.md b/rfcs/tentative_testdriver_methods.md index c53c7a7..c80f650 100644 --- a/rfcs/tentative_testdriver_methods.md +++ b/rfcs/tentative_testdriver_methods.md @@ -25,5 +25,8 @@ To answer these questions, it would be helpful if browser vendors could collabor At the very least, other vendors may wish to contribute tests which exercise the implementation to ensure it fits their needs before agreeing to a final specification. ## Risks -Methods could be added which never end up being specified, resulting in cruft and non-standardised functionality. -This is not a significant risk to the web at large because these methods only impact tests and the tests must be marked tentative, preventing them from being considered for Interop scoring, for example. +1. Methods could be added which never end up being specified, resulting in cruft and non-standardised functionality. + This is not a significant risk to the web at large because these methods only impact tests and the tests must be marked tentative, preventing them from being considered for Interop scoring, for example. +2. The API could change significantly before it becomes finalized, with many tests depending on the tentative API. + This could mean that migrating the tests to the final API requires significant effort. +3. Because of the lack of specification, other vendors interested in ensuring interoperability with the feature being tested might have to reverse engineer how the tentative API is intended to function. From eaf29129a62555800cfee5ad6689b948b735193e Mon Sep 17 00:00:00 2001 From: James Teh Date: Wed, 23 Jul 2025 17:12:19 +1000 Subject: [PATCH 3/4] Cross-reference documentation on how to mark a test tentative. --- rfcs/tentative_testdriver_methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/tentative_testdriver_methods.md b/rfcs/tentative_testdriver_methods.md index c80f650..7592f45 100644 --- a/rfcs/tentative_testdriver_methods.md +++ b/rfcs/tentative_testdriver_methods.md @@ -14,7 +14,7 @@ To answer these questions, it would be helpful if browser vendors could collabor 1. There must be publicly documented (e.g. as part of an Interop Investigation Area) consensus between at least two browser vendors and intent to work towards a specification. 2. Any tentative methods added to testdriver.js should be prefixed with `tentative_`. 3. Any utility methods which call tentative testdriver methods should also be prefixed with `tentative_`. -4. Any tests directly or indirectly calling tentative testdriver methods must be marked tentative. +4. Any tests directly or indirectly calling tentative testdriver methods must be [marked tentative](https://web-platform-tests.org/writing-tests/file-names.html#:~:text=.tentative,-:%20). ## Alternatives considered 1. Avoid any changes to the WPT repository until the specification is finalised. From 58e569423b52c076704afc86ebf2a6f09d1a317c Mon Sep 17 00:00:00 2001 From: James Teh Date: Wed, 23 Jul 2025 17:21:49 +1000 Subject: [PATCH 4/4] Clarify that this is about WebDriver endpoints not defined in any specification, not just the WebDriver specifications. --- rfcs/tentative_testdriver_methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/tentative_testdriver_methods.md b/rfcs/tentative_testdriver_methods.md index 7592f45..d6d975d 100644 --- a/rfcs/tentative_testdriver_methods.md +++ b/rfcs/tentative_testdriver_methods.md @@ -1,7 +1,7 @@ # RFC 226: Tentative testdriver methods ## Summary -Allow tentative methods to be added to testdriver.js that are not yet included in the WebDriver or WebDriver BiDi specifications. +Allow tentative methods to be added to testdriver.js which use WebDriver endpoints not yet defined in a specification. This should only be allowed where there is consensus between multiple browser vendors and where there is clear progress towards a specification. ## Background