[feature] radr::find_common_end customisation point

Author: h-2

docs/caching_begin.md

@@ -71,12 +71,13 @@ We want to find out how many times the predicate is evaluated when using the ada
 | `for (size_t i : rad) {}` (2nd+)                           |  9          |
 | `for (size_t i : rad │ radr::take(100)) {}` (comb.)        |  9          |
 
-This tables illustrates the differences and similarities between the standard library and our library:
+This tables illustrates the similarities between the standard library and our library:
   * Constructing the adaptor and iterating over it once adds up to 15 predicate calls.
   * The second (and any further) iteration only performs 9 calls.
   * Note that since neither `std` nor `radr` cache the end, the *last* five 1s will always be parsed. *More on this below*.
 
-Finally, we see that the libraries differ when the adaptor is combined with another one, e.g. `take(100)`.
+They differ in that some of the work happens on construction in RADR.
+And finally, we see that the libraries differ when the adaptor is combined with another one, e.g. `take(100)`.
 The standard library adaptor resets its cache when being moved or copied, because the caching mechanism in the standard library is *non-propagating*.
 This is not known to most users of `std::ranges::`, and it is even more surprising since "building up" ranges pipelines in multiple steps is a frequently recommended practice.
 
@@ -95,11 +96,11 @@ In our library `radr::filter` is not common by default. This has the advantage t
 
 |  `radr::`                                                                    | # calls     |
 |------------------------------------------------------------------------------|------------:|
-| `auto rad = std::ref(vec) │ radr::filter(even) │ radr::to_common;` (constr.) |  15         |
+| `auto rad = std::ref(vec) │ radr::filter(even) │ radr::to_common;` (constr.) |  12         |
 | `for (size_t i : rad) {}` (1st)                                              |  4          |
 | `for (size_t i : rad) {}` (2nd+)                                             |  4          |
 | `for (size_t i : rad │ radr::take(100)) {}` (comb.)                          |  4          |
 
-As you see, the range is parsed once completely on construction.
+The head of the range is parsed on construction of `filter` (6 calls), and the tail is parsed on construction of `to_common` (reverse find, also 6 calls).
 After this, the adaptor needs to parse neither the head, nor the tail of the underlying range on a full iteration.
 The standard library does not offer such facilities.

docs/getting_started.md

@@ -45,6 +45,7 @@ We fundamentally differentiate three types of ranges:
 
 Multi-pass ranges can be accessed multiple times while single-pass ranges cannot generally be read again from the beginning.
 See [Range properties](./range_properties.md) for formal definitions.
+Many types and concepts in the library use the abbreviation `mp` for "multi-pass" or `sp` for "single-pass".
 
 A *range adaptor* is a range that is created on top of another range, typically referred to as the *underlying range* or
 the *original range*.

include/radr/custom/find_common_end.hpp

@@ -0,0 +1,86 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Copyright (c) 2023-2025 Hannes Hauswedell
+//
+// Licensed under the Apache License v2.0 with LLVM Exceptions.
+// See the LICENSE file for details.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#pragma once
+
+#include <iterator>
+
+#include "tags.hpp"
+
+namespace radr
+{
+
+//=============================================================================
+// Wrapper function find_common_end
+//=============================================================================
+
+struct find_common_end_impl_t
+{
+    //!\brief Call tag_invoke if possible; call default otherwise. [it, sen]
+    template <typename It, typename Sen>
+    constexpr It operator()(It b, Sen e) const
+    {
+        static_assert(std::forward_iterator<It>,
+                      "You must pass a forward_iterator as first argument to radr::find_common_end.");
+        static_assert(std::sentinel_for<Sen, It>,
+                      "You must pass a sentinel as second argument to radr::find_common_end.");
+        static_assert(!std::same_as<Sen, std::unreachable_sentinel_t>,
+                      "You must not pass infinite ranges to radr::find_common_end.");
+
+        if constexpr (std::same_as<It, Sen>)
+        {
+            return e;
+        }
+        else if constexpr (requires { tag_invoke(custom::find_common_end_tag{}, std::move(b), std::move(e)); })
+        {
+            using ret_t = decltype(tag_invoke(custom::find_common_end_tag{}, std::move(b), std::move(e)));
+            static_assert(std::same_as<ret_t, It>,
+                          "Your customisations of radr::find_common_end must always return the iterator type.");
+            return tag_invoke(custom::find_common_end_tag{}, std::move(b), std::move(e));
+        }
+        else
+        {
+            auto ret = b;
+            std::ranges::advance(ret, e);
+            return ret;
+        }
+    }
+
+    //!\}
+};
+
+/*!\brief Given an iterator-sentinel pair, return an iterator that can be used as the sentinel.
+ * \param[in] b The iterator.
+ * \param[in] e The sentinel.
+ * \returns An iterator that can be used with \p in to represent the same range as `(b, e)` but where iterator and sentinel have the same type.
+ * \pre `(b, e)` denotes a valid, finite range.
+ * \details
+ *
+ * NOOP if \p b and \p e already have the same type.
+ *
+ * ### Complexity
+ *
+ * The default implementation is **linear**; the end is searched from the beginning.
+ *
+ * ### Customisation
+ *
+ * You may provide overloads with the following signature to customise the behaviour:
+ *
+ * ```cpp
+ * It tag_invoke(radr::find_common_end_tag, It, Sen);
+ * ```
+ *
+ * They must be visible to ADL.
+ *
+ */
+inline constexpr find_common_end_impl_t find_common_end{};
+
+} // namespace radr

include/radr/custom/subborrow.hpp

@@ -209,6 +209,13 @@ struct subborrow_impl_t
         }
     }
 
+    //!\brief Call tag_invoke if possible (even without size); call default otherwise. [it, sen, not_size]
+    template <borrowed_mp_range URange, detail::is_iterator_of<URange> It, typename Sen>
+    constexpr auto operator()(URange && urange, It const b, Sen const e, detail::not_size const) const
+    {
+        return operator()(std::forward<URange>(urange), b, e);
+    }
+
     //!\brief Call tag_invoke if possible; call default otherwise. [i, j]
     template <borrowed_mp_range URange>
         requires(std::ranges::random_access_range<URange> && std::ranges::sized_range<URange>)

include/radr/custom/tags.hpp

@@ -14,10 +14,13 @@
 namespace radr::custom
 {
 
+struct rebind_iterator_tag
+{};
+
 struct subborrow_tag
 {};
 
-struct rebind_iterator_tag
+struct find_common_end_tag
 {};
 
 } // namespace radr::custom

include/radr/rad/filter.hpp

@@ -23,6 +23,7 @@
 #include "../detail/semiregular_box.hpp"
 #include "../generator.hpp"
 #include "../range_access.hpp"
+#include "radr/custom/tags.hpp"
 
 namespace radr::detail
 {
@@ -66,31 +67,78 @@ class filter_iterator
         using RIt     = filter_iterator<Iter, Iter, Func>;
         Iter new_uend = sen.base_iter();
 
-        if constexpr (std::bidirectional_iterator<Iter> && !std::same_as<Iter, Sent>)
+        RIt rit{std::move(it).func(), std::move(it).base_iter(), new_uend};
+        RIt rsen{std::move(sen).func(), new_uend, new_uend};
+        return borrowing_rad{std::move(rit), std::move(rsen), s};
+    }
+
+    /*!\brief Customisation for subranges.
+     * \details
+     *
+     * This potentially returns a range of different filter_iterators. In particular, the returned
+     * ones are always built on iterator-sentinel of the same type.
+     */
+    template <borrowed_mp_range R>
+    constexpr friend auto tag_invoke(custom::subborrow_tag, R &&, filter_iterator it, filter_iterator sen)
+    {
+        return subborrow_impl(it, sen, not_size{});
+    }
+
+    //!\overload
+    template <borrowed_mp_range R>
+    constexpr friend auto tag_invoke(custom::subborrow_tag,
+                                     R &&,
+                                     filter_iterator it,
+                                     filter_iterator sen,
+                                     size_t const    s)
+    {
+        return subborrow_impl(it, sen, s);
+    }
+
+    //!\brief Customisation to create common sentinel with actual underlying end.
+    constexpr friend filter_iterator tag_invoke(custom::find_common_end_tag,
+                                                filter_iterator it,
+                                                std::default_sentinel_t)
+    {
+        Iter ubeg = std::move(it).base_iter();
+        Sent uend = std::move(it).base_sent();
+        Func fn   = std::move(it).func();
+
+        Iter it_end{};
+
+        /* search backwards from end if possible */
+        if constexpr (std::bidirectional_iterator<Iter> && std::same_as<Iter, Sent>)
         {
-            if (it != sen)
+            it_end = uend;
+            while (it_end != ubeg)
             {
-                /* For filter iterators on non-common ranges, sen might not represent the "real"
-                 * underlying end, because the "caching mechanism" implemented in operator++
-                 * does not take effect.
-                 * For these ranges, we do the tiny hack of creating a "filter_iterator-on-common",
-                 * and then we decrement which will move onto the last valid underlying element.
-                 * The we increment that to get the real underlying past-the-end iterator.
-                 *
-                 * A cleaner solution would be to introduce a `to_common` customisation point.
-                 * This would allow avoiding the back and forth.
-                 * We might do this in the future!
-                 */
-                RIt rsen{sen.func(), new_uend, new_uend};
-                --rsen;
-                new_uend = std::move(rsen).base_iter();
-                ++new_uend;
+                --it_end;
+                if (std::invoke(std::cref(fn), *it_end))
+                {
+                    ++it_end;
+                    break;
+                }
             }
         }
+        /* search from beginning but store element behind last matching instead of underlying end */
+        else
+        {
+            bool empty = true;
 
-        RIt rit{std::move(it).func(), std::move(it).base_iter(), new_uend};
-        RIt rsen{std::move(sen).func(), new_uend, new_uend};
-        return borrowing_rad{std::move(rit), std::move(rsen), s};
+            for (auto it = ubeg; it != uend; ++it)
+            {
+                if (std::invoke(std::cref(fn), *it))
+                {
+                    it_end = it;
+                    empty  = false;
+                }
+            }
+
+            if (!empty)
+                ++it_end;
+        }
+
+        return {std::move(fn), std::move(it_end), std::move(uend)};
     }
 
 public:
@@ -126,17 +174,8 @@ class filter_iterator
 
     constexpr filter_iterator & operator++()
     {
-        if constexpr (std::same_as<Iter, Sent>)
-        {
-            if (auto tmp = std::ranges::find_if(++current_, end_, std::ref(*func_)); tmp != end_)
-                current_ = std::move(tmp);
-            else // cache the real underlying end
-                end_ = current_;
-        }
-        else
-        {
-            current_ = std::ranges::find_if(std::move(++current_), end_, std::ref(*func_));
-        }
+        current_ = std::ranges::find_if(std::move(++current_), end_, std::cref(*func_));
+
         return *this;
     }
 
@@ -187,29 +226,6 @@ class filter_iterator
     {
         std::ranges::iter_swap(x.current_, y.current_);
     }
-
-    /*!\brief Customisation for subranges.
-     * \details
-     *
-     * This potentially returns a range of different filter_iterators. In particular, the returned
-     * ones are always built on iterator-sentinel of the same type.
-     */
-    template <borrowed_mp_range R>
-    constexpr friend auto tag_invoke(custom::subborrow_tag, R &&, filter_iterator it, filter_iterator sen)
-    {
-        return subborrow_impl(it, sen, not_size{});
-    }
-
-    //!\overload
-    template <borrowed_mp_range R>
-    constexpr friend auto tag_invoke(custom::subborrow_tag,
-                                     R &&,
-                                     filter_iterator it,
-                                     filter_iterator sen,
-                                     size_t const    s)
-    {
-        return subborrow_impl(it, sen, s);
-    }
 };
 
 // iterator-based borrow
@@ -291,21 +307,29 @@ inline namespace cpo
  *   * radr::constant_range
  *
  * It does not preserve:
- *  * std::ranges::sized_range
- *
- * **In contrast to std::views::filter, it DOES NOT PRESERVE:**
+ *   * std::ranges::sized_range
  *   * radr::common_range
  *   * radr::mutable_range
  *
  * To prevent UB, the returned range is always a radr::constant_range.
  *
- * Use `radr::filter(Fn) | radr::to_common` to make the range common, and use
- * `radr::to_single_pass | radr::filter(Fn)`, create a mutable range (see below).
- *
  * Construction of the adaptor is in O(n), because the first matching element is searched and cached.
  *
  * Multiple nested filter adaptors are folded into one.
  *
+ * ### Notable differences to std::views::filter
+ *
+ * Like all our multi-pass adaptors (but unlike std::views::filter), this adaptor is const-iterable.
+ *
+ * Does not preserve radr::common_range. Use `… | radr::filter(Fn) | radr::to_common` if you want a common range.
+ * This has the added benefit that the real underlying end is used and the tail of the underlying range is not searched repeatedly.
+ * See [caching end](docs/caching_begin.md#caching-end) for more details.
+ *
+ * Does not preserve radr::mutable_range, i.e. it always returns radr::constant_range.
+ * This prevents undefined behaviour by accidentally changing values that invalidate the predicate.
+ * If you want a mutable range, use the single pass adaptor like this:
+ * `… | radr::to_single_pass | radr::filter(fn)`
+ *
  * ## Single-pass ranges
  *
  * Requirements:
@@ -315,6 +339,13 @@ inline namespace cpo
  * The single-pass version of this adaptor preserves mutability, i.e. it allows changes to the underlying range's elements.
  * It also allows (observable) changes in the predicate.
  *
+ * ### Notable differences to std::views::filter
+ *
+ * In C++26 the single-pass version of std::views::filter was made const-iterable (but not the multi-pass version).
+ * This is counter to any consistency in the range concepts.
+ *
+ * All our multi-pass adaptors are const-iterable, but non of our single-pass adaptors are.
+ *
  */
 inline constexpr auto filter = detail::pipe_with_args_fn{detail::filter_coro, detail::filter_borrow};
 } // namespace cpo