Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Peekable::until #75540

Closed
wants to merge 4 commits into from
Closed

Conversation

apelisse
Copy link

@apelisse apelisse commented Aug 14, 2020

Useful to parse since you can read from an iterator until a specific
predicate becomes true, without consuming the positive item.

Following up on #72310 (comment). I've also reused the same "issue", I know nothing about the process.

Useful to parse since you can read from an iterator until a specific
predicate becomes true, without consuming the positive item.
@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @joshtriplett (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 14, 2020
@apelisse
Copy link
Author

cc @jyn514

@@ -1687,6 +1687,110 @@ impl<I: Iterator> Peekable<I> {
{
self.next_if(|next| next == expected)
}

/// Creates an iterator that consumes elements until predicate is
/// true, without consuming the last matching element.
Copy link
Author

@apelisse apelisse Aug 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// true, without consuming the last matching element.
/// true, without consuming the last, matching element.

/// ```
/// [`skip_while`]: trait.Iterator.html#method.skip_while
#[unstable(feature = "peekable_next_if", issue = "72480")]
pub fn until<P: FnMut(&I::Item) -> bool>(&mut self, func: P) -> Until<'_, I, P> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't think be take_until? We should also link take_while here for a non-consuming version. This is what I am looking for.

peekable: &'a mut Peekable<I>,
flag: bool,
predicate: P,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
}
}

/// documentation for more.
///
/// [`until`]: trait.Peekable.html#until
/// [`Iterator`]: trait.Iterator.html
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cross doc link, we probably don't need this. @jyn514 would be happy if you remove this

/// closure on each element of the iterator, and consume elements
/// until it returns true.
///
/// After true is returned, until()'s job is over, and the iterator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// After true is returned, until()'s job is over, and the iterator
/// After `true` is returned, `until()`'s job is over, and the iterator

I: Iterator,
{
peekable: &'a mut Peekable<I>,
flag: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove flag? And use predicate: Option<P> to indicate that it can still be iterated on?

type Item = I::Item;

#[inline]
fn next(&mut self) -> Option<I::Item> {
Copy link
Contributor

@pickfire pickfire Aug 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip_while uses fn check most likely to optimized the assembly (size and cache), how does this assembly looks like? @lzutao do you think this will have an impact? SkipWhile

impl<I: Iterator, P> Iterator for SkipWhile<I, P>
where
    P: FnMut(&I::Item) -> bool,
{
    type Item = I::Item;

    #[inline]
    fn next(&mut self) -> Option<I::Item> {
        fn check<'a, T>(
            flag: &'a mut bool,
            pred: &'a mut impl FnMut(&T) -> bool,
        ) -> impl FnMut(&T) -> bool + 'a {
            move |x| {
                if *flag || !pred(x) {
                    *flag = true;
                    true
                } else {
                    false
                }
            }
        }

        let flag = &mut self.flag;
        let pred = &mut self.predicate;
        self.iter.find(check(flag, pred))
    }

fn size_hint(&self) -> (usize, Option<usize>) {
let (_, upper) = self.peekable.size_hint();
(0, upper) // can't know a lower bound, due to the predicate
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we override try_fold and fold too?

@pickfire
Copy link
Contributor

pickfire commented Aug 15, 2020

After working with iterators on fish-manpage-completions for quite a bit, I find this function lacking and we used the one from itertools peeking_take_while. I do think that this will introduce potential footgun for users, I hit quite a few myself too and it is hard to debug and fix, ideally we should write it down somewhere in the docs.

In case someone use peekable on a consuming iterator like SkipWhile, the original iterator will get consumed too, this took me quite a few hours to figure out the bug and fix it. An ASCII diagram to illustrate this

iterator -> | 1 2 3 4 5
iterator.skip_while (!= 2) -> 1 | 2 3 4 5
iterator.skip_while.peekable.take_until (== 3) -> 1 2 | 3 4 5 (note we only iterated 2 here)
The next time we iterate
iterator -> 1 2 | 3 4 5
What we wanted
iterator -> 1 | 2 3 4 5
What should be done
iterator.peekable.skip_while.take_until (not possible right now)

I feel confused myself now too, or maybe this should be discussed more in peekable. The issue I faced is fixed in rust-dc/fish-manpage-completions@efb321b

But the solution is as simple as moving peekable to iterator as early as possible to avoid any side effects.

@apelisse
Copy link
Author

Yeah, I didn't know about peeking_take_while before, and that actually solves my problem so I don't have as much interested in doing this. I'm happy to polish this PR if we want to get it merged, but I don't have a strong case to push it now.

@jyn514 jyn514 added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Aug 28, 2020
@jyn514
Copy link
Member

jyn514 commented Aug 28, 2020

Should this be closed? It sounds like @apelisse has lost interest and I don't know anyone else planning to use this API.

@apelisse
Copy link
Author

I'm happy to close, feel free to re-open if needed.

@apelisse apelisse closed this Aug 28, 2020
@pickfire
Copy link
Contributor

I do am interested in this since I need it but not that important to me to focus on this right now. It may create more footgun that I need to see into peeking.

@jhpratt
Copy link
Member

jhpratt commented Sep 3, 2020

FWIW I'd be interested in doing whatever changes the libs team desires. The lack of an API like this threw me off when I ran into it, and took a big of debug prints to figure out what was happening.

@jyn514
Copy link
Member

jyn514 commented Sep 3, 2020

@jhpratt sure, feel free to make a new PR with the changes from this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants