shuffling revisited

[Johannes Weißl]

Hello,

I took a better look on the shuffling issue, and I think the situation
could be improved this way:

1. A new flag "played" for shuffle_track. The flag is set once a track is
chosen *automatically* from the shuffle list. A track with the "played" flag
set is never chosen for the next song (but it can be selected manually!).
On reshuffle (through :shuffle or when auto_reshuffle is triggered), all
"played" flags are set to zero.

2. Two new flags "played_album" and "played_artist", which become active if
aaa_mode changes. This makes it possible to e.g. listen randomly through all
songs of an album, without loosing all the other "played" flags (which is good,
otherwise there is the probability that recently played tracks are selected).

This makes it possible to fix this bug [1] and to have this feature [2].

[1] https://sourceforge.net/mailarchive/message.php?msg_id=26625224
[2] https://sourceforge.net/mailarchive/message.php?msg_id=19460317


What do you think?



Johannes

[Johannes Weißl]

Hello,

test branch is here:
https://gitorious.org/~jmuc/cmus/jw-cmus/commits/shuffle

The current behaviour is this:
- If a song is selected in shuffle mode (manually or automatically), it
is never selected automatically again until all songs have been played
- Separate shuffle for each aaa_mode (all/artist/album)

Please test and/or post opinions!

[Jason Woofenden]

Post by Johannes Weißl
Hello,
https://gitorious.org/~jmuc/cmus/jw-cmus/commits/shuffle
- If a song is selected in shuffle mode (manually or automatically), it
is never selected automatically again until all songs have been played
- Separate shuffle for each aaa_mode (all/artist/album)
Please test and/or post opinions!

This seems to work when the end of track is reached, but not when I
do player-next.

Set-up steps:

1) I select a track

2) it plays on random for a few tracks

3) I select the same track in step 1 again

Then either:

4-working) I let it reach the end of the track (it plays a random
track

or

4-broken) I hit player-next before the track finishes. This plays
the same track again that was played after this track the first
time in step 1.


This is the use case I was hoping would be improved with new
shuffle code. I do this fairly frequently. I usually have it on
random, and I often have a new favorite song or two that I play
regularly throughout the day/week. This unfortunately (unless I
call :shuffle manually) results in the same sequence of tracks
playing repeatedly.

If I just hit player-prev then I'd expect player-next to get me
back to the same track, but after selecting a track manually (with
the enter key) player-next should not follow the same shuffle order
that was used after that track last time, it should select a random
(not yet played) track.


I hope that made sense. Please let me know if it didn't and I'll
write up an example.

Take care, - Jason

[Johannes Weißl]

Hi Jason,

thanks for testing!

Post by Jason Woofenden
This seems to work when the end of track is reached, but not when I
do player-next.

Yes, this was deliberately. I thought player-next (and player-prev)
should always be deterministic...

Post by Jason Woofenden
This is the use case I was hoping would be improved with new
shuffle code. I do this fairly frequently. I usually have it on
random, and I often have a new favorite song or two that I play
regularly throughout the day/week. This unfortunately (unless I
call :shuffle manually) results in the same sequence of tracks
playing repeatedly.

Yes, this is annoying and should be finally fixed now :-)!

Post by Jason Woofenden
If I just hit player-prev then I'd expect player-next to get me
back to the same track, but after selecting a track manually (with
the enter key) player-next should not follow the same shuffle order
that was used after that track last time, it should select a random
(not yet played) track.

Hmm, I guess this makes sense! I have a new logic that fixes this:
- player-prev increases an integer (i++) every time it is called
- if called manually, player-next will only deterministically go forward
i steps (i-- afterwards)
- manual track selection resets i (i=0)

Is that right?

I pushed a patch to the branch so you can check it out!


Johannes

[Johannes Weißl]

Hi everyone,

Post by Johannes Weißl
- player-prev increases an integer (i++) every time it is called
- if called manually, player-next will only deterministically go forward
i steps (i-- afterwards)
- manual track selection resets i (i=0)
Is that right?
I pushed a patch to the branch so you can check it out!

Has anyone tested this or maybe can test this? It is in my "next" or
"shuffle" branch for some months now:
git clone -b shuffle git://gitorious.org/~jmuc/cmus/jw-cmus.git

If there is positive feedback for the strategy I will fine tune the
patch and send it, so this can be (finally!) fixed :-).


Johannes

[Jason Woofenden]

Post by Johannes Weißl
Hi everyone,

Post by Johannes Weißl
- player-prev increases an integer (i++) every time it is called
- if called manually, player-next will only deterministically go forward
i steps (i-- afterwards)
- manual track selection resets i (i=0)
Is that right?
I pushed a patch to the branch so you can check it out!

Has anyone tested this or maybe can test this? It is in my "next" or
git clone -b shuffle git://gitorious.org/~jmuc/cmus/jw-cmus.git

Oy, sorry I didn't do this months ago.

I just fiddled with it for a while. It's definitely an improvement!

Thanks again for all your work!

What I particularly like: (in shuffle mode) after selecting
manually (with the enter key) a song that has played recently, what
follows that song is randomly selected (and is not the song that
played after that track last time.

What could be further improved: player-prev 1) after manually
selecting a track 2) after the manually selected track ends and the
next track is playing.

I couldn't really figure out the pattern with #2 above. Sometimes
it went to the manually selected track, sometimes it took going
back a few more times to get to the manually selected track.

player-prev while the manually playing track is playing doesn't
ever seem to go back to the track that was playing when I manually
selected. Seems to me it should.



I'm just speaking here from my expectations and reactions as a
user. I'm not thinking about how it's implemented.

As a user, I expect to be able to use player-prev to get to
everything that has player recently, sorted chronologically.

Much like browsers work actually. (And largely like undo history in most
programs.) I think the browser parallel is more useful though,
because you can manually go to a page (type it's address or hit a
bookmark or something) and this adds it to the end of the history,
even if it's there earlier in the history. And if I use the back
button (or player-prev in cmus) and then manually select a track,
I'm fine with the "future part" of the history disappearing (as it
does with undo histories and browsers.)

(by "future part" I mean the part of the history that was
accessible with player-next or the forward button in browsers or
redo in editors.)



OK, now putting on my geek hat... and guessing about
implementation. I've got the impression that the shuffle is
achieved by essentially having a hidden playlist with all the
tracks in it, (and in which the order has been randomized.) And I
therefor presume that the player-prev/player-next functions just
step through this hidden playlist. A near-enough implementation of
what I'm looking for could be achieved by just re-ordering this
playlist slightly when a track in the list is manually selected:

The manually selected track could me moved so that it follows the
currently playing track in the hidden playlist.

that would work pretty well for the use case where you keep
selecting a track to play periodically while it's on random. But...
I'm thinking it would not be ideal if you hit player-prev a few
times, then manually selected a track (because your manually
selected track would be followed by the ones you player-preved
past. So maybe the manually selected track should be moved just
past the track which is furthest forward in the prev/next history.
In other words, the "next"-most track, from which you went
backwards with player-prev.

Hope I'm making sense there, and that I'm not totally wrong about
how the shuffle stuff works.


I can't believe this program keeps getting better. It's already so
good!

- Jason

[Cyrus Vafadari]

Can I unsubscribe

Post by Jason Woofenden

Post by Johannes Weißl
Hi everyone,

Post by Johannes Weißl
- player-prev increases an integer (i++) every time it is called
- if called manually, player-next will only deterministically go forward
i steps (i-- afterwards)
- manual track selection resets i (i=0)
Is that right?
I pushed a patch to the branch so you can check it out!

Has anyone tested this or maybe can test this? It is in my "next" or
git clone -b shuffle git://gitorious.org/~jmuc/cmus/jw-cmus.git

Oy, sorry I didn't do this months ago.
I just fiddled with it for a while. It's definitely an improvement!
Thanks again for all your work!
What I particularly like: (in shuffle mode) after selecting
manually (with the enter key) a song that has played recently, what
follows that song is randomly selected (and is not the song that
played after that track last time.
What could be further improved: player-prev 1) after manually
selecting a track 2) after the manually selected track ends and the
next track is playing.
I couldn't really figure out the pattern with #2 above. Sometimes
it went to the manually selected track, sometimes it took going
back a few more times to get to the manually selected track.
player-prev while the manually playing track is playing doesn't
ever seem to go back to the track that was playing when I manually
selected. Seems to me it should.
I'm just speaking here from my expectations and reactions as a
user. I'm not thinking about how it's implemented.
As a user, I expect to be able to use player-prev to get to
everything that has player recently, sorted chronologically.
Much like browsers work actually. (And largely like undo history in most
programs.) I think the browser parallel is more useful though,
because you can manually go to a page (type it's address or hit a
bookmark or something) and this adds it to the end of the history,
even if it's there earlier in the history. And if I use the back
button (or player-prev in cmus) and then manually select a track,
I'm fine with the "future part" of the history disappearing (as it
does with undo histories and browsers.)
(by "future part" I mean the part of the history that was
accessible with player-next or the forward button in browsers or
redo in editors.)
OK, now putting on my geek hat... and guessing about
implementation. I've got the impression that the shuffle is
achieved by essentially having a hidden playlist with all the
tracks in it, (and in which the order has been randomized.) And I
therefor presume that the player-prev/player-next functions just
step through this hidden playlist. A near-enough implementation of
what I'm looking for could be achieved by just re-ordering this
The manually selected track could me moved so that it follows the
currently playing track in the hidden playlist.
that would work pretty well for the use case where you keep
selecting a track to play periodically while it's on random. But...
I'm thinking it would not be ideal if you hit player-prev a few
times, then manually selected a track (because your manually
selected track would be followed by the ones you player-preved
past. So maybe the manually selected track should be moved just
past the track which is furthest forward in the prev/next history.
In other words, the "next"-most track, from which you went
backwards with player-prev.
Hope I'm making sense there, and that I'm not totally wrong about
how the shuffle stuff works.
I can't believe this program keeps getting better. It's already so
good!
- Jason

[Jason Woofenden]

Post by Cyrus Vafadari
Can I unsubscribe

'Course you can. The link is hidden in the headers of every e-mail
to the list (as with most e-mail lists). Here's a more visible link:

https://lists.sourceforge.net/lists/listinfo/cmus-devel

[Johannes Weißl]

Hi Jason,

Post by Jason Woofenden
What I particularly like: (in shuffle mode) after selecting
manually (with the enter key) a song that has played recently, what
follows that song is randomly selected (and is not the song that
played after that track last time.

Thanks for testing! Yes, every track has a flag "played" now, and if a
song got selected randomly it is not selected again until all other
songs are played.

Post by Jason Woofenden
player-prev while the manually playing track is playing doesn't
ever seem to go back to the track that was playing when I manually
selected. Seems to me it should.

Yes, this is the current behaviour, although I have to admit it doesn't
make to much sense...

Post by Jason Woofenden
As a user, I expect to be able to use player-prev to get to
everything that has player recently, sorted chronologically.

Yes, I think you are right... it should probably behave like undo/redo
in browsers or editors.

Post by Jason Woofenden
OK, now putting on my geek hat... and guessing about
implementation. I've got the impression that the shuffle is
achieved by essentially having a hidden playlist with all the
tracks in it, (and in which the order has been randomized.) And I
therefor presume that the player-prev/player-next functions just
step through this hidden playlist.

Correct!

Post by Jason Woofenden
A near-enough implementation of what I'm looking for could be achieved
by just re-ordering this playlist slightly when a track in the list is
The manually selected track could me moved so that it follows the
currently playing track in the hidden playlist.

I also had that idea... I think it would be possible, I can't remember
why I didn't implement this... maybe I thought it was messier than
introducing the "played" flags.

Post by Jason Woofenden
that would work pretty well for the use case where you keep
selecting a track to play periodically while it's on random. But...
I'm thinking it would not be ideal if you hit player-prev a few
times, then manually selected a track (because your manually
selected track would be followed by the ones you player-preved
past. So maybe the manually selected track should be moved just
past the track which is furthest forward in the prev/next history.
In other words, the "next"-most track, from which you went
backwards with player-prev.

Yes, I think that makes sense! I guess this should also happen when
the next track is selected automatically? E.g. if you go back three
steps, listen to the song "X" until the end, the next song is the track
after the "next"-most track (from which you went backwards).

So you can say a new track selection "snaps" you out of the undo/redo
search to the first "unknown" (= future) track.

Or should the history be rewritten then (because if you hit player-prev
this time, you would maybe expect "X" and not the same three tracks as
before)?

Post by Jason Woofenden
Hope I'm making sense there, and that I'm not totally wrong about
how the shuffle stuff works.

Great suggestions actually, maybe the next shuffle prototype will be
even better :-). Thanks a lot!

Post by Jason Woofenden
I can't believe this program keeps getting better. It's already so
good!

Hehe, we are nowhere near finished!


Johannes

[Paul Kramer]

Hi,

Post by Johannes Weißl
Thanks for testing! Yes, every track has a flag "played" now, and if a
song got selected randomly it is not selected again until all other
songs are played.

I am wondering what happens internally when Shuffle is turned off and
then back on again. Is the "played" flag removed? I'm not really sure
which behavior would be best here. On the one hand, if you accidentally
hit the "s"-key and turn shuffle off, it would be nicer, if the flag
stayed. On the other hand, if you turn shuffle off for a longer time and
then back on, the preferred behavior would probably a "reset" of the flag.

Also: What happens when during play a filter is activated or changed?
What happens when you switch from "play playlist" to "play
[artist,album,all] from library"?

PS: I love this change. It makes shuffling so much more intuitive.
Thanks! =)

Cheers,
Paul

Post by Johannes Weißl

Post by Jason Woofenden
player-prev while the manually playing track is playing doesn't
ever seem to go back to the track that was playing when I manually
selected. Seems to me it should.

Yes, this is the current behaviour, although I have to admit it doesn't
make to much sense...

Post by Jason Woofenden
As a user, I expect to be able to use player-prev to get to
everything that has player recently, sorted chronologically.

Yes, I think you are right... it should probably behave like undo/redo
in browsers or editors.

Post by Jason Woofenden
OK, now putting on my geek hat... and guessing about
implementation. I've got the impression that the shuffle is
achieved by essentially having a hidden playlist with all the
tracks in it, (and in which the order has been randomized.) And I
therefor presume that the player-prev/player-next functions just
step through this hidden playlist.

Correct!

Post by Jason Woofenden
A near-enough implementation of what I'm looking for could be achieved
by just re-ordering this playlist slightly when a track in the list is
The manually selected track could me moved so that it follows the
currently playing track in the hidden playlist.

I also had that idea... I think it would be possible, I can't remember
why I didn't implement this... maybe I thought it was messier than
introducing the "played" flags.

Post by Jason Woofenden
that would work pretty well for the use case where you keep
selecting a track to play periodically while it's on random. But...
I'm thinking it would not be ideal if you hit player-prev a few
times, then manually selected a track (because your manually
selected track would be followed by the ones you player-preved
past. So maybe the manually selected track should be moved just
past the track which is furthest forward in the prev/next history.
In other words, the "next"-most track, from which you went
backwards with player-prev.

Yes, I think that makes sense! I guess this should also happen when
the next track is selected automatically? E.g. if you go back three
steps, listen to the song "X" until the end, the next song is the track
after the "next"-most track (from which you went backwards).
So you can say a new track selection "snaps" you out of the undo/redo
search to the first "unknown" (= future) track.
Or should the history be rewritten then (because if you hit player-prev
this time, you would maybe expect "X" and not the same three tracks as
before)?

Post by Jason Woofenden
Hope I'm making sense there, and that I'm not totally wrong about
how the shuffle stuff works.

Great suggestions actually, maybe the next shuffle prototype will be
even better :-). Thanks a lot!

Post by Jason Woofenden
I can't believe this program keeps getting better. It's already so
good!

Hehe, we are nowhere near finished!
Johannes
------------------------------------------------------------------------------
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev

[Johannes Weißl]

Hi Paul,

Post by Paul Kramer

Post by Johannes Weißl
Thanks for testing! Yes, every track has a flag "played" now, and if a
song got selected randomly it is not selected again until all other
songs are played.

I am wondering what happens internally when Shuffle is turned off and
then back on again. Is the "played" flag removed? I'm not really sure
which behavior would be best here. On the one hand, if you accidentally
hit the "s"-key and turn shuffle off, it would be nicer, if the flag
stayed. On the other hand, if you turn shuffle off for a longer time and
then back on, the preferred behavior would probably a "reset" of the flag.

Yes, difficult case. Currently, the played-flags are not cleared if
shuffle is turned on or off, you have to use the :shuffle command for
that!

Post by Paul Kramer
Also: What happens when during play a filter is activated or changed?
What happens when you switch from "play playlist" to "play
[artist,album,all] from library"?

I fear that filters screw this up, because the way filters work now,
tracks are really removed from the library, and added again if the
filter is cleared. This is definitely not what we want!

The "played" flag is designed as bitmask, so there are separate states
for artist/album/all!

Post by Paul Kramer
PS: I love this change. It makes shuffling so much more intuitive.
Thanks! =)

Thanks! I don't really know what to do about this filtering issue
though... filtering always randomizes the hidden shuffle playlist. Maybe
we need to introduce a "hidden" (= filtered) flag for every track, what
do you/others think about that? This was already suggested by Timo in
his "TODO" mail shortly before he left.

I tried not to do that, because it means many parts of the program have
to be adapted (many !track.hidden checks).


Johannes

[Paul Kramer]

Hi Johannes,

Post by Johannes Weißl
Hi Paul,

Post by Paul Kramer

Post by Johannes Weißl
Thanks for testing! Yes, every track has a flag "played" now, and if a
song got selected randomly it is not selected again until all other
songs are played.

I am wondering what happens internally when Shuffle is turned off and
then back on again. Is the "played" flag removed? I'm not really sure
which behavior would be best here. On the one hand, if you accidentally
hit the "s"-key and turn shuffle off, it would be nicer, if the flag
stayed. On the other hand, if you turn shuffle off for a longer time and
then back on, the preferred behavior would probably a "reset" of the flag.

Yes, difficult case. Currently, the played-flags are not cleared if
shuffle is turned on or off, you have to use the :shuffle command for
that!

I think it is fine, if you can use the :shuffle command (I didn't know
there was such a thing).

Post by Johannes Weißl

Post by Paul Kramer
Also: What happens when during play a filter is activated or changed?
What happens when you switch from "play playlist" to "play
[artist,album,all] from library"?

I fear that filters screw this up, because the way filters work now,
tracks are really removed from the library, and added again if the
filter is cleared. This is definitely not what we want!
The "played" flag is designed as bitmask, so there are separate states
for artist/album/all!

Post by Paul Kramer
PS: I love this change. It makes shuffling so much more intuitive.
Thanks! =)

Thanks! I don't really know what to do about this filtering issue
though... filtering always randomizes the hidden shuffle playlist. Maybe
we need to introduce a "hidden" (= filtered) flag for every track, what
do you/others think about that? This was already suggested by Timo in
his "TODO" mail shortly before he left.

This sounds like a good, clean idea, but the programming overhead might
be too big (I am thinking of new features where you will always have to
include the !track.hidden check). But I cannot think of another way. The
procedure of deleting and adding the songs again sounds like much
trouble; I am very surprised, that it still works so fast (even with
large libraries).

Post by Johannes Weißl
I tried not to do that, because it means many parts of the program have
to be adapted (many !track.hidden checks).
Johannes
------------------------------------------------------------------------------
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev

Paul

[Johannes Weißl]

Hi Paul,

Post by Paul Kramer

Post by Johannes Weißl
Thanks! I don't really know what to do about this filtering issue
though... filtering always randomizes the hidden shuffle playlist. Maybe
we need to introduce a "hidden" (= filtered) flag for every track, what
do you/others think about that? This was already suggested by Timo in
his "TODO" mail shortly before he left.

This sounds like a good, clean idea, but the programming overhead might
be too big (I am thinking of new features where you will always have to
include the !track.hidden check). But I cannot think of another way. The
procedure of deleting and adding the songs again sounds like much
trouble; I am very surprised, that it still works so fast (even with
large libraries).

Well, it can maybe be encapsulated in a function. Well, the whole rewrite
with rbtrees was to make filtering faster, so that live filtering works
even though the tracks are really removed/added instead of just hidden.

Maybe the best would be a second tree for the filtered library, because
otherwise you maybe have to do thousands of ->next checks to find the
next unhidden track...


Johannes

[Paul Kramer]

Hello,

you are my hero for attacking this problem! Thanks.

Regards,
Paul

Post by Johannes Weißl
Hello,
https://gitorious.org/~jmuc/cmus/jw-cmus/commits/shuffle
- If a song is selected in shuffle mode (manually or automatically), it
is never selected automatically again until all songs have been played
- Separate shuffle for each aaa_mode (all/artist/album)
Please test and/or post opinions!
------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay