Your Digital Media Has Never Looked So Good

 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Channel Store "Sort By" broken!?

Sat Jul 18, 2015 4:31 pm

This is weird. Kind of hard to notice at first but seems the "Sort By' functionality of the web channel store is broken. Bear with me and follow the steps:
  1. Go to https://www.roku.com/channels#!browse/games/by-ratings (I am choosing sort by "highest rated" because is easier to spot the problem)
  2. What you see will look something like this (GandK Reversi is on top for me)
    sort working
    Image
  3. It only makes sense that the order in which they are sorted now will be monotonically decreasing (i.e. non-increasing) re the stars rating, yes? So we start at 4.5, then eventually go to the apps with 4 stars, then 3.5 and so on. Let's gingerly scroll down and make sure that is the case...
  4. Wait! What is this? Suddenly there is a jump in the stars, the sort is broken! (currently i see this at first between "Dance Party"=3.5, followed by "Monkey Pole Climb"=5)
    sort broken
    Image
  5. Then the order continues for a while in decreasing order and - BAM! - it happens again, this time from "Casino Slots" 2.5* it jumps to "Robreed - Flippin' Awesome"=5*, then starts going down.
  6. Next hiccup is from "Marble Puzzle"=3.5 jumping up to "Gem Girl"=4.5
  7. And one more, when from "GlobalTrade"=3.5 it goes up to "Treasures"=5
I think this (or that) picture visualizes how the channel rating goes down- it is more of a "saw-tooth"-shaped (dis)order rather than a single downward slope:
Image

PS. i notice the pattern 6666 in this - the sort reversal/singularity happens every 6 rows. Or maybe it's in batches of 24 channels (6 rows x 4 per row), though that doesn't sound as ominous. The point is that sort only works within small portions of the list and there is no way to see which is the best-rated item overall. I suspect the same is true for sorting by "most popular" but have no handy example to illustrate the dis-order there.
Last edited by EnTerr on Fri Aug 14, 2015 12:16 pm, edited 2 times in total.
 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Fri Jul 24, 2015 10:30 am

Roku guys -
would you kindly comment on this issue?

Am i getting something wrong - or do you recognize this as a bug?
Not only as a user - but as a developer whose app standing is affected by defects in your sort - i "have a horse in this race".
 
User avatar
BradC
Posts: 305
Joined: Sun Dec 16, 2012 1:50 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 3:44 pm

the channel store is a mess.

last time I brought this up, my top selling channels got suddenly moved to the very bottom of the list, even though 2 were on the top selling paid list at the time.


good luck. ;)
 
User avatar
RokuJoel
Posts: 1758
Joined: Mon Nov 14, 2011 5:22 pm

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 4:10 pm

As requested, chiming in here: yeah, we know it is kinda borked. Bugs have been filed on things like @BradC's loss of channel ranking. On the device, Game section ranking is now done manually based on a report run once per week, someday when the Game section ranking algorithm is repaired it will go back to automatic rolling 7 day popularity ranking.

- Joel
 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 4:28 pm

BradC wrote:
the channel store is a mess.
last time I brought this up, my top selling channels got suddenly moved to the very bottom of the list, even though 2 were on the top selling paid list at the time.

I agree it's a mess and RokuCo's web-dev team does not compare favorably to the core developers. I am looking for an English phrase and so far found the somewhat crass "[One's] mouth is writing checks [one's] ass can't cash". And what i mean by that is they re-designed the web site for "responsive design" in irresponsible manner. Someone is learning-on-the-job there and we the internet users of the world are their guinea pigs. Oh wait - here is another one "their eyes are bigger than they can stomach".

I remember you questioning the different order of channels in the web vs the player stores, as well as the variation between different player models - and i thought the explanation someone Roku* gave made sense, that each line of players uses their own stats, based on how downloads go for that model - where the web store uses aggregate stats.

But i doubt the Co would have maliciously downgraded your channels, it seems out of character. HOWEVER, this is why it is important the rating/sorting to be fair and transparent - so that good, prolific developers like you @BradC do not get alienated!

Back from the general to the specific: so yeah, when i see that sort-by-rating does not work, that's depressing. Even if i were to get 5-out-of-5 rating, i'd still be at the end of the list.
 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 4:52 pm

RokuJoel wrote:
As requested, chiming in here: yeah, we know it is kinda borked. Bugs have been filed on things like @BradC's loss of channel ranking. On the device, Game section ranking is now done manually based on a report run once per week, someday when the Game section ranking algorithm is repaired it will go back to automatic rolling 7 day popularity ranking.

Thank you RokuJoel - i think this is quite helpful lessening anxiety!
So the popularity ranking is done by a recent performance (be it rolling or fixed weekly term) and not since the Big Bang. That may explain what BradC observed.

Is it the #new-installs or the #times-the-game-was-started in that period, the one that counts?
 
User avatar
BradC
Posts: 305
Joined: Sun Dec 16, 2012 1:50 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 4:53 pm

EnTerr wrote:
i doubt the Co would have maliciously downgraded your channels


nah, I certainly don't think that's what happened. I can be annoying but not THAT bad. :) My intended point was just that bringing it up didn't help anything, and it's been a pretty big frustration for me.
 
User avatar
RokuJoel
Posts: 1758
Joined: Mon Nov 14, 2011 5:22 pm

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 4:55 pm

I don't think ratings is just the number of stars, but also the number of people who have the channel installed who selected those stars, and may also include the number of active users of those channels vs inactive i.e. has the channel been launched in the past 7 days.

Also, the web store uses a different algorithm from the Device Channel Store, and my comment about BradC's channels was specifically about the device store. The web store, well... there is a new release of the Web Channel Store going out some time tomorrow, so let us know if it improves this situation or not. I'll also point the web team to this thread.


- Joel
 
User avatar
RokuJoel
Posts: 1758
Joined: Mon Nov 14, 2011 5:22 pm

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 5:05 pm

EnTerr wrote:
So the popularity ranking is done by a recent performance (be it rolling or fixed weekly term) and not since the Big Bang. That may explain what BradC observed.


Yes, the ranking has always been on a 7 day rolling schedule. What happened to BradC and a few other game developers is why we moved this category from automatic to manual control. As it didn't appear the problem would be fixed in a short period of time, the business team decided to take over managing this until the issue gets fixed. The current positions of channels in the Games section are now manually adjusted on a non-rolling 1 week schedule.

- Joel
 
User avatar
BradC
Posts: 305
Joined: Sun Dec 16, 2012 1:50 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 5:37 pm

thanks Joel.
♦MiniGolf♦HangMan♦Brain Puck♦Retro Tennis♦BORK♦FLIP♦Pathogen♦Pathogen 2♦Shut the Box♦Birdie♦Logic♦Dots♦Pool♦küglo♦Bubble Wrap♦Trivia Channel♦Mancala♦Air Hockey♦Weather♦CAMERA♦Your Photos Screensaver♦Desert Beauty Screensaver♦Wild Lakes Screensaver♦
 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Tue Jul 28, 2015 9:52 pm

RokuJoel wrote:
Also, the web store uses a different algorithm from the Device Channel Store [...] The web store, well... there is a new release of the Web Channel Store going out some time tomorrow, so let us know if it improves this situation or not. I'll also point the web team to this thread.

After some meditation, i think the problem has something to do with channels being loaded in batches of 24 (6 rows x 4)!

Initially there are 12 rows of channels (48) but as you scroll further, it adds 6 more and then 6 more, and then the last batch. And the breaks in "highest rated" sorting happen after every 6 rows. Pure coincidence? I think not - rather the sort is done correctly only inside each new batch...

I'll keep my eyes peeled for that new version for sure.
 
EnTerr
** Valued Community Member **
Topic Author
Posts: 3834
Joined: Sun Jan 02, 2011 2:41 am

Re: Channel Store "Sort By" is broken!? (Sawtooth syndrome)

Fri Aug 14, 2015 4:21 pm

RokuJoel wrote:
... there is a new release of the Web Channel Store going out some time tomorrow, so let us know if it improves this situation or not. I'll also point the web team to this thread.

Seems the new version (btw, channel store got new URL this time: http://channelstore.roku.com) has fixed something, because the next day my app jumped from near the end of the "Highest Rated" sort to its "rightful" place in the 1st quartile (around #23). And reviewing the whole list now, the "sawtooth" jump every 24 is not there.

However... there is something fishy with the default sort "Most Popular", https://channelstore.roku.com/browse/games/popularity

For example, comparing web store to a Roku 3 store, "Games" category:
  • How come "Mazes", which is #5 on the player, is #105 (4th last) on the web? In what world the paid version "Mazes Plus" (#87 on the web), which gets ballpark 50x less downloads than the free "Mazes" is "more popular" than the free one? Because that's what the web list shows.
  • How is it that "Red Card Rampage", "Tangrams" and "World Famous Places Puzzle" always stay at the end of the web list (positions 106, 107, 108)? Where in the player list they are at #34, 70, 69.
  • How come FlipFlop is #10 on web, #65 on roku 3? That does not make sense - should be the opposite because FlipFlop is OpenGL game and would be expected to be higher on Roku 3 but lower in the totals.
  • "2048 TV" and "2048 Challenge" are #52 and 53 on a Roku3 but #60 and #35 on the web. Which does not make sense, because 2048-Challenge is Marmalade and compatible with only some of the players. Confirmed by the fact that 2048-TV has 4x the number of ratings and therefore approximately 4x more installs. Both are paid at $1. How is it then that 2048-Challenge is #35 on the web?!

Who is online

Users browsing this forum: No registered users and 8 guests