Your Digital Media Has Never Looked So Good

 
jamesonf
Topic Author
Posts: 4
Joined: Mon Jul 31, 2017 2:36 pm
Contact:

Firmware 7.7.0 Video Crashes

Mon Jul 31, 2017 2:45 pm

We have been having a lot of issues with our channel (CHIVE TV) crashing on many different devices at many different locations. We believe the issues have started on 07/21, and we also believe that the crashes have something to do with the firmware update that was released 06/20. We checked the BrightScript Crash Logs and haven’t found any recent crashes.

We tested the channel using a Roku 3600x, Roku 2, and a Roku 3.  Throughout our tests the video player plays through the content just fine, then randomly we get a garbled up screen with lots of colors (see attached pictures). After some time the screen will go black and remote control input buttons do not respond. Eventually the Roku screensaver will kick in, but we must restart the Roku for it to operate normally. All of the crashed devices are on the newest 7.7.0 firmware build 4116. From our experience, the video crash occurs at different times and on different video segments. We've attached a few different screenshots here.

When I sideloaded the channel I was able to view the console as it crashed. The video component reported an unknown event, but it was still reporting to our Google Analytics. By “unknown” event - it was not a documented event within here: https://sdkdocs.roku.com/display/sdkdoc/roVideoPlayerEvent

It’s important to note that we have not changed anything on our side. No new release, no changes to our video streams, no changes to the backend code (our API).

Has anybody else had issues with the latest 7.7.0 firmware build 4116?  Any help would be greatly appreciated as this issue is very urgent for us.

ImageImageImageImage
 
User avatar
Tyler Smith
Posts: 108
Joined: Thu Apr 14, 2016 10:51 am
Location: Nova Scotia, Canada
Contact:

Re: Firmware 7.7.0 Video Crashes

Thu Aug 03, 2017 4:41 am

I have not seen this issue in our apps using the Scene Graph Video component.
We test on the Roku 2, 3, and premiere models.
Tyler Smith
Senior Developer, REDspace
redspace.com
 
User avatar
RokuKC
Posts: 274
Joined: Wed Sep 10, 2014 10:44 am
Location: Roku HQ

Re: Firmware 7.7.0 Video Crashes

Thu Aug 03, 2017 3:41 pm

jamesonf wrote:
...
All of the crashed devices are on the newest 7.7.0 firmware build 4116. From our experience, the video crash occurs at different times and on different video segments. We've attached a few different screenshots here
...


Are you still reproducing this issue?

If so, can you provide the model number and/or serial number of some device(s) that are showing the problem?
If you can also (or instead) provide a media stream URL that you think reproduces the issue, that would be most helpful.
 
Bacon
Posts: 383
Joined: Fri Nov 01, 2013 6:48 am

Re: Firmware 7.7.0 Video Crashes

Thu Aug 03, 2017 7:01 pm

Issue is happening with Giantbomb's beta RSG app too.

https://www.giantbomb.com/forums/bug-re ... ge-8655810
 
jamesonf
Topic Author
Posts: 4
Joined: Mon Jul 31, 2017 2:36 pm
Contact:

Re: Firmware 7.7.0 Video Crashes

Thu Aug 03, 2017 7:34 pm

We are still crashing our devices.  We got in contact with Roku Partner Success and they said they have had multiple reports with the same issue.  From what it sounded like, it was crashing devices that had a serial number beginning with "5S" or "5F."  They said there was a new update out, so I'm upgrading our devices from 4116 to 4119 in hopes that this will fix it.
 
User avatar
RokuNB
Posts: 306
Joined: Fri Mar 31, 2017 2:22 pm

Re: Firmware 7.7.0 Video Crashes

Fri Aug 04, 2017 12:04 am

jamesonf wrote:
We are still crashing our devices.  We got in contact with Roku Partner Success and they said they have had multiple reports with the same issue.  From what it sounded like, it was crashing devices that had a serial number beginning with "5S" or "5F."  They said there was a new update out, so I'm upgrading our devices from 4116 to 4119 in hopes that this will fix it.

Let me dispel some confusion - the source of "5S" or "5F" prefixes was the list of device S/N that you (Chive) provided - then you were asked if there are other kinds. It's not clear yet if there are multiple reports - yours is the only solid lead, another one may be unrelated. I don't think anybody on our side has been able to reproduce the issue, so exact steps to reproduce will be and helpful?
 
jamesonf
Topic Author
Posts: 4
Joined: Mon Jul 31, 2017 2:36 pm
Contact:

Re: Firmware 7.7.0 Video Crashes

Fri Aug 04, 2017 1:01 am

Well drats, I had assumed the serial prefix was a lead for the overall issue and not just related to us.

We can't get this to reproduce 100% of the time. This is what I normally do:
  1. Launch CHIVETV 1.1 - Build 7
  2. Goto Party Mode
  3. Select "CHIVETV" (first and only option)
  4. Let video play

We've had some customers report that the video crashes about 7 minutes in.  I, on the other hand, don't crash until 90 - 120 minutes in. Most of the time I crash on different video segments too.  We have re-uploaded the videos to our video platform to make sure it wasn't some random video error.

Here are a few things we know:
  • Most of our devices are running 3500x, 3600x, or 4200x
  • Some of the crashed devices have crashed with the Roku directly plugged into HDMI ports as well as plugged into HDMI splitters
  • Crashes have occurred with 1080 and 720
  • We've re-uploaded videos to ensure codecs via our video platform was not an issue
  • We force 720 quality (even if you're in 1080) with H.264 at 2500 bitrate

A few days ago I enabled some debug output and sideloaded the channel. The results were interesting:
03:41:29 : video position:  358 /  436
03:41:30 : video position:  359 /  436
03:41:31 : video position:  360 /  436
03:41:32 : video position:  361 /  436
03:41:33 : video position:  362 /  436
03:41:34 : video position:  363 /  436
03:41:35 : video position:  364 /  436
03:41:36 : video position:  365 /  436
03:41:37 : video position:  366 /  436
03:41:38 : video position:  367 /  436
03:41:39 : video position:  368 /  436
03:41:40 : video position:  369 /  436
[**] 03:41:40 : other unknown event
[***] 03:41:40 : message: 
[***] 03:41:40 : index:  0
03:41:40 : Stream Started
03:41:40 : Status Message - startup progress
03:41:40 : video buffer progress (video):  99.9
03:41:40 : Status Message - startup progress
03:41:40 : video buffer progress (video):  99.9
03:41:42 : Status Message - startup progress
03:41:42 : video buffer progress (video):  100
03:41:42 : Status Message - start of play
03:41:42 : video position:  371 /  436
03:41:46 : video position:  375 /  436
03:41:48 : video position:  376 /  436
03:42:01 : video position:  390 /  436
03:53:51 : sending GA event
04:08:51 : sending GA event
(the GA event continues on for 3+ hours until I woke up to close the app)


I was outputting every second the video player updated the position. Out of no where we hit this "unknown event" (I spoke about this in the first post), then the video rebuffered, then the Roku timestamps started lagging.  We finally hit "sending GA event" - which occurs every 15 minutes.  What it seems like what happen: video was playing fine, something hiccuped, video component crashed, but event loop was able to still run.

And just to clarify, our CHIVETV channel is still currently running on the 2D API, so we're using the old video component.  We have a dev build for RSG, which I left running last night on firmware 7.7.0 build 4116 and there didn't seem to be any crashes.
 
jhoya
Posts: 6
Joined: Mon Dec 12, 2016 2:07 pm

Re: Firmware 7.7.0 Video Crashes

Mon Aug 07, 2017 6:07 pm

I have the exact same problem, same screen output on crash. Crashes after 10-60 minutes on Streaming Stick (s/n 5S*), not Express or Premiere.

Software: New RSG channel, tested successfully on 3600X stick in June and early July

Hardware: Firmware 7.7.0, Build 4116

Fails on model 3600X s/n
5S4671153046
5S4627160751
and more than 10 others

looks like this:

 Image
 
User avatar
RokuNB
Posts: 306
Joined: Fri Mar 31, 2017 2:22 pm

Re: Firmware 7.7.0 Video Crashes

Tue Aug 08, 2017 10:08 am

jhoya wrote:
I have the exact same problem, same screen output on crash. Crashes after 10-60 minutes on Streaming Stick (s/n 5S*), not Express or Premiere.

Email partnerSuccess@roku.com with details on your channel (name, ID), as this might be of help reproducing. Also helpful might be mentioning who is your video host and details on the video encoding.

Ditto @jamesonf re video encoding - if you can provide details or try tweaking it to see if particular settings avoid it.
 
User avatar
RokuNB
Posts: 306
Joined: Fri Mar 31, 2017 2:22 pm

Re: Firmware 7.7.0 Video Crashes

Thu Aug 17, 2017 1:36 pm

@jamesonf, @jhoya -
has the issue you experienced disappeared over the last few days or is it still happening in exactly the same way?
 
jamesonf
Topic Author
Posts: 4
Joined: Mon Jul 31, 2017 2:36 pm
Contact:

Re: Firmware 7.7.0 Video Crashes

Thu Aug 17, 2017 5:47 pm

@RokuNB,

The issue was never resolved. We have a few Rokus on the beta receiving the beta firmware updates as well.  We have been working hard at getting an RSG build running stable, and we just submitted it today to Roku.

As far as we can tell, Scene Graph hasn't crashed using the same api endpoints, videos, and codecs.
 
JordanT92
Posts: 8
Joined: Wed Jan 28, 2015 6:38 am

Re: Firmware 7.7.0 Video Crashes

Fri Aug 18, 2017 3:17 am

I'm having the same issue on a few boxes running software I'm developing. Happens at random and can be stopped by quickly pressing the back button closing the video node. If you leave it a few seconds it no longer responds and has to be restarted.

The issue has manifested on multiple Roku 2's, multiple Roku Streaming Sticks (3500EU) but not on any Roku Express boxes (imported to UK from US).

Only started happening in the last few weeks, coinciding with the latest update.
 
User avatar
destruk
Posts: 2509
Joined: Sat Dec 18, 2010 4:58 pm

Re: Firmware 7.7.0 Video Crashes

Fri Aug 18, 2017 8:57 am

For Roku3 you might want to test firmware 7.7.0 build 4117-04

I don't have a working roku2 anymore to test with.
 
User avatar
RokuNB
Posts: 306
Joined: Fri Mar 31, 2017 2:22 pm

Re: Firmware 7.7.0 Video Crashes

Fri Aug 18, 2017 10:50 am

jamesonf wrote:
The issue was never resolved. We have a few Rokus on the beta receiving the beta firmware updates as well.  We have been working hard at getting an RSG build running stable, and we just submitted it today to Roku.

As far as we can tell, Scene Graph hasn't crashed using the same api endpoints, videos, and codecs.

Yes, i noticed you have been pushing RSG build but it might not be the difference between SDK1 and RSG. In your initial report (8/4) you pointed it's happening with your RSG code too. I don't know of any change made here that would have resolved _only_ the RSG side.

But was a specific change we made on our side on Monday that may have fixed both cases. Can you very specifically ask around and see if the issue has happened since Tuesday in your production SDK1 app?
 
JordanT92
Posts: 8
Joined: Wed Jan 28, 2015 6:38 am

Re: Firmware 7.7.0 Video Crashes

Sat Aug 19, 2017 10:35 am

RokuNB wrote:
jamesonf wrote:
The issue was never resolved. We have a few Rokus on the beta receiving the beta firmware updates as well.  We have been working hard at getting an RSG build running stable, and we just submitted it today to Roku.

As far as we can tell, Scene Graph hasn't crashed using the same api endpoints, videos, and codecs.

Yes, i noticed you have been pushing RSG build but it might not be the difference between SDK1 and RSG. In your initial report (8/4) you pointed it's happening with your RSG code too. I don't know of any change made here that would have resolved _only_ the RSG side.

But was a specific change we made on our side on Monday that may have fixed both cases. Can you very specifically ask around and see if the issue has happened since Tuesday in your production SDK1 app?


If it helps, I've had the issue happen yesterday on a Roku 2 running the latest firmware, it happened about 20 minutes into a test MP4 I let run (h264 aac). This was with a RSG video node. I pressed back after a few seconds and it went back to the previous screen without issue, from there I was able to play the video again. Usually it would crash the entire box though and it would have to be restarted.

Who is online

Users browsing this forum: No registered users and 2 guests