Time drift between commercial detection and playback

OK, I've now done everything I can think of to confirm that the recording with the 30 second time drift is in fact a fully functional dvr-ms file.

I've used fixDVRMSduration and forced it to run, transcoding the file back into a dvr-ms and double checking the duration. It took a while, but the resultant file is the same 1:04:54.

I then played the file in Media Center with commercial skipping disabled and my stopwatch running. I came back at 1:04 to see exactly when it ended. According to my stopwatch, it played for slightly less than 1:04:54.

There are no glitches in the playback while playing it, yet Show Analyzer still thinks that it is 30 seconds longer than it really is.

I also have confirmed that there is indeed a drift that slowly grows throughout the TV show. The first commercial is more accurate and each commercial gets worse and worse until the very end with an error of 30 seconds.

This evening I'll try creating my own xml file and tweak all of the time stamps by the total drift error (3894/3924). If it is truly a drift, the result should be perfect commercial skipping. If the 30 second error grows at certain points within the show, I should be able to get an idea where within the show it develops. Also if this process works, you could probably implement a quick fix in your code if you wanted to.

Jere, I'd be happy to send this recording to you if you have somewhere I can send a file that is 5 GB in size. I do have a 20 GB monthly bandwidth cap, but I don't typically use it all.

One other thing to note: I checked a few other recordings from Discovery HD and found they all exhibit a large drift error. Recent shows recorded from Fox HD and Showcase HD on the other hand get analyzed without as much drift (1-2 seconds per previous posts).

[Jere_Jones]

Can you run SA on the file with the switch --framedetail? That will output another file with some, well, interesting data.

Jere

Originally Posted by Jere_Jones

Can you run SA on the file with the switch --framedetail? That will output another file with some, well, interesting data.

Jere

The frames file is enormous, but here is a small portion of it:

------------------------------------
File information
------------------------------------
File name: E:\recorded tv\Fire Monster Garage_TDCHD_14_04_2008_21_58_01.dvr-ms
File size: 5.02524853 GB
Image size: 1280x720
FPS: 59.94
------------------------------------
0000052 0:00:00.867 Not Blank 95.58 44.37 -99.00 0.00 0.00 0.00 0008/0711 0008/1098 00 I
0000053 0:00:00.884 Not Blank 95.31 43.74 -99.00 0.00 0.00 0.02 0008/0711 0008/1184 00 P
0000054 0:00:00.900 Not Blank 95.09 43.28 -99.00 0.00 0.00 0.01 0008/0711 0008/1227 00 P
0000055 0:00:00.917 Not Blank 94.76 42.89 -99.00 0.00 0.00 0.01 0008/0711 0008/1248 00 P
0000056 0:00:00.934 Not Blank 94.61 42.61 -99.00 0.00 0.00 0.01 0008/0711 0008/1259 00 P
0000057 0:00:00.950 Not Blank 94.30 42.33 -99.00 0.00 0.00 0.01 0008/0711 0008/1264 00 P
0000058 0:00:00.967 Not Blank 93.91 42.24 -99.00 0.00 0.00 0.01 0008/0711 0008/1267 00 P
0000059 0:00:00.984 Not Blank 93.46 42.11 -99.00 0.00 0.00 0.01 0008/0711 0008/1268 00 P
0000060 0:00:01.001 Not Blank 93.13 42.05 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000061 0:00:01.017 Not Blank 92.59 41.90 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000062 0:00:01.034 Not Blank 92.00 41.76 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000063 0:00:01.051 Not Blank 91.32 41.64 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000064 0:00:01.067 Not Blank 90.81 41.42 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000065 0:00:01.084 Not Blank 90.25 41.20 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000066 0:00:01.101 Not Blank 89.82 41.07 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000067 0:00:01.117 Not Blank 89.43 40.88 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000068 0:00:01.134 Not Blank 89.15 40.82 -99.00 0.00 0.00 0.01 0008/0711 0008/1269 00 P
0000069 0:00:01.151 Not Blank 88.85 40.77 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000070 0:00:01.167 Not Blank 88.68 40.79 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000071 0:00:01.184 Not Blank 88.48 40.71 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000072 0:00:01.201 Not Blank 88.34 40.62 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000073 0:00:01.217 Not Blank 88.19 40.50 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000074 0:00:01.234 Not Blank 88.02 40.42 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000075 0:00:01.251 Not Blank 87.90 40.31 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000076 0:00:01.267 Not Blank 87.82 40.24 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000077 0:00:01.284 Not Blank 87.50 40.30 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000078 0:00:01.301 Not Blank 87.43 40.23 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000079 0:00:01.317 Not Blank 87.29 40.13 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000080 0:00:01.334 Not Blank 87.11 40.13 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000081 0:00:01.351 Not Blank 86.98 40.09 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000082 0:00:01.368 Not Blank 86.91 40.11 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000083 0:00:01.384 Not Blank 86.80 40.12 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000084 0:00:01.401 Not Blank 86.88 40.09 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000085 0:00:01.418 Not Blank 86.59 40.21 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000086 0:00:01.434 Not Blank 86.59 40.22 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000087 0:00:01.451 Not Blank 86.50 40.22 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000088 0:00:01.468 Not Blank 86.37 40.25 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000089 0:00:01.484 Not Blank 86.30 40.17 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000090 0:00:01.501 Not Blank 86.28 40.12 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 B
0000091 0:00:01.518 Not Blank 86.11 40.15 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P
0000092 0:00:01.534 Not Blank 86.17 40.02 -99.00 0.00 0.00 0.01 0008/0711 0008/1270 00 P

To give you an idea of the amount of error and where it is within the show I have manually tweaked the commercial xml file to align it with the commercials during playback.

Here is the original xml file, as created by Show Analyzer:

<root>
<commercial start="867.950416667" end="964.513550000" />
<commercial start="1408.807400000" end="1502.284116667" />
<commercial start="1805.653850000" end="1913.227983333" />
<commercial start="2249.914333333" end="2347.261583333" />
<commercial start="2995.058733333" end="3133.663866667" />
</root>

Here is what I ended up with by manually tweaking until all of the commercials were (nearly) perfectly skipped:

<root>
<commercial start="855.3" end="951.7" />
<commercial start="1392.3" end="1485.1" />
<commercial start="1788.0" end="1894.1" />
<commercial start="2228.5" end="2324.8" />
<commercial start="2967.5" end="3106.7" />
</root>

Comparing the commercial locations:

Using the format: Commercial #, actual starting location (s), starting error (s), ending error (s)

1, 855.3, 12.6s start error, 12.8s end error
2, 1392.3, 16.5s start error, 17.2s end error
3, 1788.0, 17.7s start error, 19.1s end error
4, 2228.5, 21.4s start error, 22.5s end error
5, 2967.5, 27.6s start error, 27.0s end error (note that the end is also slightly off in School House)

For almost all of these, when using School House it was able to detect the correct commercial locations but for some reason there is a time shift/error.

As you can see, even the very first commercial has shift of approximately 12.7 seconds. By the end of the 1 hour show, commercial detection had drifted off by almost 28 seconds, an increase in "drift" by 15 seconds.

[Jere_Jones]

Can you email me the frames file?

Jere

Originally Posted by Jere_Jones

Can you email me the frames file?

Jere

I don't have your e-mail handy so I tried to reply with it as an attachment (I only just now figured out I can use attachments).

Unfortunately I keep getting a database error:
----Database error
The Dragon Global Forums database has encountered a problem.Please try the following:

• Load the page again by clicking the Refresh button in your web browser.
• Open the www.dragonglobal.org home page, then try to open another page.
• Click the Back button to try another link.

The www.dragonglobal.org forum technical staff have been notified of the error, though you may contact them if the problem persists.
We apologise for any inconvenience.----


I have e-mailed it to the contact link above (support@dragonglobal.org)?

[Jere_Jones]

jere at dragonglobal dot org

That always reaches me.

Jere

Originally Posted by Jere_Jones

jere at dragonglobal dot org

That always reaches me.

Jere

Jere, I sent the file to your ftp site, but for some reason your e-mail seems to be down (or hotmail is screwed up and can't send my e-mail to you).

Thanks again.

[Jere_Jones]

Since I've moved over to dragonglobal.biz, I haven't gotten the email setup yet.

I've got the file but I haven't found a solution yet.

Jere

Originally Posted by Jere_Jones

Since I've moved over to dragonglobal.biz, I haven't gotten the email setup yet.

I've got the file but I haven't found a solution yet.

Jere

Hi Jere,

Have you had a chance to look at the file and look for the problem?

I was reading through some of the other posts, and I too noticed a distinct decrease in detection accuracy some time after 0.93ish.

I'm convinced that the problem is something slightly changed in the mpeg-2 format used, or possibly an update released through Windows Update. I don't think 0.94, 0.95, or 0.96 is the actual problem (I actually reinstalled 0.91 and it now can't detect commercials very well with my content either - just like 0.96).


Also, in my case, the problem has become much, much worse over the past few months. In the beginning, once in a while commercials got messed up, but now almost every single recording fails to detect commercials without major errors in the detection. Digging into the problem, they all experience timing errors and drift.

Reading through the comments, I suspect the same issue the file I sent you has is causing much of the problems that other are experiencing.

Is there anything else I can send you to help get Show Analyzer working again?